How Many Reviews Are Too Many?

| Does the 3,250th review of Harry Potter and the Deathly Hallows have any influence on an Amazon customer? An Economist article says it does. In fact, says the article, the more online reviews a product has, the more likely people are to buy it. If reviewers know the reviews they write have influence, it may help answer Levitt’s earlier question: Why do so many people post reviews on Amazon? So what motivates blog readers, whose comments often number in the 100′s on this blog? One Freakonomics reader, Conor Lennon, recently emailed to say that he rarely bothers posting a comment unless he knows it will show up on the first page of comments (there are only 25 comments per page). He wonders if other commenters on this blog feel the same. [%comments]

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 62


  1. C. Larity says:

    FIRST!

    What was the question?

    Thumb up 0 Thumb down 0
  2. Quin says:

    Yes we do.

    Thumb up 0 Thumb down 0
  3. Ben says:

    Generally, I only comment if the topic is of real interest to me and if I feel I have something to add that could further the discussion.

    Being on the first page helps ensure that my opinion will become part of the dialogue, but I’m okay with the second. From the third page on, returns seem to diminish.

    Also: First page!

    Thumb up 0 Thumb down 0
  4. mkl says:

    I think 17th is the one that goes over the bar of too many.

    Thumb up 0 Thumb down 0
  5. Bucky says:

    I agree with Conor Lennon. At a certain point, nobody is going to read your comment, so it’s not worth the effort. The person who posts comment #387 in a thread (I usually stop reading after the first 20 or so) may really want to get something off their chest and have no other way to do it, but to me, it seems kinda pointless. You could have the absolute definitive comment on an issue, but once it’s buried that deep, you may as well just keep it to yourself . . .

    Thumb up 0 Thumb down 0
  6. Fred T. says:

    Oftentimes, I judge a product at Amazon based on how many reviews it has.

    For instance, if a product has 10 reviews and a 4.5 rating, I’m not as inclined to buy it compared to a competing product with 500 reviews and a 4.0 rating. And a product with 5,000 reviews and a 4.0 rating is likely going to get my attention more than the 500 review product.

    Basically, the more reviews there are, the more the product has been vetted and the more likely the rating (and comments) are reflective of the actual product and not just what the zealots have to say.

    Thumb up 0 Thumb down 0
  7. Lorenz says:

    well, if it shows up on the first page or now, the main incentive is often the discussion that involves around the comments imho

    Thumb up 0 Thumb down 0
  8. Michael Bishop says:

    The article did not say that the 3,250th review would increase sales. Even if their was an association between reviews and sales at these high numbers, the causation probably runs more the other direction. The article did find causality when the number of reviews is small.

    Thumb up 0 Thumb down 0
  9. Jesse says:

    It’s definitely no fun commenting when too many other people have. Unless there’s a competition going, there’s usually no reason to think anyone will read the comment, and it’s harder to track through all the comments to see if your point has already been made. I don’t have a strict limit, but usually if one or two pages of comments have already been made, I hold back unless I (think I) have a real nugget.

    Thumb up 0 Thumb down 0
  10. Jonathan Baird CFP(R) says:

    yep, my limit is about 20 or 30. Anymore than that and I don’t comment.

    Thumb up 0 Thumb down 0
  11. Michael S. says:

    I wasn’t going to post, but then I realized I could be on the first page.

    I agree with Connor though. I rarelyl post a comment on a blog unless I know it will be one of the first.

    Thumb up 0 Thumb down 0
  12. Jay Stevens says:

    …which is a long way of saying that some folks feel the “cost” of being buried in the comments is worth the “pay” of the ability to comment…the more prestigious or well-read the source (like, the NYTimes), the more likely commenters are willing to be buried…or something…

    Thumb up 0 Thumb down 0
  13. doug says:

    Yes we do, also.

    Thumb up 0 Thumb down 0
  14. Jonathon says:

    I find that most comment threads devolve into something petty within the first 100 comments. If there’s still anything interesting to say by that point, it’s usually drowned out by the crazy people who somehow end up personally offended by anyone else’s opinion.

    Thumb up 0 Thumb down 0
  15. Jonathon says:

    I find that most comment threads devolve into something petty within the first 100 comments. If there’s still anything interesting to say by that point, it’s usually drowned out by the crazy people who somehow end up personally offended by anyone else’s opinion.

    Thumb up 0 Thumb down 0
  16. Trevor L says:

    Yes–I only post if I can get on the first page. And even that hasn’t gotten me a book deal.

    Thumb up 0 Thumb down 0
  17. Carrie says:

    I comment whenever I am really interested in the topic- even if I’m the 80th. Although, I’m also the type who skims the comments until they hit 250 or so.

    Thumb up 0 Thumb down 0
  18. Peter T says:

    If I ever feel like giving up on reading the comments and just skipping to the end, I don’t bother because I know most other people will do the same. That is unless the author is asking for a direct response in the comments.

    Thumb up 0 Thumb down 0
  19. Jackie says:

    I only post if I have something relevant to say about the OP. If there are more than 3 pages on this blog however, I won’t post. Chances are someone else said something similar enough to my point.

    Thumb up 0 Thumb down 0
  20. BSK says:

    I do. Often after the first few posts, the conversation is at a point where it’s too hard or too late to add anything meaningful. Most people read the first few and then respond, unless engaged in a specific back and forth

    Thumb up 0 Thumb down 0
  21. mus says:

    i’ll have to agree with the premise, we all know reviewers are vastly different so the only thing i look at when buying online is the average number of stars and the number of reviews.

    Newegg.com has a slightly better system when they show how many 5 stars, 4 stars, 3 stars etc.

    More reviews means the stronger the consensus and i generally like what most people like.

    Thumb up 1 Thumb down 0
  22. Dean Rodrigues says:

    I press page down 3 times. Any comments that fall within the remit are read, anything else gets missed. On that basis, I does not look like I will be reading this particular comment.

    Thumb up 0 Thumb down 0
  23. David says:

    Isn’t this testable? Can you fiddle with the number of comments shown on the first page and see if the average number of comments goes up or down?

    Thumb up 0 Thumb down 0
  24. Jesse says:

    Prefer to be in top 25? Yes. My ideas are likely to be read if they’re on the first page. Very few people read the comments on page #8 of 15.

    This is particularly notable on other NYT pages where readers can “Recommend” posts they believe have merit. A mediocre post on the first few pages may have hundreds who like it, while a superior post in the last few pages has at best a handful.

    Note: At time of writing 20 comments are visible. I assume this will end up on page 2-3.

    Conditions under which I’ll post anyway include:
    – Sharing my perspective where it seems valuable
    – Presenting new ideas or a calm summary statement
    – Hoping to shape future comment discussion
    – Having a strong personal reaction to the issue
    – Just wanting to reply to an interesting topic

    Thumb up 0 Thumb down 0
  25. Mike Kriskey says:

    I refuse to comment unless I’ll end up on the first page.

    Thumb up 0 Thumb down 0
  26. Andrew says:

    There is an excellent book by Jonah Lehrer, “How We Decide,” and it shows that usually 4 items are enough to make a reasonable decision, and that the more there are then that, the more confused we are, and humans have an even harder time making a decision.

    Thumb up 0 Thumb down 0
  27. Mark W says:

    I make a comment when I have something to say.

    Even when it is well down the list, I feel better when I have “had my say” even if I am pretty sure noone cares or reads it, at least I got it off my chest.

    Thumb up 0 Thumb down 0
  28. J Chang says:

    Makes sense – if something has a rating with many reviews then it’s helpful to know what a larger sample of people are saying.

    As for posting – will try to be in the early comments for sure. It’s too much effort to avoid duplication of old comments or sift through hundreds of responses.

    Thumb up 0 Thumb down 0
  29. Notorious GDP says:

    # of reviews is probably a proxy for popularity. It makes sense that sales increase when the reviews increase. Inferring causation from this is a lot harder. An opportunity for an IV approach?

    Thumb up 0 Thumb down 0
  30. Ylan says:

    I rarely post… because I don’t read any off the comments. The blog comment format is not conducive to conversation… just ranting.

    If you want a good online conversation format, look to usenet interface, like tunderbird, but certainly not Google Groups.

    Rant over

    Thumb up 0 Thumb down 0
  31. Liz says:

    Who moderates the comments? Even if yours is several pages in, you can rest assured that at least that one person has read it.

    Thumb up 0 Thumb down 0
  32. Caliphilosopher says:

    I make comments here because I’m interested in how actual economists deal (if they ever do) with philosophical problems found in economics. It all started with a paper posted on this blog that hopefully will NEVER get published, due to the fact that it is poorly written (as far as the conclusions that the authors try to draw).

    Thumb up 0 Thumb down 0
  33. Uthor says:

    Personally, I used to only post if I had something I felt I could contribute compared to other posters. For example, I would comment if there was a common misprint in a book to make others aware of it.

    In the last couple years, I’ve signed up for databases/catalogs for my books, music, and comics. All the sites I use have spots for reviews. I’ve also decided I need to practice my writing a bit more as all I’ve ever written was work related emails. So, I’ve been trying to do a review or two every month on something. Since it’s already written, I will just go ahead and copy/paste into Amazon.

    Thumb up 0 Thumb down 0
  34. Andrew says:

    Unless we’re talking about when the first book was released, I can’t imagine that any cares about the review of a Harry Potter book.

    The equation is simple.
    Harry Potter = Sales

    Thumb up 0 Thumb down 0
  35. Lord says:

    My limit is around 50 and then only if particularly relevant. Anything with over 100 and I won’t even bother to read them.

    Thumb up 0 Thumb down 0
  36. Larry Underwood says:

    As a first time author of a decent book, I look every day on that Amazon site, hoping at least one more person took the time to write something. After nearly three months, 12 people have reviewed it, and most have been great as far as the rating goes, but I agree that you’ve got to have a lot of volume to be considered a viable entity.

    Realizing that some people get offended when someone shamelessly self-promotes their book, but not really worrying about that too much, because I’ve got bills to pay; please go check out “Life Under the Corporate Microscope”, my irreverent perspective of life in Corporate America, with a company called Enterprise Rent-a-Car.

    Thumb up 0 Thumb down 0
  37. and by says:

    I just like to lip off to the bloggers. I can’t believe you take this from total strangers.

    Thumb up 0 Thumb down 0
  38. Heavy D says:

    My system is to only read the entries which are prime numbers. (Can you please bump me to 41?)

    Thumb up 0 Thumb down 0
  39. Joe Smith says:

    If there are already 50 comments someone else has usually said what I would say and probably better than I could.

    There is an information theory aspect of this. The collective information content of the comments goes up as the log of the number of comments ( in other words, the incremental contribution by a new comment is inversely proportional to how many comments have preceded it ).

    Thumb up 0 Thumb down 0
  40. Catchwa says:

    Huh?

    “In fact, writes Mellor, the more online reviews a product has, the more likely people are to buy it”

    isn’t this just another correlation vs. causality thing?

    Surely the more people that buy a product, the more customers there are to write online reviews.

    Thumb up 0 Thumb down 0
  41. Shrikant says:

    (Didn’t bother to read thru the earlier comments, so might be repeating a point here)

    Obviously the more reviews a product has, the more likely people are to buy it – the devil you know and all that.

    I know for a fact that I base my online buying decisions (also) on the number of reviews about a product. it just shows that people feel strongly about it one way or another to expend time on talking about it..

    Thumb up 0 Thumb down 0
  42. Liz says:

    What would happen if the order of comments on a page was determined by merit, by the cumulative opinion of comment readers, sort of like how Google search results can be moved up or down the page according to searcher preferences? The good ones might be less likely to get lost in the piles that way.

    Thumb up 0 Thumb down 0
  43. ethicalBob says:

    I always just skip to the last couple of reviews to see who the culprit was! :-)

    Thumb up 0 Thumb down 0
  44. Science Minded says:

    Dear Calli;

    Yes, tis true– We often begin with false concepts only to then sort out the truth.

    Thumb up 0 Thumb down 0
  45. Srinivasan says:

    It seems to me that web pages could help late-commenters by randomizing the sequence of comments.

    In particular, choose the comments that appear on the first page at random.

    Would this increase the effectiveness of the comments overall? Would this make people more likely to comment in general?

    Thumb up 0 Thumb down 0
  46. Marc Resnick says:

    I would be interested to know how many comments to your own posts that you read. Sometimes, my comments are directed to the general reading audience, but I suspect the majority are really mostly for the author of the post. If you tell us that you read the first 50 comments to your posts, then I would comment up to that point.

    Thumb up 0 Thumb down 0
  47. Justin says:

    Newegg.com has a feature that allows you to sort product categories by most reviews. I, and I’m sure many others, immediately sort using this criteria when shopping for parts that we aren’t familiar with.

    Thumb up 0 Thumb down 0
  48. pat toche says:

    funny that, I was posting a review on amazon just one minute ago.

    I did not, however, ask myself why. Did I do the right thing?

    Thumb up 0 Thumb down 0
  49. HM says:

    yes, at #49, I only post if I am part of the first 25.

    Thumb up 0 Thumb down 0
  50. Caliphilosopher says:

    Science Minded -

    It’s good to know that there are economists who recognize that there might be problems with methodology/initial assumptions or presuppositions.

    Sometimes, I feel as if these worries are lost on this blog.

    Thumb up 0 Thumb down 0
  51. Michelle says:

    I review on Amazon for the same reason that I post blog comments: A vain desire to shout about me, me, me! into the void.

    Thumb up 0 Thumb down 0
  52. kreg says:

    I’m the type who’ll read every comment if the topic is interesting, the idea of only posting if you know you’ll be on the front page seems more to do with getting attention and recognition than with furthering the discussion at hand, if you have a point that is valid does it become *less* valid because it’s buried on page three or four?

    Thumb up 0 Thumb down 0
  53. Fred H Schlegel says:

    Really depends on the type of blog. Kreg above indicates the quality of comments count. I would agree with that… I seldom comment if I don’t feel like reading through what was said earlier in the thread.

    Thumb up 0 Thumb down 0
  54. Jeremy says:

    When I want to comment, it is usually based on the content of the post, rather than other people’s comments. I rarely read other people’s comments (I barely have enough time to keep up with all the posts).

    So I just hit Ctrl-End and comment away.

    Thumb up 0 Thumb down 0
  55. David Chowes, New York City says:

    People generally like to see their comments in print[?]. E.g.,
    this post.

    Thumb up 0 Thumb down 0
  56. Ari says:

    I think people are encouraged to write even more reviews if they see previous visitors to a site have done the same. At http://www.MrMovietimes.com you can see a waterfall effect of more reviews being written after each and every visit.

    Thumb up 0 Thumb down 0
  57. Kaveman Koder says:

    If I can’t be on the front page, I’ll opt for being one of the last posters. People will often click the last page link to see where the comments lead.

    Thumb up 0 Thumb down 0
  58. kip says:

    Regarding comments:

    If I want *people* to read it, I will only post if it looks like I am going to be on the first page.

    If I want a particular person to read it, I might post starting with “@dave” syntax, if there haven’t been that many posts since the one I’m responding to.

    If I want the blogger to see it (as in this case), I will post no matter how many pages there are. (I operate under the assumption that most bloggers read all of their own comments.)

    Thumb up 0 Thumb down 0
  59. Bobby G says:

    I would post a reply here but who checks the 54th comment?

    Thumb up 0 Thumb down 0
  60. achilles3 says:

    depends on the subject…

    Thumb up 0 Thumb down 0
  61. Karen says:

    The probability I will post on a given topic or read through all the posts has very little to do with the number of posts already up there and a lot more to do with how bored I am at work that day.

    Thumb up 0 Thumb down 0
  62. Travis says:

    Firstly, let me say that I have been following this blog for nearly 18 months and this is only my second post, even though there have been MANY topics where I have had a strong opinion that I would have LOVED to voice.
    Given that I’ve only posted once previously, here are the reasons I choose NOT to post.
    I believe that the main motivation to participate in any discussion is because you will gain something from it, either recognition or learning. Unfortunately, on the Freakonomics blog, I don’t feel there is much opportunity to gain either.

    Recognition
    There are so many Freakonomics POSTS, I have a hard time keeping up and reading them all. If this is the case, I’m fairly certain that the blog authors don’t read the comments. I don’t even know if anyone involved in the blog reads all the comments (which is why I’ll also email this comment to the editor.) Also, there is no way to subscribe to comments for a specific post to “stay in the discussion”. You have to continually go back to the blog to read new comments.
    While I’m sure there are many astute blog readers, I’m not certain I would value recognition from a reader. I doubt anyone would care if I told them that their comment was exceptional.

    Learning
    The comments aren’t moderated, other than for offensive content. Consequently, there are many comments that aren’t worthwhile.
    Also, it takes too much time to sift through them all. So, if there is a good comment from which you learn, it may not be worth the time it took to find it.

    At the end of the day, it feels like, when you enter a comment, it just goes into oblivion because you don’t get much/any worthwhile feedback. Even having your comment on the first page, Connor Lennon’s goal, doesn’t necessarily mean much, other than you submitted your comment quickly. I think that, if the Freakonomics blog authors/editors want more and better comments, you have to be more selective about the comments that get published. Why do athletes want a gold medal? Because they are so difficult to win. Filtering comments may not lead to a larger quantity of submissions, but it would probably lead to better discussions.

    By the way, I don’t think comments are similar to product reviews on Amazon.

    Thumb up 0 Thumb down 0