Does Reviewer Quality Matter?

Photo: iStockphoto

You can buy almost anything online these days — hotel reservations, books, movies, etc. — but how much does reviewer quality matter to online shoppers?  A lot, according to research from Anindya Ghose and Panagiotis G. Ipeirotis.  In a previous paper, the pair noticed that “demand for a hotel increases if the online reviews on TripAdvisor and Travelocity are well-written, without spelling errors; this holds no matter if the review is positive or negative.”  In a more recent paper, Ghose and Ipeirotis find similar trends for products on Amazon.com.

Even more fascinating, Ipeirotis writes on his blog that:

An online retailer noticed that indeed products with high-quality reviews are selling well. So, they decided to take action. They used Amazon Mechanical Turk to improve the quality of its reviews. Using the Find-Fix-Verify pattern, they used Mechanical Turk to examine a few millions of product reviews. (Here are the archived versions of the HITs: FindFix,Verify… and if you have not figured out the firm name by now, it is Zappos :-) ) For the reviews with mistakes, they fixed the spelling and grammar errors! Thus they effectively improved the quality of the reviews on their website. And, correspondingly, they improved the demand for their products!
While I do not know the exact revenue improvement, I was told that it was substantial.

(HT: Marginal Revolution)

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 10

View All Comments »
  1. Phil says:

    Are there any theories for why better spelling and grammar improve sales, even for bad reviews?

    Thumb up 0 Thumb down 0
  2. Snidely says:

    Wow, that opens up a lot of potential for abuse. While fixing the “errors” we also corrected a bunch of grammatical mistakes like when a reviewer mistakenly used the word “sucks” when they clearly meant “rocks!”. Not sure this should be allowed…

    Thumb up 2 Thumb down 0
  3. Eric M. Jones says:

    Anybody who has a writer-friend has been begged to write a really great review for him/her. So reviews need to be discounted one or two notches.

    Thumb up 0 Thumb down 0
  4. Chris says:

    I bet the reason there is a relationship between good writing and purchasing is an implied class relationship.
    We (at least I, with and n=1) infer things about the economic status of a reviewer based on their writing style. Good writing= educated, wealthy, higher class, while bad writing =poor uneducated, and lower class. Everyone wants to be like the upper classes.

    Thumb up 0 Thumb down 0
    • Adam says:

      But the boost to sales happened whether the review was positive or negative. If you perceived someone as higher class and they hated the product, wouldn’t that be a disincentive to buying the product?

      Thumb up 0 Thumb down 0
      • Chris says:

        No necessarily. Depending on the content of the review, I might read that as “oh, that rich jerk didn’t like it, because they ______, but I in my infinite wisdom know how to solve that problem.” Whereas a review that sounds like it was written by a poor person, positive or negative, might make me not want to be associated with the product in any way, because it associates the product with being uneducated. Look at Dubner and Levitts paper (and post, somewhere) on baby names for an example. Poor white folks copy rich white folks baby names, and when a name popular among rich folks starts to become popular among the poor, its rich girl popularity falls.

        Thumb up 0 Thumb down 0
  5. Caitlin says:

    I find these research findings very interesting. It boggles my mind how a simple change of grammatical errors can increase the demand of products on the site. When looking at a demand schedule, the demand curves shifts up and to the right when grammatical errors are fixed, meaning that the quantity demanded will increase at every price. Consumers like to see that reviewers have spent time and effort to write an articulate response, instead of a run-on sentence like “it aint got any diffarant colors in the high heals.” I think that in the mind of consumers, a grammatically correct review reflects the worth of the good or service. If a poorly written review, good or bad, is on the website, consumers will look at it and think that the particular good is not worth their money. With the editing of reviews on sites like Amazon.com and Zappos.com, the demand for their products has jumped substantially. If all websites that sell goods were to edit all the reviews, would the demand increase? Or would the editing just become an afterthought in the mind of consumers because every website does it? I guess only time will tell to see if other companies catch on to this easy way to increase the demand in their products.

    Thumb up 0 Thumb down 0
  6. Joe Dokes says:

    I am not surprised, I find that both good and bad reviews are helpful in making a decision about whether to buy a product or not. A well thought out review is especially helpful. Thus, multiple spelling and or punctuation errors detract from the review and by extension the product itself.

    I’ve made it a point not to buy products that have NO reviews. I don’t like being a guinea pig. So even a product with mixed reviews is more likely to get a purchase from me, than a product with no reviews.

    I would take this further, when I read blogs and news sites that have a comments section, if the comments are poorly written or inflammatory it detracts from the news site regardless of the merits of the original article. I’ve quit reading some news sights simply because of the negative comments. I’ve also made it a point not to read the comments at some sights simply because the quality of the discussion is so low, as to detract from an otherwise good site (I’m speaking of you NYT).

    Thumb up 0 Thumb down 0
  7. Surprised. says:

    So they changed people’s reviews? That sounds profoundly unethical, dishonest, and suspect, even if they only “innocently” fixed grammar and spelling errors. I wouldn’t want to post reviews on any site that was going to edit or “correct” them.

    Thumb up 1 Thumb down 0
  8. Bruce C. says:

    One reason I can see for even negative reviews enhancing sales due to their quality is because typically a quality negative review is more informative about why a particular product sucks. It may be that the reviewer’s issues with a product won’t apply to other consumers, so they have enough information to discount the negative review. A review that just says “a cheap piece of junk” or worse with misspellings, would imply generally poor workmanship of the product and be more of a discouragement to buy.

    For reviews where the information content is basically the same (as, presumably, with before and after the Zappos edits), I would point to the class relationship others have mentioned. Well-written reviews imply that the product is of interest to educated consumers, and I would bet that “uneducated” consumers are less likely to read the reviews in the first place. So well-written reviews imply to the consumers who read reviews that the product was interesting to their peers, which will at least increase the hit rate on the number of people who view the product details.

    Thumb up 0 Thumb down 0