Can You Trust Peer Reviews?

First, let me say that I am a big fan of peer reviews. I use them to learn about the experiences people have with products I an interested in, before I buy. I also recommend them to my client companies, that is, if they are going to set them up and manage them in a “Social” media manner. Companies like BazaarVoice do a great job for setting up authentic review programs for their clients, programs that benefit both the business and the customers. On their site they provide statistics on the efficacy of such program. Have a look at the case study on Petco.com, one of their sterling example (Under Learning and industry stats).

Second, in this blog post I am focused on the policies and procedures of the review sites rather than the peer reviews per se.

Here’s the issue. Peer reviews work when they authentically capture the opinions of other people and accurately represent true range of reviews posted about a product or company. But now we are discovering that some peer review sites are manipulating the reviews so they can garner more advertising review from sponsors.

The peer review site in the headlight at the moment is Yelp. They provide local peer reviews of restaurant, nail salons and other local businesses. To date they are in 24 cities and get about 20 million readers per month. A lot of people rely on them.

Peer reviews are posted whether a business has an account on Yelp or not. Any business can sign up for a free account to enhance their presence. The free account let’s them list special offers, post pictures and send messages to customers. They can even elect to feature a review on their “ad.” But, they can’t change or reorder or delete any of the reviews from the main site. Yelp.com makes money by selling upgraded features to small businesses.

Over the past few month we are learning that while businesses can’t delete or reorder reviews (a good thing), Yelp.com can and does. Business owners are complaining that if they decline to upgrade their account when approached by Yelp sales, positive 5-star reviews start disappearing from their review list and negative reviews get pushed closer to the top. Yelp’s official response is that old reviews get removed automatically but that doesn’t explain why a two month old positive review disappears and an eight month old negative reviews remains.

Some businesses have been systematically documenting things and claiming extortion. Including cases where Yelp’s salespeople have promise to remove some negative reviews and push some positive reviews to the top of the list. Law suits are in the works

To the extent that this is happening, Yelp is creating a systematic bias in the reviews customers see. Businesses that pay for advertising get better reviews posted. Businesses that don’t pay get a more negative slant.

How are you to know if there is some sort of one-sided agenda at work? Transparency. Sites that want to be believed should post their policies and procedures clearly and visibly on their site. If a site doesn’t, people should either ignore them or take their reviews with a bigger grain of salt than normal.