In the stars
Coarse star ratings and other online user feedback aren’t that useful
Feedback. Who needs it? Or, more precisely, who needs it in its modern form, delivered digitally, shared globally, ossified for all time and crystallised by coarse star ratings?
The more informal, diffuse and indirect kind has always been there. It’s called the marketplace. If they’re queuing in the rain outside the establishment next door while your place is empty, that’s feedback. If volume share starts slipping, you know you have to improve at least one aspect of the product/price mix.
The market itself was a language and behind it were the literal voices of people talking to one another, occasionally sharing good or bad experiences of products and services they’d just tried. Word of mouth was the invisible, unmonitored network of millions of tiny interactions – the nervous system that modulated the market’s expressiveness. It could be perceived but it couldn’t be chased.
Today’s marketers actively chase something they call EWOM, seeking to keep it ever-more positive. But the resonance in nomenclature is misleading, since the differences between this and classical word of mouth – let’s call it CWOM – are legion and lead to opposite outcomes.
With CWOM, the dialogue was face-to-face.
The recipient, therefore, had a means of assessing its validity: they knew if the person had a vested interest or whether their pronouncements were generally best taken with a pinch of salt.
The anonymity afforded by EWOM – with artificial usernames rife – is its notorious Achilles heel. There is no way to know if reviews and ratings are motivated by commercial interest. And no way of assessing whether a post would ever be taken seriously if you could just get a glimpse of the person behind it.
Moreover, CWOM was time-sensitive. The fragments would be shared informally and disappear into the ether. A comment in the pub, a don’t-touchthat-brand-with-a-bargepole in the canteen, an intimate, shared recommendation between two friends. There and gone. If opinions subsequently changed, so did the timbre of the conversation.
Conversely, EWOM preserves stale hates or loves forever, digitally and ludicrously ossified long after their purpose has been served – and, if hostile, a permanent blemish on the target brand, whatever improvements it might since have made.
Most significant of all, marketers couldn’t eavesdrop on CWOM. That meant they couldn’t seek to inf luence it directly but, instead, had to do the much more helpful thing of keeping standards high in order to keep it positive.
The modern practice of hounding, cajoling or bribing customers to give positive digital feedback achieves the opposite effect by taking management eyes off the substantive product or service.
It could be argued, of course, that none of this really matters if digital user feedback is providing consumers with more balanced information and helping them reach better consumption decisions. The trouble is, the opposite is true.
In a 2016 study from the University of Colorado (see panel), the team isolated average user ratings for 1,272 products in 120 categories and compared them with evidence-based quality scores from Consumer Reports (the US equivalent of Which?).
What they discovered was astounding: the two are virtually uncorrelated. More crucially, average user ratings are not predictive of resale value (where relevant) – whereas Consumer Reports scores are. The tendency of consumers to “put enormous weight on the average user rating” is costing them money.
So who needs feedback? The feedback industry does – Tripadvisor, Yelp, Angie’s List and all the others that turn consumer opinion into corporate profit. Taken as a whole, I give this bunch a two-star score. Here’s what they need to do to improve on that.
1 Warn on sample size. The US study noted that people are blind to the unreliability of small samples. The industry could use lighter-weight typography, or weaker icons, for average user ratings derived from a non-significant sample.
2 Banish anonymity. Or, at the very least, get people to declare that they have no commercial interest before letting them hold forth.
3 Cull ancient ratings and reviews.
That might get you guys up to a “three” rating. Still well short of the five-star splendour of the language of the marketplace and classical word of mouth.