When products in the United States are given a numeric rating, most ranking systems use a “bigger-is-better” method in which a higher score reflects better quality. According to a new report co-authored by Ellie Kyung, a professor of business administration at the Tuck School of Business, when this method is used, scores are likely to affect U.S. consumer buying patterns because bigger-is-better rating systems are the cultural norm. Yet if a product is rated by a “smaller-is-better” structure, consumers are less likely to perceive changes in scores.
The report, titled “When Bigger is Better (and When it is Not): Implicit Bias in Numeric Judgments,” tested whether the rating of products on a scale opposite of the normative would impact consumers’ attitudes toward those products.
“One of the things we wanted to test was whether [different rating systems] could affect people’s willingness to pay for actual objects,” Kyung said.
Research for the study consisted of several experiments involving consumers’ attitudes toward products based on a given quality rating. In each test, half of the participants were shown a product with a rating on a bigger-is-better scale, while the other half were shown the same product with an equivalent rating translated into a smaller-is-better scale.
Some of the experiments for the study were conducted using Dartmouth undergraduate students as test subjects. In one of the experiments, 72 students were instructed to make auction-style bids on a stainless-steel coffee mug. Half of the students had been told the mug scored 6.1 on a scale where 1 is unsatisfactory and 7 is very good. The other half had been told the mug scored 1.9 on a scale where 1 is very good and 7 is unsatisfactory.
The experiment found that the students who were told the bigger-is-better rating were willing to pay on average over 1.5 times more for the mug than the students who were given the same rating on a smaller-is-better scale.
In the report, the authors find that the results of this and other similar experiments are explained by a cognitive bias of consumers toward the rating system with which they are most familiar.
“If you’re using kind of a rating scale where people have to make some kind of translation, it essentially wipes out the difference in perceived quality,” Kyung said.
As a result of the study, Kyung said that companies might want to consider adjusting their rating scales based on the rating system familiar to their target audience.
Associate dean of the faculty and Tuck professor Richard Sansing said that the report’s conclusions align with a number of psychological tests measuring human response patterns.
“There’s this whole literature on cognitive interference and how that affects the judgments that people make,” Sansing said.
As an analogy, Sansing described a study in which individuals are told to hit a buzzer when they see the word “green.” But when some individuals are shown the word “green” colored in red letters, they are thrown for a loop, he said, which affects their ability to respond.
“I wouldn’t be surprised if that kind of bias or interference carries over into when you’re making an actual purchasing decision,” Sansing said.
Kyung co-authored the report with Aradhna Krishna, a marketing professor at the Ross School of Business at the University of Michigan, and Manoj Thomas, a marketing professor at the Johnson Graduate School of Management at Cornell University.
Thomas said that although the results of the study may be surprising to readers, they were consistent with the authors’ expectations. He said he enjoyed the “brilliant experience” of discussing and debating the topic with Kyung and Krishna.
“We had a lot of fun doing that, and I think we are all kind of proud of the outcome,” Thomas said.
The report was published on the website of the Journal of Consumer Research in December 2016 and is expected to be published in the journal’s print edition in June.
