When "Wisdom of the Crowd" Goes Awry? Investigating the Effectiveness of the Aggregated Helpful Vote on Online Product Reviews

Project: Research

View graph of relations


Helpful vote is a common feature on many websites that utilizes the “wisdom of the crowd”. It allows visitors to vote on whether a piece of information (e.g. a product review) posted on the website is helpful to them. The aggregated helpful vote is supposed to reflect the underlying quality of the information. Given a plethora of information available on the Internet, helpful vote lowers people’s search costs to locate the most valuable information.The effectiveness of helpful vote is based on the belief of “wisdom of the crowd.” Although the “wisdom of the crowd” effect has been supported in both academic research and popular press, recent studies show that under certain conditions, aggregated judgment may lead to inaccurate information. For example, Lorenz et al. (2011) show that social influence may undermine the “wisdom of the crowd” effect. Further, inspired by Arrow’s impossibility theorem, List and Pettit (2002) show that voting based on multiple criteria may result in a set of logically inconsistent aggregated judgments and thus, also undermine the “wisdom of the crowd”.Motivated by these studies, this project examines the effectiveness of helpful vote on product review websites. Specifically, we intend to address three questions:whether the aggregated helpful vote reflects the underlying quality of a review,under what conditions the aggregated helpful vote reflects the underlying quality of a review, andhow we can design better aggregation mechanisms to help people locate truly helpful reviews.We argue that the aggregated helpful votes will not reflect the underlying quality of a review because ofsocial influence (i.e., the existing helpful vote affects future helpful vote),multiple criteria for judging helpfulness and the subjectivity of the judgment, andnon-random information selectivity (i.e., consumers select reviews based on existing helpful vote and other cues provided by the website).We test our hypotheses and develop new aggregation mechanisms using a multi-method approach that combines verbal protocol analysis, computational modeling, and archival data analysis. Specifically, we first investigate how individual consumers cast helpful vote via verbal protocol analysis in an experimental setting. Based on this understanding, we develop computational models to simulate reviews, consumers, and their helpful votes. We then validate and refine the models by comparing the simulated helpful vote with real-world data collected longitudinally from Amazon.com. Finally, we develop and test different aggregation mechanisms that can accurately capture “wisdom of the crowd.”


Project number9041830
Grant typeGRF
Effective start/end date1/01/1313/06/16