Collective Intelligence Expressed through Activity in Social Media: A Model and Empirical Evaluation

Description

The proposed research seeks to understand and evaluate the ability of collectives of non-experts to predict the outcome of uncertain events or to answer difficult questions, exhibited through their aggregate actions in social media sites (i.e., websites on the Internet such as Google, Wikipedia, or Twitter). Making judgments about unknowns is difficult. Individuals, non-experts and experts alike, are not particularly good at it, and even group performance does not provide expected synergies and performance increases. Surprisingly, collectives of non-experts seem to perform judgment tasks well, accurately estimating unknowns or forecasting uncertain events. Collectives outperform experts on questions such as estimating movie box office successes or failures (e.g., predictions on the Hollywood Stock Exchange), outcomes of sports events (Yahoo! Pickem) or political elections, commodity prices (gold, oil), as well as new product delays (Boeing 787). Even more interestingly, this appears to be the case not only if collectives offer their judgments directly in answer to questions, but also, when their activity, such as information search in social media is observed. The proposed research seeks to investigate the ability of collectives to forecast and make judgments based on their activity in social media.Three objectives will be targeted. First, is this is a phenomenon that can be modeled and validated? Second, what are the critical assumptions, which determine success or failure of collective intelligence, i.e., the multiplicity of shared, previously private insights and the role of diversity (or independence among collective members)? Third, what are the individual estimation methods underlying collective intelligence and how does aggregation impact the overall outcome? The search for answers to these three questions defines the three objectives of this project. The project will address these three objectives with a three-phase study, whereby collective intelligence performance will be evaluated through three sets of experiments. The study is expected to find that social media activity driven by the interest to gather knowledge or generate other value will be a good predictor of unknowns. The study is also expected to challenge existing assumptions concerning collective intelligence, by rebalancing the views on the importance of a multiplicity of insights versus diversity within the collective. In doing so it will inform the ongoing discussion on the relative performance of individuals, nominal groups, and real groups. Finally, the study promises new insights into individual prediction and judgment methods, the error correcting characteristics of collaboration, and the meaning of expertise.The study is perceived as of high value, given the ongoing research interest in human judgment and decision-making, the attempts to support this difficult activity with better information systems, and the constant need of organizational practice to make decisions under uncertainty and with insufficient information. Applications for Hong Kong abound, including for instance better prediction of property markets, or early warning of impending diseases such as H1N1.