The wisdom of reluctant crowds

Estimating is difficult. This is true whether the task requires forecasting uncertain future events, or whether the estimation task is complex in itself and based on insufficient information. As a result, even perceived experts frequently estimate poorly. Surprisingly, recent research suggests that groups of non-experts can outperform individual experts, given certain conditions are met. The resulting capability has been described as collective intelligence or "wisdom of crowds". Yet crowds (and individuals) do not necessarily like to make guesses, whether because it is cognitively hard, or emotionally undesirable. If crowd members prefer not to estimate, but instead seek to transfer this responsibility to others, are they able to identify good surrogates? We empirically tested these two aspects of collective intelligence. First, we explored whether collective intelligence was able to produce estimates that are significantly better than those of individuals, and second whether perceived experts as surrogate estimators were able to perform the task equally well. Our findings demonstrate good estimation ability for the crowd as well as its surrogates. We discuss implications for scenarios where estimates involve both beliefs and preferences, and where collective estimates thus have to be negotiated. Resulting requirements for information systems are outlined. © 2010 IEEE.