06. Wisdom of Crowds

James Surowiecki

The Wisdom of Crowds

Collaboration as a double-edged sword... ways to manage it re the intelligence that groups have.

'Collective wisdom' taken from average of group's guesses at things like 'guess the weight' in fairgrounds.

Cites example of Google with its page rank algorithm: a 'vote' by the internet at large re that page's value.

Prediction Markets

cf. racetrack odds: these are defined by the crowd's bets, and are pretty accurate

HP used an internal 'shares' market for a few years, and this was more accurate than their own internal sales forecasts.

Siemens using the model for software project forecasts

Elily (arm of Eli Lily) set up a market for checking six new drugs going through clinical trials

Implementation

Bottom-up decision making runs against base assumptions re power, intelligence, how corps. should work, that decisions shouldn't be made by large groups (cf. 'mass hysteria' or stock market crashes. A fear exists that groupd could also be watered-down -- real concerns and issues).

For groups to be intelligent:
1) Diversity (to avoid blandness)
2) Independence of individuals (for true opinions)
3) Make aggregation of inteligence possible

On (1) hence need for staff churn: new people 'know less' but have different ideas & opinions.

On (2) human nature is imitative: a good way to figure out what's going on. BUT, cites example of ants & how the crowd works intelligently by following those around. Humans are not ants ;o) Danger = few people exert too much influence over a group as whole = loss of (1) & (2) & therefore effectiveness.

Information cascades -- bad thing. You want individuals in the group to draw on their own information

  • Talkativeness doesn't equate with intelligence
  • Dominance of opinion etc. doesn't work
  • The more random information sources are, the better
  • Talking with like-minded people is less productive (cf. weblogs etc.)
  • -- Diversity is key

    Questions

  • Is there an optimal size & make-up for groups? Don't necessarily require management. Loose ties and independence can be very useful. Done right, even works in small groups (c. 10 people). Diverse groups appear to out-perform single-sex groups. Age and geography also must-haves.
  • 'Astro-turfing' & PR: how to deal with these? Internet offers this plus the solution (diversity), which is interesting.
  • Is there a difference between predictions and decisions? Not necessarily so different. Decision is implicit in prediction
  • Summary points

    Surely it can't be totally random: no good not having
  • any* experts there -- *some* knowledge is useful definitely. But diversity = a cancelling effect on experts' errors.
  • On competition: pulls both ways re enforcing independence.