What's Next: The Idiocy of Crowds

Collaboration is the hottest buzzword in business today. Too bad it doesn't work.

From: Inc. Magazine, September 2006 | By: David H. Freedman

In June, NASA chief Michael Griffin ordered the space shuttle to launch in early July as planned, overruling the agency's chief engineer and top safety officer--both of whom argued that the shuttle was still vulnerable to damage. A storm of criticism ensued, much of it focused on the charge that Griffin, an experienced aerospace engineer who holds six advanced degrees, was placing timetables ahead of safety. But I think the protests were rooted in something else: the fact that Griffin made the decision on his own, casting aside the consensus of other expert decision makers.

Imagine if it had been Griffin alone who believed that the shuttle might not be safe enough to fly, and the top safety and engineering people had come to the opposite conclusion. Would it have been anywhere near as controversial if Griffin had bowed to them and let the launch proceed? I doubt it. Even though it would have been the same decision, it would have come from a team of experts, rather than a high-handed boss.

Is there anyone more loathed in office culture than the autocratic decision maker who ignores the opinion of the group? It's Business 101: Get lots of input, put your heads together, reach a consensus. The primacy of groups and teamwork is so ingrained that we seldom stop to think about it anymore. Now in the age of instant messaging, wikis, social networking sites, and videoconferencing on cell phones, collaboration and consensus are gaining yet more currency. We can, and often do, get everyone to weigh in, all the time, whether it's by cell phone, e-mail, or instant message. As James Surowiecki nicely puts it in the title of his best-selling book, it's "the wisdom of crowds," and it's a glorious thing.

Or it would be, if it weren't for just one little problem: The effectiveness of groups, teamwork, collaboration, and consensus is largely a myth. In many cases, individuals do much better on their own. Our bias toward groups is counterproductive. And the technology of ubiquitous connectedness is making the problem worse.

I'll understand if you demand to see the study on that one. But it's silly to quote a single study on the failure of groups because there are so many--dozens of them, going back decades--that there's no good way to pick one out.

Our bias toward groups is counterproductive. And technology is making the problem worse.

As far back as 1972, in his now classic book, Victims of Groupthink, Yale psychology researcher Irving Janis theorizes that groups often breed a false confidence that leads to unsound decisions none of the individuals in the group would have made on their own. In the 1990s, research by Purdue psychology researcher Kip Williams shed light on "social loafing"--that is, the tendency of people in groups simply to not try as hard as individuals. In fact, the notion that individuals outthink and outdecide groups is so well established among experts that they don't bother to study it anymore. Instead, the hot question among psychologists and organizational behaviorists is why the rest of us persist in keeping this wrong-headed notion alive. "We've been trying to find out what seduces us into thinking teams are so wonderful," says Natalie Allen, a psychologist at the University of Western Ontario who has studied what she calls "the romance of teams."

So what about the wisdom of crowds? Did Surowiecki really get it wrong? Not necessarily. He simply focused on the sorts of situations in which large groups of people can in fact work pretty well. A group of investors will usually outperform a single expert; the bad opinions in the crowd tend to cancel out, so that the average is "wise." Google can tap a sea of websites to provide useful answers, and crowds have done a great job creating Linux, because in these cases useful contributions from the crowd can be leveraged while noncontributors stay harmlessly out of the way. And to his credit, Surowiecki does note that crowds often are not very wise at all.

What he glosses over, however, is the often spectacular way groups fail in the context of organizations. Consider that paragon of group magic, the brainstorming session. Bernard Nijstad, a psychologist at the University of Amsterdam, explains that if you take a group of 12 people and have half brainstorm together on a topic while the other six go it alone, all 12 will usually agree that the group experience was more productive--even though those working alone almost always end up with more good ideas. Nijstad believes it's because people in groups spend most of their time listening to others rather than thinking on their own, while lone brainstormers are forced to stew in productive but unpleasant silence. "When you're alone, it's painfully clear when you're not producing. In a group you can just sit there and not notice you're not contributing," Nijstad says. No wonder we love to work in groups.

Things only get worse when a team is charged with actually making a decision. One of the biggest problems is that it's easy for a few members of a group who think the same way--but who may be flat-out wrong--to sway the opinions of others. Consensus steadily grows until a majority is reached, at which point even people who have confidence in their dissenting, higher-quality opinion are likely to bow to the group. If you've ever wondered how Enron's managers could have convinced themselves they were running a good company, or how a jury could have found O.J. Simpson innocent, now you know. Of course, you could bring independent thinkers to your groups--but then you'll run into the problem of deadlock. "About half of all groups don't reach any conclusion at all," says Nijstad.

New technology only amplifies the problem. For starters, the tools that connect us to our colleagues make it all the easier to form brainstorming sessions wherever we are, at any time--essentially turning all decisions into group decisions. What's more, these electronic group decisions can be even more brain-dead than in-person meetings. The biggest problem: the fear of dissenting is magnified in a Web, e-mail, or instant messaging exchange, because participants know their comments can be saved and widely distributed. Instead of briefly offending six people at a meeting, you have the chance to enrage thousands.

As for the Internet and our newfound ability to tap into the masses, a more subtle form of havoc arises. Simply put, when you make it easy for everyone to put in his two cents, with little filtering or accountability, the scum tends to rise to the top. Look at it this way: How much time have you taken to post movie reviews, rate products, or help a group of strangers with a project? Not a lot, I'll bet. But malicious adolescents, really grumpy people with a lot of time on their hands, and sleazy marketers just love plastering the Internet with their rants, gripes, and plugs. For all the excitement generated by social networking sites like Facebook, how many people are actually making valuable contacts on these sites, compared with the amount of time wasted browsing through the sea of goofy material out there? (If you're the parent of a teenager and you haven't seen your kid's Facebook listing yet, let me suggest you're better off leaving things that way.) And if you use Wikipedia, you might want to double-check the facts. According to a recent article in The Guardian, every three seconds a Wikipedia page is rendered inaccurate--or more inaccurate than it was to begin with--by a hoaxer, ignoramus, or malcontent.

Does forming groups and tapping the masses ever make any sense? Of course, but only in certain well-defined circumstances, say the experts. In some cases, it's more important to achieve buy-in to a decision than it is to get the best possible decision. Or when you need to get as many ideas on the table as possible without regard for how many of them are terrible. In such cases, electronic collaborative and input-gathering tools can be put to good use.

It's not time to swear off meetings and group efforts, online or otherwise. But it makes sense to be more selective about how we enlist them. Meanwhile, we can ponder the fact that Enron is nothing but a stain, O.J. Simpson enjoys golf, and the shuttle returned safely to earth in mid-July.

Contributing editor David H. Freedman (whatsnext@inc.com) is the Boston-based author of several books about business and technology.