A War of the Worlds: Hierarchies vs. Networks

One of the epic collisions of our time is between bureaucratic hierarchies and online networks. Which is more trustworthy, informative and useful? It is a delicious revelation to learn that this drama is causing a special angst within American intelligence agencies. The great, unanswered question is whether conventional bureaucratic systems of gathering and vetting intelligence information are too inefficient and “stupid” to compete with such collective-intelligence alternatives as wikis, blogs, instant messaging and other social software.

This issue is explored in yesterday’s New York Times Magazine by Clive Thompson. The article, “ Open-Source Spying,” explains how the intelligence community?s information technology practices are alarmingly out-of-date and backward. Only recently have they begun to recognize the serious virtues of blogs and wikis, for example.

Here’s the problem: Piecing together information about possible terrorist plots is difficult because intelligence agencies have too much information and no efficient and reliable ways to sift through it all to determine what’s important. At the same time, intelligence officials are highly protective of their sources and their institutional turf, often with justification. Yet whether for lame bureaucratic reasons or to protect domestic privacy and civil liberties, as dictated by law, intelligence agencies often do not share information among themselves. This makes it harder to find the needles in the haystack.

One encouraging development is the launch of Intellipedia, modeled after Wikipedia. The system has three tiers of user-generated contributions — Top-Secret, Secret and Unclassified — and is restricted to some 3,600 employees from 16 different intelligence agencies. Here’s how the system worked in one specific incident: When a two-seater plane crashed into a Manhattan skyscraper a few months ago, an analyst created a new page on Intellipedia within 20 minutes — and “over the next two hours,” writes Thompson, “it was edited 80 times by employees of nine different spy agencies, as news trickled out. Together, they rapidly concluded the crash was not a terrorist act.” So far, Intellipedia has reportedly amassed 28,000 pages of shared intelligence.

Ah, but here’s the paradox that Intellipedia may or may not surmount. The value of the information grows only as more people can participate — yet the number of people who are authorized to participate in the Top-Secret Intellipedia is relatively small. This could mean that the most robust tier of information will be in the Unclassified Intellipedia and the least helpful tier will be Top-Secret Intellipedia.

But maybe we need to develop a radically different notion of “intelligence.” Thomas Fingar, the director of national intelligence, actually says that top-secret information is becoming less valuable:

“The intelligence business was initially, if not inherently, about secrets — running risks and expending a lot of money to acquire secrets,” he said, with the idea that “if you limit how many people see it, it will be more secure, and you will be able to get more of it. But that’s now appropriate for a small and shrinking percentage of information.”

In other words, all the derring-do of secret agents that is immortalized in Hollywood movies may be taking a back seat to online social networks. Clive Thompson writes that “in a world that is awash in information, it is possible, they say, that the meaning of intelligence is shifting. Beat cops in Indiana might be as likely to uncover evidence of a terror plot as undercover C.I.A. agents in Pakistan. Fiery sermons printed on pamphlets in the U.K. might be the most valuable tool in figuring out who’s raising money for a possible future London bombing. The most valuable spy system is one that can quickly assemble disparate pieces that are already lying around… .”

This apparent shift in the meaning of “intelligence” has hopeful implications. It suggests that the core virtues of democratic practice — open deliberation and discussion — are being ratified by one of the most anti-democratic, unaccountable institutions in the U.S. Government, the intelligence agencies. The CIA and NSA are not about to embrace the open source ethic or, for that matter, robust democratic oversight. The national security clique and the cult of secrecy are too deeply entrenched to suddenly disarm.

Still, I consider it a hopeful sign that the raw, practical power of networking technologies may be pushing them in the right direction. Even better: the rest of us who understand the power of open networks are developing greater moral and political legitimacy among citizens while gaining an “information edge” over our government. For once, a healthy competition on our turf. How far can the open-source ethic extend?