Late last year there was a story that I think didn’t get the attention it deserved. A thesis was submitted for a PhD candidate titled: “Information Sharing and Collaboration in the United States Intelligence Community: An Ethnographic Study of the National Counterterrorism Center” by Bridget Rose Nolan. While I can’t comment on its merits as a thesis it is a fascinating look at the culture and operations of the National Counterterrorism Center.
The author was a new employee of the Central Intelligence Agency when she was assigned to the NCTC in 2009. She presents a ‘bottom feeder’ view of the organization which may (in fact, I’d bet huge amounts of money on this) vary greatly with the impressions of those higher up the food chain. That being said, her observations and impressions provide insight into the day-to-day operations of the Intelligence Community. While I’ve never worked at the NCTC, Ms. Nolan’s observations ring true both to my personal observations as well as what I’ve been told by others working at various levels within the IC.
I’d therefore like to take select quotes from Ms. Nolan’s work and then expand upon them here.
Almost all of the analysts I formally interviewed as well as colleagues I spoke to during informal conversations spontaneously mentioned that they simply could not keep up with the volume of information they had to deal with in the course of a day, and that trying to manage the information overload took up a lot of their time.
Everyone feels like were deluged with information (Thanks, internet!) but the IC was hit with the criticism of not ‘connecting the dots’. One response has been to make sure that everyone has access to as much information as possible. That sounds great and has the added advantage that it’s a easy metric to trot out to demonstrate ‘improvement’ in the system (‘We’ve given our people access to X more databases since this time last year.’)
That, however, can be highly misleading for several reasons.
First can existing personnel handle the influx of new information? You’re going to be hard pressed to find people in the IC who say they’ve got boatloads of time on their hands (although, time and resource allocation is another problem that deserves its own post) and so inserting another database, information feed, whatever to the mix risks two bad outcomes:
- People just ignore the new information stream
- People incorporate the new information but every data-stream gets less time and attention
All else being equal I consider ’1′ to be the lesser of two evils.
The other, more important, question is whether added information improves analytical quality. Conventional wisdom is that ‘more information is always better’ but that’s not really true. More information makes you more confident in your decision but doesn’t improve your results. 1 This means that providing ever increasing reams of information and expecting (or demanding) that all that information be checked does little other than absorb already scarce amounts of analyst time, preventing them from doing quality analysis.
To which you can expect someone to decide the solution is to get the analysts access to one more data stream.
Lather, rinse, repeat.
At just about the same time this paper was released, Mark Stout wrote about a related topic over at War on the Rocks. He begins by reviewing the difference between secrets and mysteries:
Secrets are questions to which there is a factual answer. An example is “Where is Ayman al-Zawahiri?” There is an answer to that question, we just don’t know what it is yet. By contrast, mysteries are questions to which there is no factual answer. An example might be “What will Ayman al-Zawahiri do next week?” (Note that this is quite different from “What does Ayman al-Zawahiri intend to do next week?”) There is no factual answer to this question because it depends on future events, including interaction with other human beings, and the future is always in motion.
We don’t like mysteries because they’re messy and don’t have an answer (well, until they move from the future to the past) so we are inclined to ignore them and just treat everything like a secret.
The way to find secrets is to collect more data and somewhere in the mass of data will be the secret or pieces of a secret which can be assembled like a puzzle. In the case of mysteries, however, collecting more data is typically the wrong thing to do. More data often makes it impossible to see the forest for the trees.
But even in the cases of ‘secrets’ too much data can be a problem. One must balance the ability to collate and analyze data versus the likelihood than any new data source will contain valuable information. The more data you attempt to absorb the higher the bar should be before adding any new information. You always could replace an old, less useful data source for a new, better one but that tends not to happen. Usually we just throw new information on top of old under the assumption that ‘Just one more tiny bit of data won’t hurt. It certainly won’t impair our ability to collate and analyze information.’
Woe is he who does not heed the warnings of that line of thinking…
There are some really great quotes from analysts that try to deal with this mess. Overwhelmingly, they say they’re overwhelmed and make decisions about ignoring large quantities of incoming information. And while they say they try to systematize what they look at and what they don’t it sounds like any such rules they have developed are more ad hoc than planned or tested.
Is this really a situation we want to be in? As management continually scurries to get access to more and more information, analysts are scurrying just as quickly to figure out how to ignore that data. Without, of course, letting management know.