Tag Archives: intelligence analysis

Can a tiger attack in Siberia tell us anything about the state of affairs in Syria and Iraq?

I’m currently reading ‘The Tiger:  A True Story of Vengeance and Survival“.  At roughly the half way point the author goes on a bit of a diversion from the main narrative to explain his method for attempting to understand the motive of the tiger and how that may overlap or conflict with various humans in his story.  To do that he brings us to a brief discussion of Jakob von Uexkull and the concept of Umwelt.  As I was reading it I thought there might be some interesting applications for intelligence analysis.  So, first let me take some quotes from the text to set my own background….

When talking about understanding animal behavior, Uexkull recommended imagining “a soap bubble around each creature to represent its own world, filled with the perceptions which it alone knows.  When we ourselves then step into one of these bubbles, the familiar…is transformed.”

That bubble is referred to as the Umwelt which is different (but inseparable from) the Umgebung.  The Umgebung is the objective world or reality which none of us really see/experience because we can only access it from behind the hazy view of our soap bubble Umwelt.

Back to the book:

In the umgebung of a city sidewalk, for example, a dog owner’s umwelt would differ greatly from that of her dog’s in that, while she might be keenly aware of a SALE sign in a window, a policeman coming toward her, or a broken bottle in her path, the dog would focus on the gust of cooked meat emanating from a restaurant’s exhaust fan, the urine on a fire hydrant, and the doughnut crumbs next to the broken bottle.  Objectively, these two creatures inhabit the same umgebung, but their individual umwelten give them racially different experiences of it.

Our Umwelt (if you’d like a more scientific analogy) might be thought of like your DNA.  Not only is every species different but there are also differences between individuals in a species.  One would expect that the umwelt of two people (regardless of their upbringing, ideology, etc.) would be much similar than that between a person and a bird, for example.

One of the challenges we have in intelligence analysis is correctly identifying the interests, priorities and motives of opponents.  There are techniques like ‘devil’s advocacy’ or ‘red teaming’ that can have real value but also are susceptible to biases like mirror imaging if the practicioners don’t have sufficient traning and experience in them.  This, in turn, can lead to false levels of confidence and poor decision making.

It may be possible to use the concept of Umwelt to reduce the risk of such failings, even though that wasn’t the original intent of this process.  Again, back to the author:

One way to envision the differences between these overlapping umwelten is to mentally color-code each creature’s objects of interest as it moves through space; the graphic potential is vast…and it can be fine-tuned by the intensity of a given color, the same way an infrared camera indicates temperature differences.  For example, both dog and mistress would notice the restaurant exhaust fan, but the dog would attach a ‘hotter’ significance to it-unless mistress happened to be hungry too.

And this was the point I thought about the conflict in Syria and Iraq.  In that conflict we have literally dozens of interested parties; nation states (Iran, Iraq, Turkey, Syria, Jordan, United States, etc., etc.,) and non state actors (ISIS, al-Nusra, FSA, various Kurdish factions, ExxonMobil, various business interests, etc.), each with their own umwelt.  It’s simply not possible to hold all the various perspectives and motivations of all the players (or, I suspect, even the ‘major’ players) in ones head.  I think even a written document (the form we see many intelligence documents come in) can adequatly give the consumer a suitable perspective to take into account the moving parts necessary to craft good decisions.

Usually what happens is that we simplify the problem to a ‘managable’ level of actors and then proceed.  Provided this reduced number of actors are the ones who have the ability to dominate events that’s probably ‘good enough’.  It does not, however, take into account the possibility that ‘insignificant’ actors occasionally have an outsized influece under special circumstances.  Essentially what you’re doing is trading the risk of surprise for simplicity.  This may be a good deal…but if we don’t decide on that tradeoff early on we can forget that we’re even taking the risk.

Here’s one example of the sort of product you’ll see.  They certainly get more complex to cover more nuance and include more actors but I’m not convinced that lends itself to more understanding or better decision making.

There may be a graphic product, however, that captures the varied interests of the players as well as the (estimated) intensity of those interests.  From that point, it should be easier to both estimate future decisions of each actor as well as assist in making more effective decisions ourselves.  And we needn’t confine this to only the extremely complex cases like we see in the Middle East now.  While probably too complex and time consuming for every case you could certainly apply it to long standing criminal organizations as well as terrorist ones.

Here’s one graphic product that hints at what I’m talking about but it’s only looking at behavior (conflict/support) and doesn’t even get into the ‘why’ question or intensity of interaction.

I dont’ have a fully formed idea of what this product would look like yet but give me some time…

Anatomy of a (sub)standard Intelligence Product

Last time I wrote about how we still don’t do a good job of classifying terrorist actions.  As an example of that I used this alleged intelligence product and what I’d like to do today is run through why I think it’s not up to snuff.

First things first. What’s with that color? I am all about encouraging analysts to experiment with their products to make them more relevant and make sure they ‘stick’ with their audience more but I’m not sure about this color choice.  It’s very non-traditional. (Update:  I’ve just looked at a downloaded copy and it’s a much cleaner and more traditional light blue. I would just delete this but here’s a good example of one of the pitfalls of critiquing something on the web…nertz to me!)

So non-traditional in fact that it reminded me of a scene in Yes, Prime Minister.  You can see the whole episode at the end of this post but here’s the money quote:

All I can say is, if that’s what you’re going to say, I suggest a very modern suit, hi-tech furniture, high-energy yellow wallpaper, abstract paintings. In fact, everything to disguise the absence of anything new in the actual speech.

You can download the presentation here:

Terrorism Powerpoint Presentation

Ok, so this is a joint FBI/Pennsylvania State Police product.  It’s unclear who the audience is but it is worthwhile to note that there are no classification markings on the document. By default that would make this ‘unclassified’ but I find that hard to believe.  This could be another (along with the weird color) be an indicator that this is a fraudulent document.  But, it could also be that this was an internal document or a draft and in those cases we could just be seeing a bit of sloppy work.

We open with a definition of ‘Domestic Terrorism’.  I’d like to see a citation for that but perhaps that’s given in the talk that (I hope) would go along with the slide.  It appears to be from the U.S. criminal code and given the probable audience (law enforcement officials) here let’s not deduct anything.

Then they use a definition of Eco-Terrorism from the Anti-Defamation League.  I’m less enamored with this slide.  Is there no official definition of this term?  If not, why not?  Does this mean that the ADL is official government policy? I’m usually a big proponent of reaching out to outside experts but if you’re going to flip back and forth between official and unofficial terms, definitions, assertions and opinions you should make it clear which is which and I’m not sure a parenthetical note here makes the grade.  Again, this might be something discussed in the talk but I’m going to make a deduction here.  Also, they fact that they changed the font to underscore a point but picked a color that actually makes it blend into the background isn’t particularly good.

I’ll also recommend you note the quote they highlight.  ‘Eco-terrorists’ are defined as the ‘most active’.  What I believe the authors are trying to do with that quote is use ‘most active’ as a synonym for ‘most dangerous’.  That’s not particularly clear, however.  They time frame they use is long (two decades…that’s an entire generation) and it’s not clear when that damage occurred.  What if $99 million dollars of that damage and 90% of all incidents occurred prior to 1996? What if they occurred after 2012?  I suspect you’d get two very different responses to just how threatening and active ‘Eco-Terrorists’ are.

Now…this is interesting.

The title of slides 2-4 go:

  • Domestic terrorism defined
  • Eco-terrorism
  • Environmental Extremists

This is the narrative path they want you to go down. Graphically, it looks something like this:


That is known as the old switcheroo.  What it should look like is this.

slide2To explain that you probably want a slide order of:

  • Environmental Extremists
  • Domestic terrorism defined
  • Eco-terrorism

This may seem like a small thing but it really sets the stage for what may be a whole host of problems down the road.  If one of your foundational propositions is that all extremists are terrorists that’s a problem and will lead you down a road towards illegal and unconstitutional activities.

How do I know this isn’t just sloppy work and they meant slide two and not slide one?  Bullet two:

  • Nonhierarchical and autonomous with lone offenders and small cells posing the greatest threat of criminal activity. Ecoterror cells are extremely difficult to identify and infiltrate;

In the slide about ‘extremists’ they go right to talking about ‘Ecoterror cells’.  No distinction between the two is made.  That’s simply wrong.

It’s also interesting to note that they quote the ELF ‘credo’.  If you’re going to quote stuff to support your case you should also explain the stuff that undermines it.  Most eco-animal rights extremists renounce violence against people.  In fact, it’s usually a central tenet of their belief structure.  To ignore that in favor of cherry picking the stuff that makes them sound more dangerous is disingenuous.

On slide 5, note the criminal activity identified:

  • Criminal activity has ranged from graffiti and trespassing, to vandalism, sabotage and arson;

I won’t belabor this point because it’s settled now but in what bizarro world is graffiti, trespassing and vandalism rise to the level of a terrorism investigation?  Only if those things are combined with the threat of violence should it be.  Otherwise we’re talking about criminal activity that is easily handled by local law enforcement and handled quite well under the criminal justice system.

And on slide 6, in the last bullet we finally get to this:

  • Historically, activities have not intended to harm individuals.

That’s clearly a throwaway line.  After six slides about how dangerous they are and their targeting priorities we get a brief statement about how they did things…historically.  You know…like back in ye olden days.

Slide 8 gets a bit weird.  There’s no reason why this event should be here.  You see a statement that looks like it could have come from any activist organization.  It talks about online activism to achieve legal (and pretty mainstream) ends.

Slide 9…again.  The security camera hunting campaign is interesting.  While Earth First did carry it so did other groups.  It might be worthwhile to see  if there were any reports of security cameras being attacked.  It might be worthwhile to see if other campaigns like this have been announced and how successful they were.  But, if you’re cherry picking your facts you probably don’t want to ask (and definitely don’t want to answer) those questions.

The information from START is good….but it doesn’t really say what the author(s) want it to say.  Here’s the paper that they quoted.  This slide is designed to say “Danger! Danger!” But let’s look at what the data shows…

  • 239 attacks from 1995-2010 (15 years or roughly 16 attacks per year worldwide)
  • 66% occurred in the West (roughly 11 per year)
  • 42% of those attacks that took place in the West “resulted in substantial or very substantial property damage and
    financial losses” (that’s a total of 66 attacks or 4 attacks per year in all Western nations)

I find I can’t really say much more about this because the slide does such a poor job of mangling the original research that we just need to bury this and move on.

(Protip:  If you’re going to quote someone else’s work it’s a good idea to quote it correctly and understand it.  Just sayin’)

Then we get a number of slides about civil disobedience actions.  Without the discussion notes we can only speculate about how these were described which I won’t do here.  At no point, however, is it made clear why this is anything more than a local law enforcement issue.  Ok, a bunch of people are protesting and trespassing.  Get the paddy wagon, boys, and lock ‘em up or move them along.

Slides 16-22 finally give us something of a threat.  Various incendiary and explosive devices along with a report of a shooting.  It is important to note that the presentation doesn’t link any one of these events to environmental extremists.  There are a whole lot of reasons why people might do these things without being affiliated with the environmental movement.  Disgruntled workers come to mind.  The way this information is presented, however, you are practically forced to come to the conclusion that the crazy environmental types are behind these.  They may but that’s not clear from the information provided.

It’s frustrating that I probably just spent more time reviewing this product that the author(s) spent constructing it but there you go.  Don’t let this happen to you.

Watch Yes Prime Minister 1.2 -The Ministerial Broadcast in Comedy | View More Free Videos Online at Veoh.com

Peeking behind the curtain of the IC

Late last year there was a story that I think didn’t get the attention it deserved.  A thesis was submitted for a PhD candidate titled:  “Information Sharing and Collaboration in the United States Intelligence Community: An Ethnographic Study of the National Counterterrorism Center” by Bridget Rose Nolan.  While I can’t comment on its merits as a thesis it is a fascinating look at the culture and operations of the National Counterterrorism Center.

The author was a new employee of the Central Intelligence Agency when she was assigned to the NCTC in 2009.  She presents a ‘bottom feeder’ view of the organization which may (in fact, I’d bet huge amounts of money on this) vary greatly with the impressions of those higher up the food chain.  That being said, her observations and impressions provide insight into the day-to-day operations of the Intelligence Community.  While I’ve never worked at the NCTC, Ms. Nolan’s observations ring true both to my personal observations as well as what I’ve been told by others working at various levels within the IC.

I’d therefore like to take select quotes from Ms. Nolan’s work and then expand upon them here.

Almost all of the analysts I formally interviewed as well as colleagues I spoke to during informal conversations spontaneously mentioned that they simply could not keep up with the volume of information they had to deal with in the course of a day, and that trying to manage the information overload took up a lot of their time.

Everyone feels like were deluged with information (Thanks, internet!) but the IC was hit with the criticism of not ‘connecting the dots’.  One response has been to make sure that everyone has access to as much information as possible.  That sounds great and has the added advantage that it’s a easy metric to trot out to demonstrate ‘improvement’ in the system (‘We’ve given our people access to X more databases since this time last year.’)

That, however, can be highly misleading for several reasons.

First can existing personnel handle the influx of new information?  You’re going to be hard pressed to find people in the IC who say they’ve got boatloads of time on their hands (although, time and resource allocation is another problem that deserves its own post) and so inserting another database, information feed, whatever to the mix risks two bad outcomes:

  1. People just ignore the new information stream
  2. People incorporate the new information but every data-stream gets less time and attention

All else being equal I consider ‘1’ to be the lesser of two evils.

The other, more important, question is whether added information improves analytical quality.  Conventional wisdom is that ‘more information is always better’ but that’s not really true.  More information makes you more confident in your decision but doesn’t improve your results. 1  This means that providing ever increasing reams of information and expecting (or demanding) that all that information be checked does little other than absorb already scarce amounts of analyst time, preventing them from doing quality analysis.

To which you can expect someone to decide the solution is to get the analysts access to one more data stream.

Lather, rinse, repeat.


At just about the same time this paper was released, Mark Stout wrote about a related topic over at War on the Rocks.  He begins by reviewing the difference between secrets and mysteries:

Secrets are questions to which there is a factual answer.  An example is “Where is Ayman al-Zawahiri?”  There is an answer to that question, we just don’t know what it is yet.  By contrast, mysteries are questions to which there is no factual answer.  An example might be “What will Ayman al-Zawahiri do next week?”  (Note that this is quite different from “What does Ayman al-Zawahiri intend to do next week?”)  There is no factual answer to this question because it depends on future events, including interaction with other human beings, and the future is always in motion.

We don’t like mysteries because they’re messy and don’t have an answer (well, until they move from the future to the past) so we are inclined to ignore them and just treat everything like a secret.

The way to find secrets is to collect more data and somewhere in the mass of data will be the secret or pieces of a secret which can be assembled like a puzzle.  In the case of mysteries, however, collecting more data is typically the wrong thing to do.  More data often makes it impossible to see the forest for the trees.

But even in the cases of ‘secrets’ too much data can be a problem.  One must balance the ability to collate and analyze data versus the likelihood than any new data source will contain valuable information.  The more data you attempt to absorb the higher the bar should be before adding any new information.  You always could replace an old, less useful data source for a new, better one but that tends not to happen.  Usually we just throw new information on top of old under the assumption that ‘Just one more tiny bit of data won’t hurt.  It certainly won’t impair our ability to collate and analyze information.’

Woe is he who does not heed the warnings of that line of thinking…

YouTube Preview Image

There are some really great quotes from analysts that try to deal with this mess.  Overwhelmingly, they say they’re overwhelmed and make decisions about ignoring large quantities of incoming information.  And while they say they try to systematize what they look at and what they don’t it sounds like any such rules they have developed are more ad hoc than planned or tested.

Is this really a situation we want to be in?  As management continually scurries to get access to more and more information, analysts are scurrying just as quickly to figure out how to ignore that data.  Without, of course, letting management know.

  1. For more see here

Establishing intelligence priorities (an alternative)

One of the challenges intelligence personnel face is the lack of clear priorities.  This should be taken care of in the Planning and Direction process but it’s difficult and many a risk-averse officials would prefer to make no decision (or make a non-decision) which protects them from blame if something goes wrong but allows them to take credit if things go right.

As a result, an analyst shop may suffer from lack of focus and drift from one crisis to another, constantly reacting and always finding itself late to the party.

In those cases, I submit an imperfect solution.  This shouldn’t be your first choice since your decision makers should be the ones making decision but if that doesn’t look like it’s going to happen this will enable you to get some focus in your shop AND apply a methodology to that focus so that you can articulate why you prioritized the way you did.

There’s lots of opportunity to experiment or modify this system to fit your particular circumstance but this will give you something you can begin to make some decisions off of.

1) Gather together a representative sample of your customers.  If customers aren’t available then get people who interact with those customers on a regular basis.  Keep the groups relatively small (20 or 25 is probably the most you’ll be able to handle in one session).

2) Conduct a structured brainstorming session asking two questions.  First, who (specifically) do you see as the primary customers of the intelligence shop.  Second, what (specific) subjects are primary customers of your intelligence services interested in?

3) Take the answers back and collate them.  Be careful not to lump answers into too broad of categories.  When in doubt treat answers as separate entities.

4) When you have a collated list (and it could be quite long) have each person in your shop rank those lists (the one for customers and the other for products) from most to least important.  When asking people to make this ranking decision it would be a good idea to provide them with any contextual information that you deem relevant.  This might include mission statements, legislative or regulatory requirements, etc.  I’d recommend ranking them in a simple 1 to whatever list.

5) Once you’ve received all the rankings add them up.  For example Topic A receives a ranking of ‘1’ from one analyst, ‘7’ from another and a ’12’ from a third.  Give than entity a ranking of ’20’ (1+7+12).  This will give you a consensus ranking for your shop.

6) You know should have two ranked lists.  Take some manageable number from each (5, 10, 20?) and create a matrix with customers on one axis and product types on the other.

Here’s a sample of what this looks like (click to enlarge):


7)  Now comes the fun part.  You know have a list of product topics and customers.  You go through each cell and identify the elements of product definition (scope, purpose, type of output, etc) for each cell.  So, perhaps an elected official will need a strategic level policy briefing in order to help propose legislation on a topic while a law enforcement official will need to know where to deploy her resources with a GIS predictive analysis.

8) Now that you’ve identified the full range of ‘high priority’ customers, topics and product types you (or someone) has to make a decision of which to do in what order.

This doesn’t eliminate the need for someone to make a decision but it does make it easier to do so for decision makers.  Instead of trying to whittle down all the potential threats in the world and juggle all the variables of customer, purpose, etc. this narrows down their choices to something which is (hopefully) manageable.

The state of terrorism reporting

Will Potter from Green is the New Red writes a post about a leaked FBI Intelligence Report from early 2012.  I like Potter’s work but he’s an advocate for a particular cause and so we don’t always come from the same place.  I think his attempts at equivalency (‘This is the state of the government’s “terrorism priorities.” Favorable media coverage exposing animal cruelty–at a lab that was fined for abuses–is on par with weapons of mass destruction.’) are wrong and that his arguments miss the important pathologies these products demonstrate.

The report in question can be found here.  The first thing I’d like to recommend is look at the titles of each of the subjects (emphasis added):

  • AR Extremists May Increase Criminal Activity after News Program Featuring Animal Research Airs
  • Anti-Abortion Extremist Activity May Increase after Graphic Advertisement
  • Planned “International Judge Muhammad Day” Could Increase Threats to Homeland or Escalate Anti-Islamic Sentiments
  • LCN Arrests May Attract Media Attention
  • Pimps Likely Transport Children across State Lines for Prostitution at Major Public Events

In five of the six headlines the FBI has inserted ‘weaselly’ words.  Sure terrorist might to this or might do that or…they may just buy a cute cat.

This cat MAY be a T-9000 terminator sent back in time to kill you.

This cat MAY be a T-9000 terminator sent back in time to kill you.

Let me be clear that I have NO insight into how the FBI produces (or produced…they may no longer make these things) these products but this looks like an instance where institutional demands trumped any sort of intelligence value.  There’s nothing theoretically wrong with intelligence products that come out of a regular basis like this one (weekly).  Problems arise, however, when expectations of what those products should look like outstrip reality.

I suspect various functionaries have decided that it would not be acceptable for the F.B.I. to put out something called a ‘Weekly Intelligence Report’ and not have something to say.  I’d be willing to put money down that conversations similar to this actually take place:

-Mid-Level flunky:  Hey!  The Weekly Intelligence Report is almost due. What do we have to put in it?

-Bottom feeder analyst:  Things are actually pretty quiet.  We’re not seeing any new threats this week.

-Mid-Level flunky: Not acceptable! How about we just put in something about that TV show coming up and say it might encourage terrorists to attack

-Bottom feeder analyst:  Uh…but there’s no evidence of that.

-Mid-Level flunky:  Yeah, but terrorists COULD be motivated to attack. Put it in there!

The other tell in situations like this is phrases peppered in the product like the following:

Though the FBI has no reporting indicating specific threats related to the broadcast, extremists could use information from the program to target researchers or facilities.

If you ever see wording like that you should mentally insert the following:  ‘We’re just making this shit up.’

An exception to the above rule would be if the authors actually laid out a line of reasoning why this really might happen.  Maybe similar events instigated attacks, maybe there’s something new in the subjects capabilities or intent that makes action in this case more likely.

They attempted to do that in another entry but look what they do…

While the announcement has given rise to no specific reported threats, the earlier event inspired backlash
overseas, including by terrorist groups and extremists. The proposed event could lead to similar
reactions and could give rise to threats in the Homeland or escalate anti-Islamic sentiments, potentially resulting in anti-Muslim hate crimes.

So, they link this upcoming event to the overseas backlash and that’s good.  Cause – Effect.

But then they make a leap to say that could translate to attacks here in the states 1 and we’re right back to square one with that wishy-washy ‘could’ crap.  Would it have killed them to take the extra step and spell out who has and/or who is likely to be motivated to carry out such attacks?

‘So what?’ I can hear you say.  Let them publish their bulletin and just ignore it if it doesn’t have value, right?  The problem there is this isn’t just one weekly bulletin.  This goes on all the time by hundred of agencies all around the country.  Each creating their own calorie free product loaded with speculation because they refuse to say ‘We’ve got nothing.  We’ll let you know once there’s something relevant to share.’  As a result everyone get bombarded with products like these making it difficult (if not impossible) to read and evaluate all the information that comes pouring in.

And in that vein…I’ll leave you with this.

YouTube Preview Image
  1. And we’ve got to get rid of that fascist ‘homeland’ crap

How language affects analysis

A lot of things we have no control over have profound influences on how we interpret the world around us and, therefore, the way we can conduct analysis.  One of the most basic is the language we use.  The words we have at our disposal and the ways in which we can use them.  Two recent articles talk about just that phenomenon, plus also demonstrate another current lesson relevant to intelligence analysis.

First, from NPR, is this piece which talks about how language can affect how we perceive something like direction.

Lera Boroditsky once did a simple experiment: She asked people to close their eyes and point southeast. A room of distinguished professors in the U.S. pointed in almost every possible direction, whereas 5-year-old Australian aboriginal girls always got it right.

The hypothesis is that the Australian aboriginals use language which centers around compass points rather than relative descriptions of location (X is to the left of Y) and therefore, they have a better awareness of where they are geographically.  I have no idea if it works the other way and these girls would have trouble description the position of two things in relation to each other but the point is there probably isn’t an ‘ideal’ way of describing something.  It’s just that using one system closes (or makes more difficult) the ability to use another.

The second piece is from Cracked.  I’m not sure what’s going on over there but I’ve found a couple of pieces there lately that belie the image of the site as being all penis jokes photoshopped punch lines.  Or, maybe I’m just trying to find an excuse to read all those penis jokes….

Anyway, the article gives very brief overviews of 5 ways language can screw with your worldview and one in particular jumped out at me.

Recent studies have suggested that language may act as a cue to which cultural frame of reference a given interaction belongs in…Psychologists call this phenomenon frame-shifting, and it’s basically the ability to put yourself in someone else’s cultural shoes just by speaking in their language.

For example: A test was applied to bilingual Arab Israelis who spoke both Arabic and Hebrew (two cultures that have famously held a little animosity toward each other over the years) that asked participants to record whether words had negative or positive connotations. When the test was given in Arabic, the participants picked Jewish names as being intrinsically negative, but this effect disappeared when the test was given in Hebrew. In short, their bias against Jewish names arose from the fact that they were thinking in Arabic at the time, and not because they necessarily had any deep-seated bias against Jews. Don’t go thinking that the Arabic language is somehow inherently racist — it has plenty of Jewish friends. They just go to another school; you wouldn’t know them.

Probably worth some serious consideration when thinking about sources that are translated or even among those who are non-native speakers.  In intelligence analysis you are usually working with small amounts of information and trying to divine meaning (perhaps too much) from the dribs and drabs you get.  Given how preconceived notions and cognitive biases can form so easily and early a turn of a phrase or different intonation can send even a good analyst careening off into left field.

And just so you don’t feel let down now that you made it all the way through this post, here’s a link complete with your daily allotment of penis jokes….enjoy.

Thoughts on becoming an Intelligence Analyst

I just caught this post that was written back in October by Matthew Burton 1

Matt spent a few years with the Defense Intelligence Agency and so writes with some advice about becoming an intelligence analyst.  I agree with most of what he says with a couple of changes in emphasis.  There are also a couple of points I think deserve a bit more discussion and since I’m already writing this post I might as well get into it, right?

Is analysis for you?

People enter into intelligence analysis for all sorts of reasons, some of them better then others.  I find the people who are most successful at it are the ones who see analysis as a destination rather than as a stepping stone.  I suppose that makes sense but there are enough people who use an analyst position as a ‘foot in a door’ that it warrants a bit of discussion.  If that’s your plan, it may work but be prepared for what happens if you wake up one day, it’s five years later and you’re still sitting in that analyst chair.  It’s pretty common.

Burton argues that the primary requirement…maybe even the only requirement for an intelligence analyst is the ability to write well.  Being able to convey ideas to your customer is clearly important but I’d argue that intellectual curiosity is even more important.  Most government documents are rather formulaic and you can get away with being a rather mediocre writer (Yeah, we know. eds.) but what you need is the ability to look at a subject you’re assigned and, even if it would normally bore you to tears, get yourself sufficiently interested in it to identify the relevant questions, research it thoroughly and then consider how you can convey relevant information to your audience.

Subject matter expertise can be taught but curiosity can’t and that’s a key indicator (IMHO) for what will make a good analyst.  Do they enjoy playing with ideas? Can they make connections between disparate and seemingly unrelated pieces of information to create some new way of looking at a problem?

In that regard, I’m never very impressed with particular fields of study and cringe a bit inside when I see job openings, particularly for entry level analysts, that require degrees in specific fields like criminal justice, security studies, etc.  I could totally envision a situation where a recent graduate with a degree in philosophy, sociology, psychology, or biology (to name a few) would be a MUCH better candidate as a new analyst than one in a more ‘traditional’ field.  The unfortunate thing is that those people don’t always even get the chance to throw their hat into the ring.

While not for everyone, I still recommend the military as a great introduction to the field.  You have the best opportunity to see the link between analysis and how that translates to action.  I find that to be an important lesson that, if not learned, can become a bit too theoretical for some analysts.

Check out the article…well worth the read.

  1. You probably don’t know Burton but a couple of years ago he created a great, yet widely overlooked, open source, collaborative program for Analysis of Competing Hypothesis. That deserves it’s own post but the bottom line is the guy has some really good ideas.

Can you crowdsourse the collation function?

The collation part of the intelligence cycle (if, indeed, it is a cycle) is the least appreciated and ‘sexy’ part of the whole thing.

Defined, collation (sometimes also called ‘processing’ or considered a subset of that word) is:

Once the collection plan is executed and information arrives, it is processed for exploitation. This involves the translation of raw intelligence materials from a foreign language, evaluation of relevance and reliability, and collation of the raw intelligence in preparation for exploitation. 1

In short, its converting the varied inputs of intelligence (interviews, bank records, photographs, etc.) into forms of data that allow the information to be analyzed.  Generally, it sucks.  Converting or coding one set of data, for example, into spreadsheets, databases, etc. It’s also very time consuming and can be fraught with errors.

It may interest you to know that in the law enforcement and homeland security fields there is a real dearth of standardized datasets from which to draw to assist with analysis.  Want to know if the use of explosive devices by extremists is on the increase or decline?  Well, it depends on whose data you use and none of it is great or universally accepted.

With regards to terrorism studies, START‘s Global Terrorism Database is among the best but it has problems associated with it as well.  These aren’t really defects in the START folks (they’re very nice and extremely helpful) but rather because, in part, of problems with collations.  A few examples…

Let’s say you want to capture terrorist plots and attacks in the United States.  What would you count?  When does a plot go from some knucklehead blowing off steam and talking nonsense to a real threat?  Should we even make that distinction? Is it enough to say ‘I want to attack the United States’ or would the individual need to identify a target?

Many researchers try to sidestep this issue by trying to use the criminal justice system to do their work for them.  It’s easy to count people who have been charged under terrorism statutes or convicted of terrorist crimes.

That system leaves a lot to be desires.  What happens is charges are dropped or over turned?  Many times, a terrorism arrest is announced to much fanfare only to have most of the substantive charges dropped for one reason or another. Then you over count the plots and attacks.  What if, for tactical prosecutorial reasons, the suspect is charged with something other than terrorism offenses?  Then you risk under counting plots and attacks.

This is a real issue and I’d point you to this group that was arrested in Fort Stewart, Georgia a couple of years ago as evidence of that.  A number of current and former soldiers with plans to attack an Army base, assassinate the President and with two (and possibly more) homicides and other crimes under their belt.  Never heard of this terrorist threat? I’m not surprised as they weren’t charged under a terrorist statute.  So, they fall into the ‘general crime’ category.  Clearly, (assuming the initial reports were correct) these are folks you’d want included in any analysis of the terrorist threat.

Unfortunately, this is too big of a job for state and local agencies to tackle and it’s not clear if a federal agency (the FBI or DHS probably) has the interest or will to do this.  Further, if you want widespread acceptance, such a task is probably undertaken with broad buy-in, transparency and peer review.

These sorts of things have been bumping around in my mind for awhile when I saw this story from Slate.  After the Newtown shooting, they started documenting all the firearms deaths in the United States.  Now, one year in, they’ve got more than 10,000 reports of gun deaths and, in order to analyze them, have to….(wait for it…)….collate all that data.

Rather than trying to do it in-house or hire someone to do it (or foist it off on some poor interns) they decided to try crowd-sourcing it.

Here’s how it works.  They provide you with an article and the name of a victim about a gun death.  You read it and then select if the death fits within one of the following categories:

  • murder
  • suicide
  • accident
  • shot by law enforcement
  • shot by civilian in self defence
  • other
  • link not working

The nice thing about it is, apparently, it focuses on the victim rather than just the article.  So, I suppose an article where two people died (let’s say a murder/suicide) one person may get asked about the suicide and the other the murder.  Also, they don’t just rely on one person’s response 2 but rather on a consensus of some number of respondents.  That’s a nice check on work, but as we all know, the majority isn’t always right.

Now, they did recently identify a significant flaw in their methodology.  While their data collection was exclusively based on open source reporting it quickly became clear that suicide by gun is under-reported.  On those occasions when suicides are reported (not very often) it’s almost unheard of for the report to describe the method of death.  Still, like in most things, admitting you’ve got a problem is the first step in fixing it.

Another example is through the British Library.

In 2008, the British Library, in partnership with Microsoft, embarked on a project to digitize thousands of out-of-copyright books from the 17th, 18th, and 19th centuries. Included within those books were maps, diagrams, illustrations, photographs, and more. The Library has uploaded more than a million of them onto Flickr and released them into the public domain. It’s now asking for help.

Next year, it plans to launch a crowdsourced application to fill the gap, to enable humans to describe the images. This information will then be used to train an automated classifier that will be run against the entire corpus.

The library is also soliciting ideas for how to present the collection to aid the tagging and metadata generation, and also make the pictures easier to navigate.

I wonder if such a thing couldn’t be done with terrorism?  Certainly there are dozens (probably hundreds) of agencies with some stake counter terrorism in the United States and there are far fewer terrorism incidents than gun crimes so the task shouldn’t be too onerous and you could mandate participation on a federal level or (perhaps even better) make it a condition to receive federal grants.  You could work out some formula based upon the number of personnel in an agency, the scope of their counter-terrorism mandate and make a determination of how many records they should collate.  Each record would be collated by multiple people and either accepted (in cases of significant consensus) or review by some panel when it’s not clear what the community view is.


  1. Well, close enough for purposes of this discussion
  2. Gotta keep those trolls at bay.

Can horror teach us anything about the future of terrorism?

Recently, I wrote about what I see as the futility of trying to predict and prevent the last attack.  I’d like to riff off of a portion of that where I discussed an assumption about terrorist priorities.  I wrote:

In the wake of the Boston bombing there was an assumption 1, that terrorists would seek out events where media was present so that the attack could be broadcast live.

It’s not entirely clear to me that having wall to wall coverage is the best way to accomplish what terrorists have in mind:  Notably…to spread terror. The footage of the Boston bombing was horrific but in terms of spreading terror, I think a case could be made that terror is more effectively spread when you don’t have complete coverage of the event.  Think about the discussions about what went on in the Westgate mall in Kenya.  Video of the terrorists going around asking people to recite passages of the Koran and shooting those who couldn’t might make the rounds of the seedier side of the internet but the stories can travel even faster.

And, as the writers and producers of horror can tell you, the greatest horror is that which occurs in the imagination of the listener/viewer.  The tale can take a life of its own, with each retelling, getting more horrific every time, allowing each listener to fill in the blanks of the recounting with their own worst fears.  So, if you’re a terrorist, wall to wall media coverage might be fine but so might a less conventional broadcasting of your attack.  One that promises to take on a life of its own.

And in a post 9/11 world where we’ve seen airplanes fly into buildings, maybe there’s not too much more to be wrung out of straight TV reporting.  Perhaps the next level of terrorism (at least for those who don’t have the ability to conduct similarly spectacular attacks) is to not have perfect reporting on an attack.  To leave enough gaps in the ‘story’ to allow terror to spread through misinformation and exaggeration.

Terrorists 2 won’t want less coverage of an attack but, I think they very much would like fewer documented (i.e. visual) details and a whole lot more speculation.

Out notion, therefore, of what a ‘spectacular attack’ is may have to change in the future.

  1. to be fair, this was an assumption that was much older than Boston but that attack was seen as confirming evidence
  2. And here I’m talking terrorists who have the sort of apocalyptic viewpoint like we’ve seen by al-Qaida and its affiliates.  It may go broader than that but I haven’t thought that through

Droning on about top 10 lists…

Mark Bowden has an interesting article about drones in the September issue of the Atlantic.  Specifically, I’d like to recommend the portion of the article that talks about target selection and approval.

I want to write about one brief, almost innocuous, passage in that portion of he article and how it applies to the intelligence process more broadly. In talking about the effectiveness of drones (and other means) to kill al-Qaida leadership, Bowden makes the point that drone strikes have declined in numbers.  Quoting a ‘senior White House official’ he writes:

The reduction in strikes is “something that the president directed.  We don’t need a top-20 list.  We don’t need to find 20 if there are only 10.  We’ve gotten out of the business of maintaining a number as an end in itself, so therefore that number has gone done.

I remain both amused and concerned at the number of times I see or hear about ‘top 10′ lists.  I get it we’re a base ten species.  But really, do we need to treat our counter-terrorism efforts the same way we treat a David Letterman monologue or a Buzzfeed article?   The fact that we rely so heavily on the idea of ‘top ten’ can seriously distort our understanding of the environment.

For several years I used to work on assessments of criminal street gangs and I would often get requests for the ‘top ten’ gang threats.  Sometimes the two or three ‘most serious’ gang threats (those that were the largest or most prone to violent activity, for example) would so eclipse the others that it just made no sense to include others in the same list.  The whole process was unhelpful, especially since few people would spend much time on anything other than whatever was #1 on the list.

And take counter-terrorism.  A reliance on something like ‘top 10′ threats to the U.S. implies that there are 10 threats to the country that deserve consideration.  Maybe there are 4…maybe there are 14.  It seems to me that the rational thing to do is determine criteria for what’s important and then figure out how many (or few) subjects fit that criteria.  An alternate way to go would be to identify how many threats you have the resources to address (‘We can conduct 3 investigations simultaneously.’) and then determine criteria that will identify the three most important subjects.

If we assume that threat is made up of intent plus capability what shouldn’t our priorities include the same components?  Our intent may be to eliminate all terrorism from the face of the Earth but our capabilities are far should of that so…bring them in line and get on with it.

In any case, arbitrarily asking for ‘top 10′ lists doesn’t do much of anything.  It doesn’t even give us a workable number to evaluate priorities if cognitive science to be believed.  In the Psychology of Intelligence Analysis, Richards Heuer asserts that the human mind can only juggle between seven and nine facts or bits of information at one time.  There’s been some research that indicates that was a very optimistic estimate and the real number is half that.

Top 10 lists are intellectual crutches that allow someone (the tasker…the analyst…whoever) to avoid making decisions about what’s important.  Rather than determining criteria for inclusion or exclusion, we just punt and say ‘Give the the top 10′.  And what do we do with that top 10?  How much consideration does #7 get?  Don’t most customers really spend their time looking at number 1 or 2?

So, what’s an analyst to do when asked to put together some sort of top 10 list?  Well, I think there are two ways to go about tackling this.  The first would be to develop ‘inclusion criteria’ of what it would take to make it on any list and run that by whomever created the tasking…without telling them that this might mean that more or fewer entities might make the cut.  My experience is that if you introduce that possibility too early the response you’ll get is something along the lines of ‘That’s great…but you’re going to end up with 10, right?”

You’ll want to wait until the project is well along…ideally close to being completed before introducing the possibility that your list might not hit upon that nice, round number that everyone seems to love.


Once you’ve got your criteria, the entities you’ve determined are worthy of consideration will (probably) either be less then or greater than the magic number you were assigned to cram into a list.  If it’s less and you’re still *ahem* encouraged to beef up your list to a magical number, I’d recommend using images and language throughout your document to make it clear which items on your list are not worth consideration.  Images can be quite effective in this regard and hopefully, even your overlord will, upon review, realize that including extraneous entities undermines the credibility of your project.

If you have more entities than the magic number you may be encouraged to arbitrarily create some cut off mark.  You could try to retrofit your criteria in order to do so, which may be your safest bet since it will allow you point out what is being eliminated and allowing your overlords to have the queasy feeling of wondering if eliminating terrorist group B from the list is a good idea just because they feel a bit short of their annual funding goal.

The bottom line is intelligence is about telling your customer (whether that’s a patrol cop of the President of the U.S.) what they need to know, regardless of if what they need to know if 2 or 22 things.  Don’t get sucked into cultural idioms if they don’t advance the goal of providing clear, concise, relevant information in a timely manner.