Tag Archives: Intelligence

Can you crowdsourse the collation function?

The collation part of the intelligence cycle (if, indeed, it is a cycle) is the least appreciated and ‘sexy’ part of the whole thing.

Defined, collation (sometimes also called ‘processing’ or considered a subset of that word) is:

Once the collection plan is executed and information arrives, it is processed for exploitation. This involves the translation of raw intelligence materials from a foreign language, evaluation of relevance and reliability, and collation of the raw intelligence in preparation for exploitation. 1

In short, its converting the varied inputs of intelligence (interviews, bank records, photographs, etc.) into forms of data that allow the information to be analyzed.  Generally, it sucks.  Converting or coding one set of data, for example, into spreadsheets, databases, etc. It’s also very time consuming and can be fraught with errors.

It may interest you to know that in the law enforcement and homeland security fields there is a real dearth of standardized datasets from which to draw to assist with analysis.  Want to know if the use of explosive devices by extremists is on the increase or decline?  Well, it depends on whose data you use and none of it is great or universally accepted.

With regards to terrorism studies, START‘s Global Terrorism Database is among the best but it has problems associated with it as well.  These aren’t really defects in the START folks (they’re very nice and extremely helpful) but rather because, in part, of problems with collations.  A few examples…

Let’s say you want to capture terrorist plots and attacks in the United States.  What would you count?  When does a plot go from some knucklehead blowing off steam and talking nonsense to a real threat?  Should we even make that distinction? Is it enough to say ‘I want to attack the United States’ or would the individual need to identify a target?

Many researchers try to sidestep this issue by trying to use the criminal justice system to do their work for them.  It’s easy to count people who have been charged under terrorism statutes or convicted of terrorist crimes.

That system leaves a lot to be desires.  What happens is charges are dropped or over turned?  Many times, a terrorism arrest is announced to much fanfare only to have most of the substantive charges dropped for one reason or another. Then you over count the plots and attacks.  What if, for tactical prosecutorial reasons, the suspect is charged with something other than terrorism offenses?  Then you risk under counting plots and attacks.

This is a real issue and I’d point you to this group that was arrested in Fort Stewart, Georgia a couple of years ago as evidence of that.  A number of current and former soldiers with plans to attack an Army base, assassinate the President and with two (and possibly more) homicides and other crimes under their belt.  Never heard of this terrorist threat? I’m not surprised as they weren’t charged under a terrorist statute.  So, they fall into the ‘general crime’ category.  Clearly, (assuming the initial reports were correct) these are folks you’d want included in any analysis of the terrorist threat.

Unfortunately, this is too big of a job for state and local agencies to tackle and it’s not clear if a federal agency (the FBI or DHS probably) has the interest or will to do this.  Further, if you want widespread acceptance, such a task is probably undertaken with broad buy-in, transparency and peer review.

These sorts of things have been bumping around in my mind for awhile when I saw this story from Slate.  After the Newtown shooting, they started documenting all the firearms deaths in the United States.  Now, one year in, they’ve got more than 10,000 reports of gun deaths and, in order to analyze them, have to….(wait for it…)….collate all that data.

Rather than trying to do it in-house or hire someone to do it (or foist it off on some poor interns) they decided to try crowd-sourcing it.

Here’s how it works.  They provide you with an article and the name of a victim about a gun death.  You read it and then select if the death fits within one of the following categories:

  • murder
  • suicide
  • accident
  • shot by law enforcement
  • shot by civilian in self defence
  • other
  • link not working

The nice thing about it is, apparently, it focuses on the victim rather than just the article.  So, I suppose an article where two people died (let’s say a murder/suicide) one person may get asked about the suicide and the other the murder.  Also, they don’t just rely on one person’s response 2 but rather on a consensus of some number of respondents.  That’s a nice check on work, but as we all know, the majority isn’t always right.

Now, they did recently identify a significant flaw in their methodology.  While their data collection was exclusively based on open source reporting it quickly became clear that suicide by gun is under-reported.  On those occasions when suicides are reported (not very often) it’s almost unheard of for the report to describe the method of death.  Still, like in most things, admitting you’ve got a problem is the first step in fixing it.

Another example is through the British Library.

In 2008, the British Library, in partnership with Microsoft, embarked on a project to digitize thousands of out-of-copyright books from the 17th, 18th, and 19th centuries. Included within those books were maps, diagrams, illustrations, photographs, and more. The Library has uploaded more than a million of them onto Flickr and released them into the public domain. It’s now asking for help.

Next year, it plans to launch a crowdsourced application to fill the gap, to enable humans to describe the images. This information will then be used to train an automated classifier that will be run against the entire corpus.

The library is also soliciting ideas for how to present the collection to aid the tagging and metadata generation, and also make the pictures easier to navigate.

I wonder if such a thing couldn’t be done with terrorism?  Certainly there are dozens (probably hundreds) of agencies with some stake counter terrorism in the United States and there are far fewer terrorism incidents than gun crimes so the task shouldn’t be too onerous and you could mandate participation on a federal level or (perhaps even better) make it a condition to receive federal grants.  You could work out some formula based upon the number of personnel in an agency, the scope of their counter-terrorism mandate and make a determination of how many records they should collate.  Each record would be collated by multiple people and either accepted (in cases of significant consensus) or review by some panel when it’s not clear what the community view is.


  1. Well, close enough for purposes of this discussion
  2. Gotta keep those trolls at bay.

Droning on about top 10 lists…

Mark Bowden has an interesting article about drones in the September issue of the Atlantic.  Specifically, I’d like to recommend the portion of the article that talks about target selection and approval.

I want to write about one brief, almost innocuous, passage in that portion of he article and how it applies to the intelligence process more broadly. In talking about the effectiveness of drones (and other means) to kill al-Qaida leadership, Bowden makes the point that drone strikes have declined in numbers.  Quoting a ‘senior White House official’ he writes:

The reduction in strikes is “something that the president directed.  We don’t need a top-20 list.  We don’t need to find 20 if there are only 10.  We’ve gotten out of the business of maintaining a number as an end in itself, so therefore that number has gone done.

I remain both amused and concerned at the number of times I see or hear about ‘top 10′ lists.  I get it we’re a base ten species.  But really, do we need to treat our counter-terrorism efforts the same way we treat a David Letterman monologue or a Buzzfeed article?   The fact that we rely so heavily on the idea of ‘top ten’ can seriously distort our understanding of the environment.

For several years I used to work on assessments of criminal street gangs and I would often get requests for the ‘top ten’ gang threats.  Sometimes the two or three ‘most serious’ gang threats (those that were the largest or most prone to violent activity, for example) would so eclipse the others that it just made no sense to include others in the same list.  The whole process was unhelpful, especially since few people would spend much time on anything other than whatever was #1 on the list.

And take counter-terrorism.  A reliance on something like ‘top 10′ threats to the U.S. implies that there are 10 threats to the country that deserve consideration.  Maybe there are 4…maybe there are 14.  It seems to me that the rational thing to do is determine criteria for what’s important and then figure out how many (or few) subjects fit that criteria.  An alternate way to go would be to identify how many threats you have the resources to address (‘We can conduct 3 investigations simultaneously.’) and then determine criteria that will identify the three most important subjects.

If we assume that threat is made up of intent plus capability what shouldn’t our priorities include the same components?  Our intent may be to eliminate all terrorism from the face of the Earth but our capabilities are far should of that so…bring them in line and get on with it.

In any case, arbitrarily asking for ‘top 10′ lists doesn’t do much of anything.  It doesn’t even give us a workable number to evaluate priorities if cognitive science to be believed.  In the Psychology of Intelligence Analysis, Richards Heuer asserts that the human mind can only juggle between seven and nine facts or bits of information at one time.  There’s been some research that indicates that was a very optimistic estimate and the real number is half that.

Top 10 lists are intellectual crutches that allow someone (the tasker…the analyst…whoever) to avoid making decisions about what’s important.  Rather than determining criteria for inclusion or exclusion, we just punt and say ‘Give the the top 10′.  And what do we do with that top 10?  How much consideration does #7 get?  Don’t most customers really spend their time looking at number 1 or 2?

So, what’s an analyst to do when asked to put together some sort of top 10 list?  Well, I think there are two ways to go about tackling this.  The first would be to develop ‘inclusion criteria’ of what it would take to make it on any list and run that by whomever created the tasking…without telling them that this might mean that more or fewer entities might make the cut.  My experience is that if you introduce that possibility too early the response you’ll get is something along the lines of ‘That’s great…but you’re going to end up with 10, right?”

You’ll want to wait until the project is well along…ideally close to being completed before introducing the possibility that your list might not hit upon that nice, round number that everyone seems to love.


Once you’ve got your criteria, the entities you’ve determined are worthy of consideration will (probably) either be less then or greater than the magic number you were assigned to cram into a list.  If it’s less and you’re still *ahem* encouraged to beef up your list to a magical number, I’d recommend using images and language throughout your document to make it clear which items on your list are not worth consideration.  Images can be quite effective in this regard and hopefully, even your overlord will, upon review, realize that including extraneous entities undermines the credibility of your project.

If you have more entities than the magic number you may be encouraged to arbitrarily create some cut off mark.  You could try to retrofit your criteria in order to do so, which may be your safest bet since it will allow you point out what is being eliminated and allowing your overlords to have the queasy feeling of wondering if eliminating terrorist group B from the list is a good idea just because they feel a bit short of their annual funding goal.

The bottom line is intelligence is about telling your customer (whether that’s a patrol cop of the President of the U.S.) what they need to know, regardless of if what they need to know if 2 or 22 things.  Don’t get sucked into cultural idioms if they don’t advance the goal of providing clear, concise, relevant information in a timely manner.

Chasing the last target

It’s a cliche that armies prepare to fight the last war but the same can be said for counter-terrorism agencies.

Take, for example, the recent attack in Kenya.  The Somalia based group al-Shabbab conducted an extended attack in a shopping mall there that resulted in dozens of deaths and went on for several days.

In the aftermath, one could hardly turn around without being bombarded with ‘experts’ claiming this was the ‘new face of terror’ and that we needed to be concerned about new attacks occurring at malls around the world….even in our own backyards!

And yet…I can’t help noting that I heard the same think before:

  • in 2008 after the Mumbai attacks (complex attacks are the wave of the future!)
  • in 2011 after the attack in Afghanistan at the Intercontinental Hotel (Look out!  Hotels are the next big target!)
  • in 2013 after the Boston bombings (No 5k fun run will be safe!)

Rather than look at the specific target (hotels, runs, malls, etc.) it seems we’d be better served by trying to identify the characteristics of each of these that made them attractive to terrorists and then see if there are commonalities between them.  Those commonalities may apply to only one terrorist group or cut across organizational and ideological lines.  It may apply only to certain times or be ‘eternal’ or vary geographically.

Off the top of my head, I can certainly think of some commonalities between these attacks (although, we really should be trying to look at a much larger dataset).  All of these targets attracted large numbers of people.  But not just any people…we can probably consider these people (at least loosely) as ‘elites’.  Even in the Boston Marathon, attendees at the finish line were likely to include sponsors, officials and a high concentration of people who’s death would likely evoke a reaction from those in power.

One thing I suspect we don’t see (despite what more than a few pundits claimed in the wake of Boston) is any relationship to the target being one with a media presence.  In the wake of the Boston bombing there was an assumption 1, that terrorists would seek out events where media was present so that the attack could be broadcast live.

That, in my opinion, reflect a fundamental misunderstanding of both technology today and terrorist motivations.  If your daily interactions with Twitter, YouTube and Facebook weren’t enough proof that media presence isn’t necessary to carry word about a terrorist attack, then the killing of Lee Rigby should be.  The plethora smartphones, wi-fi and social media mean that anyone can upload audio, video or pictures and, when combined with the hungry maw of the 24 hour news cycle,  there’s a good chance that dramatic footage will find its way to the media in any case.

And regarding terrorist motivations, I think there’s something to be said for less complete media coverage actually enhancing the impact of many terrorist incidents.  Not less total coverage mind you but rather less visual evidence.  But, more on that in a future post.

In short, I’m just not convinced that the location of TV cameras or a local news bureau is particularly important or, at least, not nearly as important as it was even 10 years ago.

But, assuming the next target will be exactly like the last just because the last occurred is the epitome of bad thinking based, as near as I can tell, on nothing more than a gut feeling.  This is a common error where people try to diving the future from data they have rather than the data they need.  With all due respect to a former Secretary of Defense, that’s not good intelligence practice. In situations like this you have two ‘good’ choices and (at least) one bad one.  The two good choices are:

  1. identify the data you need to collect to answer your intelligence questions and collect against it, or:
  2. admit you don’t know what data you need OR can not get it and leave that questions explicitly unanswered.

I list the second option as ‘good’ because at the very least you are clear about what you don’t know.

The bad decision (which I see all too frequently) is the one that someone tries to arrive at a decision with irrelevant or inappropriate information.  Usually this occurs because someone in the chain of command demands an answer and the organizational structure isn’t set up for a ‘We don’t know’ answer.  As a result, the desire to provide something (anything) can be great, usually too great to overcome.

Not exactly the way you want to direct your counter-terrorism assets.


  1. to be fair, this was an assumption that was much older than Boston but that attack was seen as confirming evidence

Crowdsourcing notes

In the wake of the Boston bombings there’s been numerous discussions about the pros and cons of ‘crowd-sourcing‘ intelligence.  In the most basic sense, the whole idea of soliciting information from the public is not new.  Most Wanted lists are primitive examples of crowd-sourcing but we’re talking about the modern use of the term here.  In that regard, Boston took a bit of the luster off of the idea of crowd-sourcing by wasting a whole bunch of time and effort going down dead ends (not really a big deal) and fingering innocent people (yeah, a big deal).

Arms Control Wonk takes a look at lessons learned from a slightly different version of crowd-sourcing.   Looking at the website Tomnod which organizes crowd-sourcing around projects involving satellite/aerial imagery or twitter.

I suspect the critical part of successful crowd-sourcing is providing some sort of super-structure by which people can contribute and towards specific questions.  This New Yorker article seems to be circling around that point here).

In any case, Tomnod identified several potential concerns that could scuttle a crowd-sourced project:

  • Incentivizing people to participate – Always a major concern with projects like this.  Just because you build it, doesn’t mean they’ll come.
  • Calling on the public to become whistleblowers might inadvertently put private citizens in harm’s way – Hopefully not a particularly relevant concern when looking at domestic crowd-sourcing but there are some instances when it might be relevant (for example, reporting on crime).
  • How to filter out the users who enjoy tweeting false alarms of an incoming North Korean missile attack from those who may have actually sighted something useful.

Check out the original article for a discussion about how Tomnod went about controlling for those.

The Worldwide Threat Assessment

The Office of the Director of National Intelligence has continued the most excellent tradition of releasing an unclassified threat assessment. I can’t describe what a good initiative this is as it is a big step in providing transparency into the thinking of Intelligence Community. While something like this could certainly be manipulated, it is at least one check against the powers that be cherry picking intelligence. By publicly proclaiming their analysis (broadly) to the world, the Intelligence Community have their colors nailed to the mast and face the consequences (good or bad) of their work.

I’m particularly happy with this edition of the assessment for a couple of reasons.

First, the report starts off with a discussion of the cyber threat. I am by no means a tech geek and I agree that there are many serious vulnerabilities to our electronic data and networks but I’ve been concerned by the increasingly hysterically drum beat of talks about cyber ‘Peal Harbors’, 9/11s or Kratatoas. Coupled with the ominous ‘It’s not a matter of if but when…’ statements by very serious people, these predictions would seem like a slam dunk. 1

So, I was glad to see this:

We judge that there is a remote chance of a major cyber attack against the US critical infrastructure systems during the next two years that would result in long-term, wide-scale disruption of services, such as a regional power outage.

First take away: ‘Chill the fuck out.’

Those that can carry out those big attacks are going to be nation states and countries (even ones like Iran) are going to be reluctant to do so because a cyber attack today can lead to a physical retaliation tomorrow. Just because we now pay our bills online doesn’t mean the idea of retaliation has gone away.

But, what about those crazy terrorists? They’re already under the gun (so to speak) so threats of retaliation aren’t going to mean much to them, right? Well, true, but there are a couple of reasons why we might not have to worry too much about that. First, most ‘traditional’ terrorists are still thinking in terms of traditional fighting. Taking down a multinational corporation or a power station might be a significant win for these players and a step in their ultimate plans but just about all of them have goals of temporal power and for that you need to get our from behind the keyboard and pick up a gun. That’s the first part…the second is that it doesn’t look like any such groups have the capability to conduct such attacks.

Instead of the risk coming from well planned and executed attacks over the next two years, the assessment says that our vulnerability to less sophisticated attacks having an unexpected result because of particular ‘system configurations and mistakes’. In other words, our crappy system designs might go loopy. Just like the HAL9000 in 2001.

YouTube Preview Image

Al-Qaida continues their downward spiral with the various affiliates being either concerned more with local matters (AQIM, AQI, al-Shabaab, etc.) or just suffering from a long string of general ass-kickings (AQ Central). Not much to say there other than to say that my personal opinion is that history is passing them by. They still might be able to launch attacks over the coming years but both as a terrorist organization and a broader movement, they’re starting to look like they’re past their expiration date. The thing to look out for will be those who came to political maturity over the past decade. Just as AQ is really the function of the cohort that came out of the Soviet invasion of Afghanistan, how will the next generation of…who? Islamic radicals? The global ‘have-nots’? Others communities we aren’t even aware of yet? Anyway, how will these people internalize these events (and then externalize them!)?

My prediction: I suspect we’re in a terrorism ‘lull’ that probably began in 2009 or so 2 and will go on for at least another year or two. Now, a ‘lull’ does NOT mean there won’t be any terrorism activity occurring around the world. I think, instead, that we’ll see a general retreat of international terrorist organizations from targeting the United States. Targeting U.S. interests (like the Benghazi attack which, contrary to Fox News, Sen. McCain, et. al. was NOT a sign of a resurgent al-Qaida) will probably continue but occur in places that lack adequate security (post-revolutionary Libya, Bulgaria, etc.). But attacks like 9/11, or even 7/7 3, are going to continue to recede into the realm of hypotheticals dusted off to protect pet funding projects.

I suspect 9/11 will eventually be seen as al-Qaida’s battle of Cannae. A brilliant tactical success but with no real strategic benefits for them. Yes, 9/11 did cause the U.S. to make a number of really, really (really) bad decisions that didn’t do them any favors but they didn’t really translate into any sort of net gain for al-Qaida. Maybe bin Laden’s strategy of using a 50 or 100 year time scale will prove him right but I suspect not. Trying to game the global system decades in advance is virtually impossible (at least now). If we weren’t able to do it when we were the global hegemon how can al-Qaida expect to? Indeed, they had better have a deity on their side as that’s what I think it will take.

I was also pleased to see an acknowledgment of the negative influence of environmental crime around the world:

[Environmental crime] constitutes a multi-billion dollar industry annually, endangers the environment, and threatens to disrupt the rule of law in important countries around the world. These criminal activities are often part of larger illicit trade networks linking disparate actors-from government and military personnel to members of insurgent groups and translational organized crime organizations.

One reason such crimes are so lucrative is that the enforcement and penalty mechanisms are so lax.

The really frightening part of the assessment is the section titled ‘Natural Resources: Insecurity and Competition’. It presents a bleak picture of the state of food, water and natural resources now in which many stocks are at or near capacity with little margin for unexpected (or, in some cases, expected) shocks to the system. So, who cares if a bunch of peasants in central Africa can’t get wheat? Well, what would you do if all the supermarkets ran out of food? And what if the next town over had full supermarkets but either charged all outsiders a huge markup or just didn’t let outsiders in at all, deciding to save food for their citizens? Things could get ugly really fast. I believe it was Isaac Asimov who said something like ‘No civilization is more than three missed meals from anarchy.’

The other big problem with this issue is that it’s just not a national (or international) priority. In a country where we’re still debating whether global warming is an insidious crypto-Socialist-academic plot to deprive honest, hard-working citizens of the ‘American Dream’, who in the hell is going to get support to try to address the question of food security? You can almost hear the tin foil hat brigade talking about how the market will solve everyone’s problems. Make not mistake, these are big problems that are going to create all sorts of new, more traditional problems before everyone realizes just how bad things are. The fact that us humans just aren’t that good at dealing with threats like this doesn’t fill me with a great deal of optimism.

But in the short term (perhaps the next year or three) things don’t look too bad, relatively speaking. As the assessment runs down the various regions of the world we continue to see instability (Africa), slides towards authoritarianism (Central Asia), rampant crime (Central/South America) and emerging rivals (China) but none of these things seem ready to drive the international system into crisis. The more I read these things the more I find myself thinking of the later Roman Empire. Not as a direct correlation but rather the sense of the system gradually falling apart while those at the center seem unable to focus on anything except trivial matters. Shocks to the system are dealt with in a temporary and ad hoc manner. Hopefully that impression is just my age and cynicism catching up to me. I suppose we’ll see…

  1. Of course, we should remember we’ve heard this about waves of suicide bombings, Iraqi WMD threats, ‘dirty bombs’, etc., etc., etc. So, take it for what it’s been worth thus far…
  2. But, I think it’d be safe to mark the death of Osama bin Laden as the ‘official’ beginning of the lull, even though that was in 2011
  3. Again, I’m talking about in the U.S.

On glass houses, rocks and homeland security.

The Daily Beast has an article up about intelligence reports from DHS and their faults.

Since 2003, the office has published more than 21,000 intelligence reports, averaging about 300 a month in recent years. Because of the widespread distribution, and because of past controversies like a 2009 report on right-wing extremism that irked Congress and veterans, the reports are typically stripped of sensitive detail.

Hey, no one dislikes fluff posing as ‘analysis’ more than me but I find it odd how the authors get an interesting mix of people to pile on the DHS.  Like this guy:

Mike Sena, president of the National Fusion Center Association, which represents 72 counterterrorism and crime centers around the country, says the Homeland office has improved somewhat over the years but still puts out information far too late. “If you’re accurate but not timely, it’s pretty much worthless,” Sena says.

<insert spit take here>

Are you kidding me?  Fusion center people are saying that someone else’s products are superficial and lacking in timely, specific and relevant information?  That would be funny if it wasn’t so sad.  Allow me to just say that in my experience, the DHS is held up to be the model that fusion centers are supposed to follow (and generally fall short of).

Fusion centers often are nothing more than glorified back-patting operations (with a robust data entry/retrieval capability) with little concept of intelligence analysis.

What the DHS (and virtually everyone else) is guilty of is engaging in endless CYA work under the guise of analysis.  Writing products for every two bit car show, football game and parade mentioning that someone, somewhere might want to attack it and then following it up with those immortal words:

““The <insert agency name(s) here> have identified no credible terrorist threats to the <insert event(s) here>”

To which one should ask:  “Well, then why the hell did you waste my time with this document?”

To which the answer is twofold:

  1. In case there ever is an attack, someone wants to be able to pull out a report like this and yell ‘See!  We told you! It wasn’t our fault!  Honest!’
  2. These agencies continue to be evaluated on the quantity of their output rather than their quality.  No one may read them.  No one may like them.  But, when budget time rolls around everyone wants to hear how many of these things have been published.

So, with all due respect to the National Fusion Center Association, perhaps they should spend a bit more time getting their house in order before throwing rocks at the other glass buildings on the block.


The Metaphor Program

The Atlantic has a story that IARPA (DARPA‘s smarter, yet nerdier younger brother) is soliciting bids for work on a project they’re calling “The Metaphor Program”.

“The Metaphor Program will exploit the fact that metaphors are pervasive in everyday talk and reveal the underlying beliefs and worldviews of members of a culture,” declared an open solicitation for researchers released last week.

There’s been some evidence recently released in the popular press (not coincidentally, I’m sure) about a study in which two groups of students were given a description of a city with a crime problem.  All the facts and figures were the same but what was different was in the way the problem was described.

…the researchers asked 482 students to read one of two reports about crime in the City of Addison. Later, they had to suggest solutions for the problem. In the first report, crime was described as a “wild beast preying on the city” and “lurking in neighborhoods”.

After reading these words, 75% of the students put forward solutions that involved enforcement or punishment, such as building more jails or even calling in the military for help. Only 25% suggested social reforms such as fixing the economy, improving education or providing better health care. The second report was exactly the same, except it described crime as a “virus infecting the city” and “plaguing” communities. After reading this version, only 56% opted for great law enforcement, while 44% suggested social reforms.

There are a couple of implications here for intelligence work.

First: The words used in how problems and information are presented can have a significant impact on production.  The words used in an intelligence product can have a significant impact on decision makers and policy. That impact may be intentional or not but we should be aware of it.

As Lunghu will point out, this can be a good thing.  If an analyst is doing her job, she will identify not only the information needs of her customer (and possibly secondary customers) but also the best way to ensure that message resonates with that customer.  Using appropriate and persuasive language is critical in doing so.

I, on the other hand, see a very dangerous cloud on the horizon.  While, in principle, I think Lunghu is correct I think the law enforcement community is clearly no where near ready to wield such power.  Speaking generally, analysts are not sufficiently independent from their organizational overlords to resist calls to ‘make the facts fit our position’.  Analysts (and those working in or supervising intelligence shops) are not trained specifically in the ethical quandaries of such work or effective writing.   Perhaps worst, there is no mechanism for evaluating, reviewing, commenting and/or rebutting intelligence products.  They get released into the ether and become immortal.

Therefore, what little training analysts get (IMO) needs to be focused on objectivity and logical arguments.  I would very much like to reach Lunghu’s Utopia but until then want to make sure our journey to get there doesn’t bring us to Gehenna.

In short, the community just isn’t ready to tackle this yet and has some preliminary steps to take before it will be.

Second: The study of language (and more broadly culture) isn’t a one way street.  Our culture and language may prevent us from understanding and appropriately reacting to others.

Allow me here to once again plug the book Imperial Secrets which addresses a number of these issues.

I’m really tempted to quote myself now but hesitate because I’ve raked people over the coals for doing it themselves.  Oh, screw it…(although I really recommend you read the two original posts in their entirety because there are a whole bunch of ideas worth considering.

As the British empire in India expanded, it encompassed an increasing expanse of human and physical terrain which it did not fundamentally understand, prompting the series of “information panics” which characterized the British experience in the sub-continent. pg. 30

Information panics.  Think about how many of those we’ve had in the past, leaving aside the question of terrorism.  The RIAA has been an almost permanent freak out since the introduction of Napster.  Concerns about predators on the internet.  Youth sub cultures (goth, hip-hop, metal).  Immigrants.  Religion.  As all of these issues have been absorbed in our ‘imperial’ sphere of interest they’ve generally been ignored until some trigger (either real or contrived) caused them to leap to the public consciousness and lead to a bunch of flapping about, chicken little style.  Some of these are overblown and others (going back to terrorism – the rise of violent Islamism) aren’t but what these all have in common is that they generally weren’t within the scope of what the ‘imperial power’ considered to be valuable knowledge.

British attempts to rationalize Hindu and Islamic law in India, for example, fundamentally contorted the material at hand in order to make it intelligible for officials operating from an English common-law background. Their consequent understanding misread what their subjects experienced and expected, with far-reaching ramifications for Anglo-Indian relations and the experience of religiously defined identity in South Asia. pg 10

This is a concept I still don’t think we fundamentally understand.  Before the invasion of Iraq, conventional wisdom was that Saddam’s refusal to allow wide ranging weapons inspectors in was proof that he was hiding stockpiles of WMD.  It wasn’t until later that it became clear that at least part of his motivation in keeping the existence of a WMD program ambiguous was to deter Iranian hostility.  The mind boggles at the number of times we’ve made errors both tactical and strategic because we don’t understand that not everyone shares the same reality we do.  For Americans, I suspect that even the majority of them that claim to be religious don’t experience the concept of God in the same way that people might in the NWFP or in Ghana, for example.

As another potential example I’ll submit this to you from Balko.  We can get trapped by our own language as well.  So, when we describe things as a war, should we be surprised when our solutions look distinctly warlike?


What does Stan have to say about law enforcement intelligence?

Gen. Stanley McChrystal has an article in Foreign Policy about the process by which he came to understand what’s involved in a fight with a network (as opposed to a structured, hierarchical opponent).

There’s nothing really new here (the theme is:  It takes a network to defeat a network – something people have been saying for years) but I think this is a worthwhile read because a lot of people in the law enforcement community could learn a thing or two when thinking about the role of intelligence, proactive operations and information operations with regards to criminal networks.

In fact, I think you could almost do a ‘search and replace’ feature in this article removing ‘military’, ‘Taliban’ and such terms with ‘law enforcement’, criminal networks’, etc.  Check it out:

…fashioning ourselves to counter our enemy’s network was easier said than done, especially because it took time to learn what, exactly, made a network different…an effective network involves much more than relaying data. A true network starts with robust communications connectivity, but also leverages physical and cultural proximity, shared purpose, established decision-making processes, personal relationships, and trust. Ultimately, a network is defined by how well it allows its members to see, decide, and effectively act. But transforming a traditional military structure into a truly flexible, empowered network is a difficult process.

Here’s the big part…pay attention:

Our first attempt at a network was to physically create one. …Operators and analysts from multiple units and agencies sat side by side as we sought to fuse our intelligence and operations efforts — and our cultures — into a unified effort.

It was clear, though, that in this fusion process we had created only a partial network: Each agency or operation had a representative in the tent, but that was not enough. The network needed to expand to include everyone relevant who was operating within the battlespace. Incomplete or unconnected networks can give the illusion of effectiveness, but are like finely crafted gears whose movement drives no other gears.  [emphasis added]

And yet, in law enforcement today, with our many, many fusion centers the physical part and ‘information sharing’ occupy the lion’s share of the effort.  It’s about the only thing anyone in charge of such organizations can talk about or point to in terms of ‘progress’.  Collecting people (who are anything but diverse) in one place and sending (hopefully relevant but all too frequently not) information out to ever growing lists of people who, if you talk to them over a beer, will tell you that they usually just throw it in the trash without reading it is not success.

Well, it is success because those cater to the metrics in place to determine if things are working right.

That’s unacceptable given it’s 2011.

Other characteristics deemed critical:

  • Decisions were decentralized and cut laterally across the organization.
  • The network expanded to include more groups, including unconventional actors.
  • It sought a clear and evolving definition of the problem and constantly self-analyzed, revisiting its structure, aims, and processes…

Yeah, we aren’t doing those either.  Those are just not things you’re seeing in our fusion centers or ‘intelligence led’ agencies.

Stan’s a smart dude.  Wicked smart.  Last month he gave a talk on a similar subject.  He could (should) be talking to most of the fusion center people around the country:

“You don’t give a senior leader a Blackberry or an iPhone and make them a digital leader,” McChrystal said. Today’s commanders might spend endless hours on video conferences talking to their subordinates around the world. But without a “shared consciousness and purpose” across team members who come from very different backgrounds, they might as well close their Skype windows.

At least they know what Skype is


Web 2.0 and intelligence analysis

I’m a HUGE fan of the potential of using collaborative tools (electronic and otherwise) to develop betting analytical products.  So, it was with great excitement that I read “Twitter, Facebook, and Ten Red Balloons: Social Network Problem Solving and Homeland Security” in the latest edition of Homeland Security Affairs.  From the abstract:

This essay, the winner of the Center for Homeland Defense and Security (CHDS) Essay Contest in 2010, looks at how homeland security could benefit from crowd-sourced applications accessed through social networking tools such as Twitter and Facebook.

Unfortunately, the essay suffers from a number of flaws that make its conclusions suspect.  I think it was a good effort and I wouldn’t go out of my way to criticize it here except I worry it may reflect some deep misunderstanding of the community about what makes makes such tools tick.  Such misunderstandings will (continue) to lead us down blind alleys.

The most basic misunderstanding the article makes is the nature of incentives and how they apply to collaborative efforts.  Christopher Ford, the author, says this about the lessons learned from a recent DARPA challenge to see how effective networking and crowd-sourcing can be:

…thousands of individuals can be
incentivized with a fairly small monetary

I think that’s way wrong.  I see no reason to believe that money is the motivator here.  After all, if you look at the biggest expenditures of effort in social networking (all those YouTube, Twitter, FaceBook, Wikipedia and Blog posts) there is NO financial component (and yes, I do know that some people are paid to blog or tweet but the vast majority of content has no financial reward).  Rather, people are incentivized for other, less tangible reasons.

If you want to go viral on a project handing out a couple of sesterces aren’t going to get you much.  If you’re looking for people to help you address a “myriad of homeland security applications, both physical and cyber.”  focusing on small monetary incentives ain’t going to hit your target demographic.

In all fairness, Ford does mention ‘social incentives’ but seems to focus on monetary ones.  Still, in either case he says the incentives ‘need not be robust’.  Again I disagree and will point to people who engage in developing user generated content OR participate in social activities (on the web or elsewhere).  Can we really say that the people who spend hours on things like Farmville not getting ‘robust’ social incentives?  If not, why the hell are they spending so much time on them?  I think Ford ultimately minimizes the potential value people see in activities like that.

The problem with here lies in how someone adopting Ford’s view is likely to set themselves up for failure from the start.  Where the question of incentives should be paramount if you think any old thing will amuse and enlist the masses you’re going to end up with a crappy challenge/website/app/whatever that no one wants to visit.

It’s ALL about the incentive and it better be really, really robust.

This is where the ideas of Jane McGonigal have more applicability.  See below and let’s talk about ‘need not be robust':

YouTube Preview Image

Beyond that I have strong suspicions about the application of collaborative efforts in our existing law enforcement, national or homeland security institutional structures.  What agency is going to allow problems or issues to be passed off to the hive mind?  These are people you have no authority over…who may decide they don’t like your silly challenge…who may do something totally different from what you intended.

If you want to tap into the hive mind because you think it’ll give you more accurate answers* to questions (wicked or mundane) that’s fine but you’ll need to accept a lot of inefficiency in the model.  As Ford himself admits:

The potential strength of the social networking model of problem solving lies in the sheer
volume of participants.

Well, if you’re counting a huge numbers of participants to solve each problem then you have an additional concern of the size of your participant pool.  How many people can you expect to devote their time to work on problems that are important to you?  It’s find when DARPA is the only game in town and they do one such event at a time but is it realistic to expect this is scalable?  Are you going to have agencies competing with each other for the subset of the population who would potentially be interested in such projects?  What about the other potential competitors for your participants time?  After all, there’s a pretty robust game market out there, citizen-scientists, and humble bloggers  like yours truly.

I really, really would like to see the intelligence community embrace collaborative methods for solving their problems and helping address their information needs.  It just seems to me that Chris Ford drank a bit too deeply of the Kool-Aid before he noticed all the bugs floating in the cup.

Tech in law enforcement intelligence

A couple of stories came out the other day about local law enforcement using ‘new’ technologies to help them be more effective.

The first is about Bethlehem, Pennsylvania and their use of a commercial service to map crime information (imaginatively called CrimeMapping.com).  This isn’t really new but major kudos to them for making the data publicly available.  Even better:

Users can set up alerts for specific addresses, maybe a home or a business, and select a radius, from 500 feet to 2 miles, and have crimes e-mailed to them as they are picked up in the daily data sweep.

I’m a bit dubious about the claims that the data, as presented in the website, really is a solving crimes and having a big impact on staffing decisions.  I certainly could be wrong but there just doesn’t seem to be the requisite level of detail to make those sorts of decisions.  But, there’s a reason why Bethlehem PD might be making the claim that the service has yielded early successes:

There’s more available beyond the website. Additional software and services can add unlimited data storage, allowing police to track and map all crime across years and years of history.

Crime heat mapping gives police an idea of where over time crime is more concentrated. Databases can link crimes, individuals, addresses and other information.

If you want those bells and whistles however, you’re gonna need to cough up almost $50K.  So, you might want to play up it’s value now to ask for more later.

Still, as I say, I like the website for what it is.  Easy to use, displays information pretty well.  Nice job overall and I’d certainly be interested to have it in my town.  It doesn’t have the ability to export data, do complex searches or give you much in the way of detailed reporting but perhaps that’s in the upgraded version or a law enforcement only version.

The second is about Phillipsburg, NJ which has instituted a tip line that people can text information to.  I’d be interested to know the success of things like this over the long term and imagine they’re only going to be successful if implemented in conjunction with a pretty aggressive public outreach program.  This is where community oriented policing would come in handy and I’m reminded of a story from the BBC a while ago where a British police officer handed out a number for people on his beat to text him with reports, tips, questions, etc.

So, even though I’m a bit skeptical about the long term success of such a program in isolation I give two big thumbs up to these departments for moving forward with citizen-participatory tech.

Good for them…