Tag Archives: Cognitive bias

What we have here…is a failure to communicate

I’ve become skeptical of a lot of Malcolm Gladwell’s stuff but this is a superb article about how easily the cognitive bias of ‘mirror imaging’ can have tragic consequences.

The siege of the Branch Davidian complex in 1993 was, according to Gladwell, doomed from the start because the F.B.I. and the Branch Davidians were simply occupying different realities.  The F.B.I. saw the incident as one led by a dangerous person (David Koresh) who was manipulating people through a veneer of religion.  In this construct, religion was just the tool Koresh needed wield power over his followers and to him is was purely utilitarian.  If espousing a political ideology or a different religion or belief in little green men would have gotten him to the same place he would be equally likely to adopt those beliefs in order to gain and keep his followers.

The Davidians, however, were true believers.  To them, they saw the physical proof of biblical prophecy all around them.  Talk of working things out, good plea deals or traditional hostage negotiation tactics just sounded like so much gibberish to them.  Didn’t the F.B.I. get that this wasn’t about some firearms charge?  This was about god.

Both parties were trying (and failing) to put themselves into the shoes of the other to understand them.  The F.B.I. operated from the perspective of ‘We’ve negotiated with criminals before….this guy is a criminal…therefore he’ll respond in a predictable way.’  The Davidians approached this from a religious point of view…’Anyone should be able to understand our position if they only understand scripture properly.’  Both of those assumptions were deeply flawed and in cases like this the resolution is inevitable.  The group with the better weapons will ‘win’.

But it didn’t necessarily need to end like this.  There were people who could do the ‘Davidian to F.B.I.’ translation.  That requires, however, at least one side to acknowledge that there might be another reality to translate from.  The F.B.I wasn’t willing to engage in that sort of thinking.  It’s easy to fault them for that but it’s not that easy.  Criminals conduct deception operations of greater or lesser sophistication all the time.  Law enforcement can’t indulge all of these attempts.  Still, it does appear that they lacked the ability to ever consider it.

How language affects analysis

A lot of things we have no control over have profound influences on how we interpret the world around us and, therefore, the way we can conduct analysis.  One of the most basic is the language we use.  The words we have at our disposal and the ways in which we can use them.  Two recent articles talk about just that phenomenon, plus also demonstrate another current lesson relevant to intelligence analysis.

First, from NPR, is this piece which talks about how language can affect how we perceive something like direction.

Lera Boroditsky once did a simple experiment: She asked people to close their eyes and point southeast. A room of distinguished professors in the U.S. pointed in almost every possible direction, whereas 5-year-old Australian aboriginal girls always got it right.

The hypothesis is that the Australian aboriginals use language which centers around compass points rather than relative descriptions of location (X is to the left of Y) and therefore, they have a better awareness of where they are geographically.  I have no idea if it works the other way and these girls would have trouble description the position of two things in relation to each other but the point is there probably isn’t an ‘ideal’ way of describing something.  It’s just that using one system closes (or makes more difficult) the ability to use another.

The second piece is from Cracked.  I’m not sure what’s going on over there but I’ve found a couple of pieces there lately that belie the image of the site as being all penis jokes photoshopped punch lines.  Or, maybe I’m just trying to find an excuse to read all those penis jokes….

Anyway, the article gives very brief overviews of 5 ways language can screw with your worldview and one in particular jumped out at me.

Recent studies have suggested that language may act as a cue to which cultural frame of reference a given interaction belongs in…Psychologists call this phenomenon frame-shifting, and it’s basically the ability to put yourself in someone else’s cultural shoes just by speaking in their language.

For example: A test was applied to bilingual Arab Israelis who spoke both Arabic and Hebrew (two cultures that have famously held a little animosity toward each other over the years) that asked participants to record whether words had negative or positive connotations. When the test was given in Arabic, the participants picked Jewish names as being intrinsically negative, but this effect disappeared when the test was given in Hebrew. In short, their bias against Jewish names arose from the fact that they were thinking in Arabic at the time, and not because they necessarily had any deep-seated bias against Jews. Don’t go thinking that the Arabic language is somehow inherently racist — it has plenty of Jewish friends. They just go to another school; you wouldn’t know them.

Probably worth some serious consideration when thinking about sources that are translated or even among those who are non-native speakers.  In intelligence analysis you are usually working with small amounts of information and trying to divine meaning (perhaps too much) from the dribs and drabs you get.  Given how preconceived notions and cognitive biases can form so easily and early a turn of a phrase or different intonation can send even a good analyst careening off into left field.

And just so you don’t feel let down now that you made it all the way through this post, here’s a link complete with your daily allotment of penis jokes….enjoy.

Cognition and intelligence analysis

A couple of stories have been in the press recently that have some interesting implications for intelligence analysis.

First, courtesy of Discover magazine, is this piece summarizing research that seems to indicate that people that sign their documents on the top of documents (before they’ve entered data or made a statement) their information is more accurate than if they sign at the bottom of the document (after they’ve already done the work).

People are often dishonest in little ways on forms, rounding numbers in a beneficial direction or failing to mention a relatively small item as part of a larger list. If they sign a form once they’ve done all that, they don’t go back and correct it; instead, they’ve already woven a story to themselves—consciously or not—about why what they did was perfectly fine.

It’s worth noting that most intelligence products do not have the author(s) names attached.  Now, there’s usually a very good reason for that.  Namely, that the analysis done is supposed to represent the agency’s position and not the individuals.  Additionally, there’s a security issue as well.  Knowing that analyst ‘A’ is the one who writes all the stuff about security issues in Outer Mongolia opens that analyst up to targeting and influence.

That being said, I’ve heard analysts say things like ‘I don’t care, my name’s not on this.’ Anonymity often breeds what I recently heard described as ‘a culture of compliance rather than one of performance’.  Check a box…if you get it wrong, who cares?

This isn’t just an individual issue, either.  Take a look over at Public Intelligence and you can see all sorts of examples of poor analysis (and occasionally good).  Very rarely are agencies held accountable for putting out bad, or just outright wrong, analysis so we can’t just go out and hammer analysts.

There’s got to be a way to address both problems.

The London School of Economics has this podcast about cognitive biases in support of the speakers book titled ‘The Art of Thinking Clearly‘.  It’s a fun, easy to access set of examples that demonstrate the various ways in which cognitive biases cause us to make poor decisions.

One particular point I like to emphasize when teaching critical thinking and analysis that Dobelli mentions is that what we see as cognitive biases today are actually traits that were essential for survive for much of the human (and, I suppose, pre-human) evolutionary process.  When you’re a hunter-gatherer traveling across the savannah and you see a shadow in the tall grass, your buddies to take off running.  Maybe it’s not a lion in the grass but if it is they’ve got a good shot at getting away.  Meanwhile, while you’re trying to analyze the various possible hypotheses explaining the movement, some sabre tooth is picturing you with a nice mango salsa.

Another part of the lecture reminded me of a circumstance I had where I had written a product yet it languished in editing/approval hell for an astounding 13 (!) months.  Finally I suggested officially killing the project since its contents were of dubious relevance any more and I had increasing concerns about the validity of my original findings.  My suggestion seemed to be the spark that was needed for everyone else to decide that the product needed to be disseminated right now!  Lengthy, impassioned arguments discussing my concerns were brushed aside.  After all, I was told:  ‘We’ve already spent so much time on this already…we can’t just let it go.’

When I mentioned the concept of ‘sunk costs‘ I got this sort of look:

YouTube Preview Image

For the record, I’m kind of used to those looks now…

The idea that the time spent on project X is already gone doesn’t justify spending more time on it unless project X makes sense and has value but my overlords at the time saw that past time as some sort of investment and were determined to get some sort of return on that investment.  Getting them to see the sense in the fact that their ‘return on investment’ would, in fact, just leave readers confused about why they were getting a product about an event that was a year old, took some doing.

On fauxhawks, cognitive biases and intelligence analysis

I’ve been out of the military now for slightly more than a year but still found myself adhering to AR 670-1 when it came to my trips to the barber.  Over the ears, over the collar…pretty short all over.  Some of that is necessity (my hair is very think and festooned with cowlicks everywhere and if left to its own devices would soon turn into a birds nest) but mostly it was habit.  So, I decided to change that…Here are the results 1:


Note the Hitler Pillsbury Doughboy in the background. That’s a fuck you to white supremacists, not baked goods, for the record…

Now the reaction from the people I work with was quite interesting.  My close co-workers are used to my hijinks so this was just sort of a status quo but for those a bit further out from the center of our social circle there was some consternation.  My coworkers and I received questions along the lines of:

‘What’s going on? Did he lose a bet? What does it mean?’

In short…none of these people could imagine a scenario in which someone like me (or, at least someone in our community/situation/etc.) would do this unless he was compelled to.

I, on the other hand, couldn’t imagine why I wouldn’t do such a thing.

So, what does this mean for intelligence analysis?  Part of an analyst’s job is to ‘think red’ or, consider what may motivate our foes, what priorities they may have, and what actions they may take.  Part of doing that involves avoiding the cognitive bias of ‘mirror imaging‘.  Now, I’ve been working in this particular office for a couple of years now and many of these people have seen me, heard me, had the opportunity to get to know me and with regard to my haircut they were under no pressure to reach a snap decision.  Yet, these individuals were unable to come up with potential motivations for my actions.  Were unable to put themselves ‘in my shoes’ to understand my actions.  How much more difficult when dealing with people involved in more complex activities, perhaps intentionally attempting to deceive, maybe with different cultural norms, with incomplete information and when under time pressure?

Cognitive biases aren’t something to be addressed once and then considered ‘dealt with’ for all time.  We need to be aware that they are the default setting for our brains and without active measures to control for them, we’ll slip into the same old thinking ruts that can lead to shoddy analysis.

  1. Upon seeing this, my wife said ‘Oh no! Now no one at work will take you seriously!’ I replied: ‘Trust me, my hair is NOT the reason the people at work don’t take me seriously.’