Strategies to reduce cognitive bias in intelligence analysis: Can mild interventions improve analytic judgment

Reports and studies point to cognitive bias as a possible factor in certain intelligence failures, but also suggest that cumbersome, time consuming analytic methods lack acceptance by professional analysts. A reasonable compromise, then, might be achieved by introducing effective but minimally intrusive cognitive interventions (i.e., "mental speed bumps") that induce a high-quality, high-confidence judgment. Prior studies on cognitive disfluency show promise in this regard. The objective of this research was to begin to fill significant gaps in understanding the possibilities and limits of bias-reducing analytic techniques (BRATs) in reducing the effects of cognitive bias in analysis. In doing so, this study examined four specific techniques that varied in terms of their required cognitive and physical effort. The impact of those techniques on analytic quality and confidence was assessed, and possible relationships between individual cognitive tendencies and complex judgmental task performance were investigated. The results of this study were mixed. While statistical significance was lacking in many of the comparisons, one intervention was shown to consistently outperform others and demonstrated improvement over the control group. This intervention, the Check the Item technique, required the least amount of context switching of all the interventions, lending support to the hypothesis that minimally-invasive approaches will be more effective in improving analytic quality. Observations of confidence, however, may demonstrate the opposite effect - techniques designed to improve quality may, at the same time, exacerbate an overconfidence bias. Individual cognitive tendencies showed weak correlations to confidence, but not to analytic quality, suggesting that confidence may be tied to more stable personality characteristics while quality is tied to more task-dependent factors. The utility of the Analytic Decision Task, the introduction of measures of analytic quality and improvement, and the use of response inertia as a means of detecting confirmation bias should all prove to be useful in the future studies proposed. The findings reported here should motivate further research on this topic in order to better understand how analysts make judgments and identify effective techniques to help them avoid cognitive pitfalls.