You list five options, score them against criteria, and pick the highest scoring. The experienced operator already knew what option to choose, their skill long since dropped below conscious thought. You’re slower and you’re no more accurate. Nothing is wrong with the analytical method; it was simply aimed at the wrong decision.
Chapter 12 named the capacity to disagree with a confident voice; recognition is the cognitive-science version of the same question: when expert intuition is reliable, and when it is bias dressed as competence.
🎧 Prefer to listen? This chapter is narrated in my own voice with ElevenLabs on Spotify (24 minutes).
The two poles
Pole A: analysis. The right answer comes from listing the options, comparing them against criteria, and choosing the best. Deliberate analysis makes a decision defensible.
Pole B: recognition. The experienced operator recognises a situation they have seen before, retrieves one workable course of action, and mentally simulates it forward. It often arrives as a feeling before a thought, the gut that says roll back before you could say why. Pattern recognition makes a decision fast and—under the right conditions—reliable.
Where Pole A is right
Pole A is right when the environment is irregular, feedback is delayed or absent, the decision will need to be justified in public, or the operator lacks the experience that Pole B needs. Think of a CFO presenting capital allocation to the board, a new engineering manager making their first architecture call, or anyone facing a failure mode nobody has seen before.
Analysis is a good approach in those settings. The reader analysing a new acquisition, a new market, or a never-seen failure mode is doing the right work. For the median engineering leader spending the day on AI governance, never-seen failure modes, and calls that will be justified in public, sustained analysis is the right choice. Recognition keeps the ground it has earned: well-characterised, fast, recoverable judgements.
Honour where the analytic habit came from. A leader who came up justifying every call to a board, an audit committee, or a regulator built a real discipline, one that earns trust with other people’s money and works well for the decisions it was built to serve. The error comes when that discipline crosses into a domain where a calibrated expert’s first read is the better instrument and further thought buys delay and false confidence. The training built something real. It was carried into a domain it no longer fits.
Where Pole B is right
Pole B is right when the environment is regular enough for experience to build trustworthy patterns and feedback arrives quickly enough to keep those patterns calibrated. Emergency response and surgical theatre fit. So does a deployment rollback that an expert has handled a thousand times, with the result visible within minutes.
In decisions
Pole A leaders use a checklist or formal review where the stakes are high and feedback is poor. Pole B leaders trust the experienced operator’s first reasonable option, while watching for the moment the situation slips beyond the operator’s experience.
The poles do not carry equal weight here. Recognition owns a real but bounded territory: well-characterised, fast, recoverable judgements that an expert has made a thousand times. Analysis owns the rest, including the never-seen failure mode, the call the board will see, and the AI-governance decision that often sits behind both. Most of a senior leader’s hardest calls now sit there.
A CTO or VPE should carry one irony to the CEO on a single page: as AI takes over routine recognition, it erodes the very expertise operators need to catch its mistakes. The organisation’s approach to analysis and recognition decides how it governs AI tooling, and that governance belongs with the CEO.
The CEO needs to fund ways to preserve hard-won human expertise (deliberate practice, red-team exercises, and rotation of experienced operators through the edge cases AI handles least well) even when the dashboard is green. The budget will not protect that work without a CEO-level signal. The sentence your CEO can carry to the board: “Automation is eroding the very expertise we’ll need on the day it fails.”
The Klein-Kahneman agreement
Gary Klein and Daniel Kahneman spent two decades disagreeing about whether expert intuition was real. Klein had documented it in firefighters, nurses, and chess masters. He called the mechanism Recognition-Primed Decision: the experienced operator recognises the situation as a type they have seen before, retrieves one workable course of action, and mentally simulates it forward (Klein, Sources of Power). Recognition makes the decision fast by replacing the comparison of options.
Daniel Kahneman had spent the same decades synthesising evidence against reliable intuition in stock pickers, political forecasters, and clinical psychologists. Confidence in those settings tracked the coherence of the story rather than the quality of the evidence. On the surface, Klein and Kahneman could not both be right.
They eventually agreed in a 2009 American Psychologist paper titled Conditions for Intuitive Expertise: A Failure to Disagree. Expert intuition is real where the environment is sufficiently regular to be predictable and the operator has had prolonged practice, with feedback, at learning its regularities. It is unreliable everywhere else.
The conditions decide which pole fits. Ask whether they have produced reliable expertise.
Most organisations ask whether the person making the call is senior enough. The Klein-Kahneman agreement asks whether the conditions have calibrated their recognition. Same person, different conditions, different right answer.
I recalibrate my own mentoring practice on this often. A client arrives confident in recognition built in a context that no longer applies. Another defaults to analysis on a decision their experienced operators could have called in thirty seconds. The learning lives in the conditions, not in the person.
Klein’s method underneath the framework
Klein reached Recognition-Primed Decision through fieldwork. His Critical Decision Method, a form of cognitive task analysis, asks experienced operators about recent hard incidents. The interviewer follows the story and probes the thinking: what did you see?, what made this familiar?, what would someone with less experience have done?
In regular environments, experienced operators usually retrieved one option, simulated it, and acted. They did not generate a set of alternatives. Herbert Simon had named satisficing decades earlier as a cognitive limit: taking the first option that meets the relevant criteria rather than maximising across all of them. Klein showed when that limit becomes an achievement: in regular environments, under time pressure, the expert takes the first workable option and acts.
A senior site-reliability engineer who calls a rollback in fifteen seconds may be doing exactly what experienced operators do in regular environments with rapid feedback. A manager who makes them stop and list five options imposes Pole A on a Pole B situation, slowing the decision without improving it.
Distributed cognition
Recognition reaches beyond the individual. Cognition is distributed across people, tools, and the environment. A ship’s navigation team is one cognitive system, with the navigator working as part of a wider whole. When something fails on the bridge, the configuration that brings the ship to anchor includes the charts and instruments, the bearing books and trained crew, and the procedure for fixing position.
Remove just one component and the system can no longer think in quite the same way.
Engineering organisations also hold expertise across a system. The senior engineer who knows the legacy code depends on source code, a build pipeline, and an on-call playbook; none carries the whole capability alone. Pole A leaders sometimes ask individual engineers to compensate for a poor configuration through more analysis. Pole B leaders improve the configuration itself with better tooling, documentation, and playbooks, so the system carries more of the work.
The safety-science chapters that follow make the same mechanism concrete. The operator is part of the system, so blaming them treats one component as though it were the whole. Chapter 14 takes that claim out of cognitive science and into safety.
Bainbridge’s ironies of automation
Lisanne Bainbridge wrote a five-page paper in 1983, Ironies of Automation, that names the central mechanism behind the recognition-versus-analysis choice in AI-augmented systems. Forty years before AI agents, she named the cost of letting the machine do the easy parts.
Bainbridge named two ironies. First, the designer who tries to eliminate the human operator is also a potential source of system failure. The second deserves more time. As Bainbridge put it, the designer who sets out to remove the operator still hands that operator whatever the designer couldn’t work out how to automate, so the operator ends up with a leftover, arbitrary set of tasks that nobody designed support for. Automation strips the easy parts and leaves the hard ones. Then it gives the operator little help with what remains.
Automation strips the easy parts and leaves the hard ones. Then it gives the operator little help with what remains.
Bainbridge catalogued the effects. Physical skills fade without use, so a formerly experienced operator who has spent long enough monitoring an automated process may now be an inexperienced one. Cognitive skills also need frequent use and feedback; once automated, they lose the patterns recognition depends on.
Working storage is Bainbridge’s term for the operator’s running feel for the process, richer than short-term memory, and it takes time to build. She noted that some manual operators entered a control room a quarter to half an hour before they are due to take over control so they can recover that feel. The central irony remains: the automatic control system is installed precisely because it outperforms the operator, and yet that same operator is the one charged with monitoring whether it is working.
Her verdict, in her words: “The human monitor has been given an impossible task.” The operator must catch what the automation misses in a domain where the automation already outperforms them on routine cases. Meanwhile, the automation has hollowed out the expertise built under the conditions Klein and Kahneman agreed produce reliable judgement.
The human monitor has been given an impossible task.
Lisanne Bainbridge, Ironies of Automation, 1983
Bainbridge’s irony of manual takeover follows the same mechanism. By the time a human has to take over, something has usually gone wrong with the process, so controlling it calls for unusual action, and you can argue the operator at that moment needs to be more skilled and less loaded than average, not less skilled and more loaded.
At the moment of need, the operator is less skilled and more loaded than they were before automation took the routine work.
My read is that Bainbridge’s 1983 mechanism explains AI governance better than most contemporary work: automation weakens the expertise it still depends on. The problem is older than ChatGPT. The question is whether we have any better answers than we did in 1983 and—from the evidence I’ve seen to date—we mostly don’t.
Pole B recognition works only when the operator has the orientation the situation needs. Automation that strips away the operator’s experience degrades the conditions that make Pole B trustworthy. The longer the system runs well under automation, the less reliable the operator’s recognition becomes, until the moment something goes wrong and the system needs that recognition most. Chapter 17 returns to the authority problem this creates.
The AI failure modes Bainbridge is pointing at
Frequency-versus-truth is the first failure mode. A large language model predicts the next token from statistical patterns in training data. Its output tracks the distribution of that data more than the truth of any claim. The training distribution is dominated by the average of what humans have written. That pulls the model toward the consultant-derivative regression described in chapter 1, not the contrarian signal that makes a diagnosis valuable.
The model reproduces common errors with confidence because confidence tracks coherence of story, not quality of evidence. This is the Kahneman side of the Klein-Kahneman agreement applied to the system itself. Chapter 1’s grounding argument is the response, though the leader cannot always tell from the output whether grounding is in play.
Out-of-distribution collapse is the second failure mode. AI models perform well on tasks that resemble their training data. They degrade, sometimes gradually and sometimes abruptly, on tasks outside it. The model’s confidence doesn’t track its accuracy across this boundary; the output looks the same whether the task is well within the training distribution or well outside it.
Mollick’s jagged frontier from chapter 3 names the result: high performance on one task, near-zero on a similar task, and no visible seam between them. Vibe-coded work shipped where agentic engineering was owed is the downstream consequence. The model produces plausible-looking code outside its distribution, the code fails contract tests the model was never given, and the human operator trusts the output without running the check.
Bainbridge expertise-hollowing is the third failure mode. An engineer who hands straightforward problems to the model keeps responsibility for the hard ones, exactly as in Bainbridge’s manual-takeover scenarios. At the same time, the engineer loses the practice needed to handle those hard problems: the pattern recognition built by solving easy cases and the feel for normal that makes abnormal recognisable.
The irony holds exactly. The automatic system was put in because it performs better on routine tasks, but the engineer must still catch the cases where it fails. At the moment of need, the engineer is less skilled than they were before handing the routine work over.
A worked example: the experienced incident commander
Watch a senior site-reliability engineer take command of a production incident. In the first thirty seconds, they scan the dashboard and ask one question of the on-call engineer. They identify the misbehaving subsystem, form one hypothesis, and send one diagnostic action. They don’t list five hypotheses and score them. They recognise the pattern.
Recognition is doing the work analysis would otherwise have to do. It is faster and, under the right conditions, more accurate because the SRE has seen this kind of failure many times, the dashboard provides rapid feedback, and the diagnostic action is reversible if the hypothesis is wrong.
Take away any of those conditions and Pole A becomes the right move. A new SRE may lack the pattern. An under-instrumented dashboard cannot provide quick feedback. An irreversible diagnostic action makes a wrong first read too costly. In those cases, the Pole A leader is right to ask for a checklist. Where the conditions support recognition, the Pole B leader is right to let the SRE decide. The context settles it.
The AI overlay
AI agents work like recognition machines: they identify patterns in training data and produce outputs that match. They are excellent in regular environments with rapid feedback, exactly where Pole B is right. They are unreliable in irregular environments with delayed feedback, exactly where Pole A is right.
Deploying AI in Pole A territory and trusting its confident outputs is a structural mistake. The model has no patterns to recognise there, so coherence of story masquerades as expertise. The Kahneman side of the Klein-Kahneman agreement applies to models as it does to people. Confident output without environmental regularity is bias.
Bainbridge’s ironies follow. When AI handles routine recognition well, the human operator loses the practice needed to catch the cases it gets wrong. Governing AI therefore calls for Pole A discipline: keeping human expertise alive while automation does most of the routine work. Chapter 17 returns to this AI-and-authority question.
The recognition story still holds within a smaller domain than the one a senior leader now inhabits. Once AI is in the loop, the human must keep a live, independent read on the automation because its confident output is least trustworthy in irregular territory. Sustained analysis is the governing discipline.
A leader governing AI well first identifies which decisions fall in regular environments, where AI recognition is reliable, and gets out of the way. For irregular environments, the leader uses analysis, checklists, and formal review; most consequential calls belong here. The leader also preserves the expertise automation would otherwise erode through deliberate practice, red-team exercises, and rotation through edge cases, so the human keeps an independent read on whether the automation is wrong.
Two of those three are Pole A work: formal review for the irregular calls, and the deliberate practice that keeps human judgement sharp. Applying one method to all of them is bad governance, so the discipline is routing by conditions rather than a standing default. For the AI-governing reader, those conditions point to analysis often enough that the chapter resolves there.
The diagnostic move
Three questions for last Tuesday’s decision:
- Which pole was I claiming? Did I treat the decision as one for analysis or for recognition?
- Which pole would the actual decision-process show? If a colleague watched the process, would they see deliberate option-comparison or experienced-operator recognition?
- Which pole did the conditions actually support? Regular environment, rapid feedback → Pole B. Irregular environment, delayed feedback → Pole A. Automation-mediated environment that has hollowed out the operator’s experience → Pole A by structural necessity, regardless of what the operator’s stated experience suggests. This third case is no longer the exception. For the leader whose work is increasingly AI-mediated, it is the common case, which is why the chapter resolves toward analysis for that reader.
The most common gap is between the second and third questions. Leaders trust recognition in conditions that do not support it, or apply formal analysis where the experienced operator already knew.
The exercise
Run a Klein-Kahneman conditions check on a decision your team is sitting with this week. Ask two questions aloud. Is the environment regular: have you and your team seen many cases like this? Is feedback rapid: will you know within days whether the call worked?
If both answers are yes, trust the experienced operator’s recognition and get out of the way. If either answer is no, run an analytic protocol.
Then try the check on a decision the team has already made and is now defending. Most defended decisions had at least one no answer: the operator’s confidence outran the conditions. Naming that without blame builds the check into the team’s practice.
Going upstream
Watch. Daniel Kahneman video, Thinking Fast vs. Thinking Slow (Inc. Magazine, 7 min). Companion: Gary Klein, NDM/RPD video, Introduction to NDM and RPD (9 min).
In-text: the central pair and the AI overlay. Gary Klein, Sources of Power; Daniel Kahneman, Thinking, Fast and Slow, on the Kahneman-Klein agreement (the 2009 American Psychologist paper Conditions for Intuitive Expertise: A Failure to Disagree). For the AI-automation irony forty years early: Lisanne Bainbridge, Ironies of Automation (1983; 2021 retrospective preface available online).
Also touched: Herbert Simon on satisficing, Models of Bounded Rationality.
Go deeper: the same recognition claim arrives from several further traditions, none anchored in the body above. For the OODA loop and orientation as the central node: Robert Coram, Boyd: The Fighter Pilot Who Changed the Art of War (and Boyd’s appendix essay Destruction and Creation; Fingerspitzengefühl, fingertip-feel). For the doctrinal statement of Pole B: USMC, MCDP-1 Warfighting, Chapter 4. For cognition distributed across people, tools, and environment (the USS Palau navigation case): Edwin Hutchins, Cognition in the Wild; and for joint cognitive systems as the unit of analysis: David Woods and Erik Hollnagel, Joint Cognitive Systems. For the wider bounded-rationality lineage the chapter inherits: James March, A Primer on Decision Making. And from the emotional-intelligence tradition, for the felt and bodily side of recognition a purely analytic read can’t reach: Karla McLaren, The Language of Emotions, on the four intelligences (analytical, emotional, somatic, visionary) and the cost of letting the intellect try to rule without the other three.
I work with engineering leaders on exactly this kind of paradigm work, the deeper the better. If it’s live for you, I’m happy to talk: schedule a 30-minute virtual coffee at hi.chrisgagne.com.
Some book links here are Amazon affiliate links; if you buy through them I may earn a small commission, at no cost to you.
Leave A Comment