Cobra Effect · Thinking and evidence
Outcome bias
Judge a decision by what was known, not by how the dice fell.
6 cards, read aloud in 2:25, with a test and sources.
A surgeon recommends an operation. The patient dies on the table.
Was it a bad decision? In 1988 Jonathan Baron and John Hershey put that question to people. Everyone read the same case. Same patient, same risks, same reasoning, same choice. Half were told the patient lived. Half were told the patient died.
The decision was rated worse when the patient died.
Nothing about the decision had changed. Only the dice had. People judged the surgeon by the result, not by what the surgeon knew when the choice was made. Baron and Hershey called it outcome bias.
Poker players have a blunter word for it. Resulting.
Annie Duke played poker professionally for years and wrote about it in 2018. A good bet can lose. A terrible bet can win. If you judge your play by the pot, you learn the wrong lessons from both. The cards do not tell you whether you played them well.
February 2015. The most criticised play call in American football.
One yard from winning the Super Bowl, with 26 seconds left, the Seattle coaches called a pass instead of a run. It was intercepted. New England won 28 to 24. The next morning it was the worst call in history. Duke argues it was a defensible call that went wrong. If it had scored, nobody would remember it.
Results are loud. Decisions are quiet.
A result is one roll of the dice. A decision is the bet you made before the roll. Judge only by results and you will fire the careful manager who had a bad year. And promote the reckless one who had a lucky one. Then wonder why the luck runs out.
So separate the two, on paper.
For every decision that matters, ask two questions and keep the answers apart. Was it a good decision, given what we knew then? Was it a good outcome? Four boxes. Good decision, bad luck is the box people learn the least from. It is the one to study.
Sources
- Outcome bias, Wikipedia. The 1988 surgeon study and the other places it has been found, from football to medicine. Search Baron and Hershey 1988 Outcome bias in decision evaluation for the paper.
- Thinking in Bets, Annie Duke, 2018. A poker player on separating decisions from results. The Seattle play call opens the book, and resulting is the word she gives the mistake. Her page lists the book.
- Super Bowl XLIX, Wikipedia. The game, the final drive, and the argument that followed about whether the pass was a mistake or a reasonable call that went wrong.
Nearby ideas
- Hindsight bias. Once you know the ending, it always looks obvious.
- Survivorship bias. The failures you never see can reverse the lesson you draw.
- Regression to the mean. Extreme results drift back towards normal, whatever you do.
- The framing effect. The same facts, worded differently, lead to different choices.
- Loss aversion. Losing something hurts about twice as much as gaining it pleases.
- The curse of knowledge. Once you know something, you cannot see what a beginner is missing.
- Bayesian updating. Change your mind in proportion to the strength of the evidence.
- Overconfidence. Why we feel more certain than our knowledge allows.