Cobra Effect · Statistics and probability
Regression to the mean
Extreme results drift back towards normal, whatever you do.
5 cards, read aloud in 1:43, with a test and sources.
A flight instructor swears that praise makes pilots worse.
Daniel Kahneman tells the story. Praise a cadet for a great landing and the next one is worse. Shout at a cadet for a terrible landing and the next one is better. So shouting works and praise doesn’t. Except it doesn’t.
An unusually good landing is partly luck.
The next one will probably be closer to the cadet’s average, whatever you say. An unusually bad one is partly bad luck. The next one will probably be better, whatever you say. The instructor was rewarded for punishing and punished for rewarding. By chance.
That is regression to the mean.
Francis Galton found it in the 1880s, measuring the heights of parents and children. Very tall parents have tall children, but usually less tall than themselves. Extremes are followed by something more ordinary, because extremes are partly noise.
It fools anyone who acts after an extreme.
The manager who steps in after a bad quarter and sees things improve. The remedy taken when the illness is at its worst. The sports star who has a great year, gets a magazine cover and then slumps. The cover didn’t cause the slump.
The question that saves you.
Was this extreme? If it was, the next one was always going to move back. Before you credit what you did, ask what would have happened if you had done nothing.
Sources
- Regression towards Mediocrity in Hereditary Stature, Francis Galton, 1886. The paper with the parents and the children, in the Journal of the Anthropological Institute. Search the title. Galton’s Wikipedia page puts it among everything else he measured.
- Thinking, Fast and Slow, Daniel Kahneman, 2011. The flight instructor story is here, in the chapter on regression, along with why the mind finds a causal story for every wobble. The chapter stands alone if you read nothing else.
- Regression toward the mean, Wikipedia. The mechanism, the sports and medicine examples, and the section on why it fools evaluations of treatments given to people at their worst.
Nearby ideas
- Survivorship bias. The failures you never see can reverse the lesson you draw.
- Goodhart’s law. A measure turned into a target stops telling you the truth.
- The sunk cost fallacy. Money already spent is gone, so it should not steer the next choice.
- The Monty Hall problem. Why switching doors wins twice as often as staying put.
- The birthday problem. Coincidences are far more likely than they feel.
- The gambler’s fallacy. Chance has no memory, so nothing is ever due.
- Berkson’s paradox. Filtering who you look at can create links that are not there.
- Simpson’s paradox. A trend in every group can reverse when the groups are combined.