Cobra Effect · Thinking and evidence
Correlation is not causation
Two things moving together may share a hidden cause.
7 cards, read aloud in 2:28, with a test and sources.
In July the ice cream carts do a roaring trade. In July people drown.
Plot the two across a year and the lines rise and fall together, almost perfectly. More cones sold, more people lost. Nobody has ever proposed banning the ice cream.
Summer is doing both jobs.
Heat sells cones. Heat fills the rivers and beaches with swimmers. The two lines move together because a third thing is moving them both. It is usually hiding in plain sight, and it is usually the weather, the money or the calendar.
Two things moving together is a clue, not a verdict.
Maybe the first causes the second. Maybe the second causes the first. Maybe something else causes both, or maybe you got unlucky with a small pile of numbers. Four stories fit the same chart, and the chart cannot tell you which.
Medicine learned this at enormous cost.
For years, women taking hormones after menopause had fewer heart problems than women who did not. When it was finally tested by drawing lots, the benefit for the heart did not appear. The women who took it had been healthier and better off to begin with.
The tie breaker is a coin, not an argument.
Randomise. Split people into two groups by chance, change one thing, and wait. Chance shares out everything you never thought to measure, the rich, the careful and the already ill. Then a difference at the end has nowhere else to have come from.
When you cannot randomise, you can still be careful.
In 1965 Austin Bradford Hill set out nine things to weigh before calling a link a cause. How strong it is, whether it repeats, whether more exposure means more harm, and whether the cause came first. None of them is proof alone. Together they build a case.
The question to ask the next chart you are shown.
What else was going on at the same time? And who decided which people ended up on which side? If chance did not choose, something else did, and that something is your answer.
Sources
- Correlation does not imply causation, Wikipedia. The ways two numbers can move together without one causing the other, with the classic examples and the counter arguments.
- The Book of Why, Judea Pearl and Dana Mackenzie, 2018. A statistician’s case that causation can be reasoned about properly, and a history of how long the field refused to try.
- Bradford Hill criteria, Wikipedia. The nine things Austin Bradford Hill said to weigh in 1965 before calling a link a cause, still used in public health today.
Nearby ideas
- Post hoc ergo propter hoc. Coming first does not mean causing what came after.
- The Texas sharpshooter fallacy. Draw the target after the shots and any pattern looks meaningful.
- Base rate neglect. A good test for a rare thing still gives mostly false alarms.
- Steelmanning. Argue against the strongest version of a view, not the weakest.
- Choice overload. Too many options that are hard to compare can stop people choosing.
- The razors. Simple rules for cutting away explanations you do not need.
- Survivorship bias. The failures you never see can reverse the lesson you draw.
- The Dunning Kruger effect. Why the least skilled are often the worst judges of their skill.