Cobra Effect · Statistics and probability
Berkson’s paradox
Filtering who you look at can create links that are not there.
6 cards, read aloud in 2:31, with a test and sources.
Every handsome man she dates turns out to be a jerk.
She has enough data now to call it a rule. The kind ones are plain. The good looking ones are cruel. Her friends have noticed the same thing. It is not true of men. It is true of the men she dates.
Think about who gets a second date.
Nobody dates a man who is neither kind nor good looking. So the men in her life are kind, or handsome, or both. Among those, the plain ones must be kind, or they would not be there. And a handsome one can afford to be unkind and still get the date. The link was made by her door, not by the world.
Joseph Berkson saw the same thing in a hospital in 1946.
He was a statistician at the Mayo Clinic. Doctors were using hospital records to test whether one disease went with another. Berkson showed that the records would show a link even where none existed. You are in hospital because you have something. A patient with one disease does not need the other to be there.
That is Berkson’s paradox.
When you only look at people who passed a filter, the filter creates a link between the things that got them through. Two things that have nothing to do with each other, in the whole population, look connected in the sample. Usually connected backwards. More of one, less of the other. The mathematician Jordan Ellenberg gave it the dating example in 2014.
Once you know the shape, you see it everywhere.
Why do so many famous actors seem to lack talent? Because a plain actor with no talent never got famous. Why do the fast students in the top class seem lazy? Because the slow ones had to work to get in. Why is the book always better than the film? Because bad books do not get filmed. Every one is a door with two ways through it.
The question to ask before you believe a pattern.
How did these people, or these cases, get in front of me? If there was a filter, the pattern may be the filter’s shadow and nothing more. To see the truth, you need the ones who did not pass. The plain, unkind men are out there. She just never meets them.
Sources
- Berkson’s paradox, Wikipedia. The 1946 hospital argument, the stamp collector example with numbers, and the dating example. Search Berkson 1946 Limitations of the application of fourfold table analysis for the original.
- How Not to Be Wrong, Jordan Ellenberg, 2014. The chapter on why handsome men are jerks is the clearest telling of the idea in print, with the rest of the book applying the same care to lotteries, wars and elections.
- Collider, Wikipedia. The general form. When you select on something that two causes both feed into, you create a link between the causes. Short and a little technical.
Nearby ideas
- Simpson’s paradox. A trend in every group can reverse when the groups are combined.
- Survivorship bias. The failures you never see can reverse the lesson you draw.
- Base rate neglect. A good test for a rare thing still gives mostly false alarms.
- The law of small numbers. Small samples swing wildly, and the swings look like signals.
- The prosecutor’s fallacy. A rare match is not the same as a small chance of innocence.
- The St Petersburg paradox. A huge prize is worth less to you than its average suggests.
- Benford’s law. In real data, numbers starting with 1 far outnumber those starting with 9.
- The hot hand. Streaks are mostly chance, though not always entirely.