Cobra Effect · Thinking and evidence
The Texas sharpshooter fallacy
Draw the target after the shots and any pattern looks meaningful.
6 cards, read aloud in 2:27, with a test and sources.
A Texan fires his rifle at the side of a barn. Then he fetches a paintbrush.
He walks up to the wall, finds the tightest cluster of holes, and paints a target around it. Bullseye, every time. The joke is old. Nobody knows who told it first. The mistake it names is everywhere.
Four cases of cancer on one street.
A town notices that four people on the same road fell ill in three years. There is a factory a mile away. There is a power line. There was a spill, once. Somebody draws the circle. The health department is called in.
That is the Texas sharpshooter fallacy.
You look at the data, find a cluster, and then draw the pattern around it as if you had predicted it. But random points cluster. Spill rice on a table and it clumps. With thousands of streets and dozens of cancers, some street will have four cases by chance alone. The question is not whether a cluster exists. It is whether you drew the target before or after the shots.
The health departments have kept score.
A 2012 review looked at over 400 cancer cluster investigations by American state and federal agencies across twenty years. Just one found a cause it could confirm. The suspected streets were real. The cancers were real. Almost every time, the pattern was painted on afterwards.
It hides wherever there is enough data to hunt in.
The fund manager who beat the market three years running, out of thousands of managers. The prophecy that fits, once the event has happened and you know which lines to read. The diet that worked for the five people who wrote in. Given enough shots, a tight group is guaranteed. Finding one proves nothing.
The fix is to paint the target first.
Find the pattern in one set of data. Then test it on a fresh set you haven’t looked at. If the cluster is real, it will show up again. If it was paint, the new wall will be blank. Before you believe a pattern, ask one thing. Did I predict this, or did I find it?
Sources
- Texas sharpshooter fallacy, Wikipedia. The joke, the cancer cluster examples, and how it relates to finding patterns in noise. Search Goodman 2012 cancer clusters for the review of 428 investigations.
- The Drunkard’s Walk, Leonard Mlodinow, 2008. How randomness makes patterns that look designed, told through casinos, wine ratings and fund managers. The chapters on clusters and streaks are the ones to read.
- Cancer cluster, Wikipedia. What a cluster is, why almost none of them turn out to have a cause, and the few that did, like the Woburn and Hinkley cases.
Nearby ideas
- The law of small numbers. Small samples swing wildly, and the swings look like signals.
- Confirmation bias. We look for evidence that agrees, and rarely test what could say no.
- The prosecutor’s fallacy. A rare match is not the same as a small chance of innocence.
- The Semmelweis reflex. Why experts reject evidence that says they have done harm.
- The illusion of explanatory depth. We think we know how things work until we try to explain them.
- Gell-Mann amnesia. We spot the errors in our own field, then trust everything else.
- The decoy effect. A worse option added to a menu can steer what you choose.
- The endowment effect. We value things more simply because they are ours.