Cobra Effect · Science and philosophy
Absence of evidence
Finding nothing only counts if you would have found it.
6 cards, read aloud in 2:30, with a test and sources.
A racehorse vanishes in the night. The stable dog does nothing.
Silver Blaze, 1892. Sherlock Holmes is asked what he wants to draw attention to. The curious incident of the dog in the night-time. The dog did nothing in the night-time, says the inspector. That was the curious incident.
A dog that does not bark is evidence. Of a visitor it knew.
Holmes reasons that the horse was taken by someone the dog recognised. Nothing happened, and nothing was the clue. Silence can speak. But only if you know how loud the dog would have been.
Absence of evidence is not evidence of absence. Usually true. Often abused.
Carl Sagan gave the line currency in 1995, writing about an invisible dragon in a garage that no test can detect. You cannot prove the dragon is not there. But if the dragon should have left footprints in the flour and there are none, that is not nothing. How hard you looked decides which.
The question is: if it were there, would you have seen it?
If the answer is yes, and you didn’t, then not seeing it is evidence. Strong evidence. A thorough search of the drawer that finds no keys means the keys are elsewhere. If the answer is no, because you barely looked or the thing hides well, absence tells you almost nothing. Same silence. Different meaning.
2002. A defence secretary uses the line about weapons that were never found.
Donald Rumsfeld said absence of evidence is not evidence of absence, about Iraq. As a matter of logic, he was right. Then the inspectors went in, and later the invading armies, and years of hard looking found nothing. Hard looking that finds nothing is evidence. The line is a shield, not a search.
Before you shrug at a silence, rate the dog.
Ask two things. If the thing were real, how likely is it that I would have found it by now? And how hard did I actually look? A loud dog that stays quiet is telling you something. A deaf one is not.
Sources
- Evidence of absence, Wikipedia. The logic, where the phrase came from, and the section on when absence counts as evidence and how much.
- The Demon-Haunted World, Carl Sagan, 1995. The dragon in the garage, the baloney detection kit and the rest. The dragon chapter is the one that carries the line.
- The Adventure of Silver Blaze, Arthur Conan Doyle, 1892. The story itself, half an hour long. Read it for the dog, then notice how much else Holmes deduces from what is not there.
Nearby ideas
- Bayesian updating. Change your mind in proportion to the strength of the evidence.
- Falsifiability. A claim that nothing could disprove tells you nothing.
- Survivorship bias. The failures you never see can reverse the lesson you draw.
- Clarke’s third law. Advanced technology can look like magic, and so can a good trick.
- Schrodinger’s cat. A thought experiment built to show where the quantum rules run out.
- The veil of ignorance. Write the rules as if you did not know which seat you would get.
- Pascal’s wager. When a small cost seems to buy protection from an endless loss.
- Kuhn’s paradigm shifts. How science changes, not step by step, but by revolution.