Cobra Effect · Statistics and probability
Benford’s law
In real data, numbers starting with 1 far outnumber those starting with 9.
7 cards, read aloud in 3:15, with a test and sources.
1881. An astronomer notices that his book of logarithms is dirty at the front.
Simon Newcomb uses tables of logarithms every day, like every scientist of his time. The early pages, the ones for numbers beginning with 1, are worn grey and soft. The pages for numbers beginning with 8 or 9 look almost new. Everyone uses the same book. So everyone must be looking up more numbers that start with 1.
He writes it up in two pages, and the world ignores it for 57 years.
Newcomb proposes that in real world numbers, the first digit is 1 far more often than 9. He even gives the formula. In 1938 a physicist at General Electric, Frank Benford, finds the same thing and does the legwork.
Benford checks 20,229 numbers from twenty different sources.
Rivers. Populations. Physical constants. Street addresses. Baseball statistics. Death rates. About 30% of them start with 1. About 18% start with 2. Fewer than 5% start with 9. It doesn’t matter what you are counting. The pattern is there.
Why? Because growing things spend a long time in the ones.
Take anything that grows by a steady percentage. A savings account. A town. A share price. To go from 1,000 to 2,000 it must double. From 8,000 to 9,000 is only a small step. So it lingers in the ones, and races through the eights and nines. Anything that spans many sizes, measured in any units, comes out the same way.
Fraudsters don’t know this. Auditors do.
Someone inventing expenses spreads the first digits evenly, or leans on the digits that feel random to them. Real invoices are lumpy in exactly Benford’s way. From the 1990s, the accountant Mark Nigrini turned this into an audit tool, and tax authorities use it to pick which returns to look at. It doesn’t prove fraud. It says: look here.
Where it doesn’t work, and where it has been abused.
Numbers someone assigned don’t obey it. Phone numbers, postcodes, ticket numbers. Numbers squeezed into a narrow range don’t either. Adult heights nearly all start with 1 in centimetres and 5 or 6 in feet. And it has been waved at election results as proof of fraud. Political scientists have argued it is unreliable there. Right tool, right data.
The habit. Look at the first digits before you trust the table.
If you are handed a set of figures that should be lumpy and it comes out smooth, ask who typed it. And if you ever have to invent a number, remember the worn front pages. Better still, don’t.
Sources
- Benford’s law, Wikipedia. Newcomb’s 1881 note, Benford’s 1938 paper with its 20,229 numbers, the maths of why it happens, and the arguments about using it on elections.
- Benford’s Law: Applications for Forensic Accounting, Auditing, and Fraud Detection, Mark Nigrini, 2012. The auditor’s handbook, by the man who made the law a tool. This page is Nigrini’s biography and lists his books.
- Simon Newcomb, Wikipedia. The astronomer who saw it first and moved on. A remarkable life, of which the worn logarithm pages are a footnote.
Nearby ideas
- The gambler’s fallacy. Chance has no memory, so nothing is ever due.
- The law of small numbers. Small samples swing wildly, and the swings look like signals.
- The Texas sharpshooter fallacy. Draw the target after the shots and any pattern looks meaningful.
- The hot hand. Streaks are mostly chance, though not always entirely.
- The law of large numbers. Averages settle down over many tries, but a run never gets corrected.
- Mean and median. Why the average can describe nobody at all.
- The Literary Digest poll. A small sample chosen well beats a huge one of the wrong people.
- The garden of forking paths. Choosing the analysis after seeing the data makes flukes look real.