The Question to Ask Before You Pull a Lever.

What he was looking at

Austin Bradford Hill was one of the statisticians who showed that smoking causes lung cancer. In January 1965 he gave the presidential address to the occupational medicine section of the Royal Society of Medicine, and he took on the problem every doctor in the room had: a disease shows up more often among workers exposed to something. When is that a cause, and when is it only an association?

He started by saying what the decision is actually about:

“the decisive question is whether the frequency of the undesirable event B will be influenced by a change in the environmental feature A” Hill (1965), p. 295

What he found

He didn’t hand out rules. He offered nine viewpoints to weigh before concluding that an association is a cause: strength, consistency, specificity, temporality, biological gradient, plausibility, coherence, experiment and analogy.

“Here then are nine different viewpoints from all of which we should study association before we cry causation.” p. 299

Strength is the first. Among his examples: the lung-cancer death rate in cigarette smokers was “nine to ten times” that of non-smokers, and “twenty to thirty times” in heavy smokers. And John Snow’s cholera figures, in the chart above, where houses on the polluted water supply had 14 times the deaths.

He was just as clear that no single viewpoint settles it:

“None of my nine viewpoints can bring indisputable evidence for or against the cause-and-effect hypothesis” p. 299

They’re often called the “Bradford Hill criteria.” Hill never used that name. He called them viewpoints, and he pushed back on turning them into hard-and-fast rules.

What it means

An association tells you two things move together. Hill’s question is the one that matters for a decision: if I change A, does B change?

Some of his viewpoints get you closer without an experiment: a strong association, the right order in time, more of the cause bringing more of the effect. The experiment, actually changing A and watching B, gets you closest.

Where it shows up in a revenue team

• Ice cream sales “predict” shark attacks. Hill’s question: if the town sold less ice cream, would there be fewer attacks? No. Hot days put more people in the water, and that drives both.
• Temporality. Did the change come before the result? A campaign credited with a lift that started the week before it launched fails the first test.
• Gradient. If more of the “driver” should mean more of the result, check that it does, within the same segment of customers.
• Experiment. The strongest viewpoint is usually the one you can run: a holdout. Change the lever for some customers and not others, and compare.