From The Wall Street Journal:
Say you’re tested for a medical condition and the result is positive. Your physician says that this happens with 80% of people who really have the condition and 10% of those who don’t, and that the condition only affects 1% of the population. So what is the likelihood that you have it? Worrying comes naturally to us, but probabilities don’t, as Ian Stewart explains in “Do Dice Play God?,” his entertaining guide to the mathematics of uncertainty.
A standard approach to probability is “frequentism,” in which the numerical probability of some event is interpreted as “the proportion of occasions on which it happens, in the long run.” If a fair coin is tossed many times, heads and tails will appear about equally, corresponding to probability 0.5, or 50%. But how do we know if a coin is fair? Even after a million tosses there might not be exact parity; it would require an infinite number to decide the matter completely. In fact Mr. Stewart reports an experiment which showed that if a coin is tossed identically every time—using a specially designed machine—then whichever side is initially uppermost has a slightly greater chance of being on top when the coin lands. The fairness of coin tossing really resides in the inconsistency of how it is done in practice.
An alternative to frequentism is an approach pioneered by the 18th-century philosopher Thomas Bayes, who viewed probability as “degree of belief”—how confident we are in an event occurring. This makes probabilities of one-time events more meaningful. Bayesian inference is a way of calculating probabilities from prior knowledge, and it provides the answer to the medical question posed above. The likelihood of having the condition, given a positive test result, turns out to be only 7.5%. Most people—including physicians given the question in a survey—guess a far higher figure.
Mr. Stewart describes how similarly mistaken thinking had disastrous results for Sally Clark, an English woman who suffered the loss of her baby son in 1996. The presumed cause was sudden infant death syndrome, but when Clark lost a second infant in similar circumstances she was charged with murder. An expert witness said the likelihood of one such tragedy was 1 in 8,500; of two it was 1 in 73 million. Clark was found guilty and sentenced to life imprisonment. The prosecution, however, had made two crucial errors. One was to have neglected the possibility of a genetic factor that could make a second death more likely. The other was to have focused entirely on the probability of the event when the real question was the likelihood of guilt. Imagine winning the lottery and going to collect your reward, only to find yourself being arrested. Since the odds of winning are 1 in 175 million, surely you must be a fraud. That’s the kind of argument that put Clark in jail, and although she was eventually freed on appeal, the same “prosecutor’s fallacy”
Link to the rest at The Wall Street Journal (Sorry if you encounter a paywall)
TRX, I suspect that the problem was that British lawyers rarely have an advanced mathematical education and the defence probably had no idea that they needed a statistician as an expert witness whilst the prosecution were delighted to accept without question the testimony of their “expert”.
Mind you, advanced mathematics may not have helped. The counter intuitive nature of probability questions frequently confuses the educated as the fiasco of the math Ph.Ds and Marilyn vos Savant/Monty Hall demonstrated.
We don’t need no stinkin’ statisticians.
There are two areas where everyone is an expert: Statistics and economics.
> The prosecution, however, had made two crucial errors. One was to have neglected the possibility of a genetic factor that could make a second death more likely. The other was to have focused entirely on the probability of the event when the real question was the likelihood of guilt.
—
The prosecution focused on things that were likely to get them a conviction. It’s what they do.
A defense attorney who was awake would have shredded the prosecution’s case.
Ian Stewart is a pretty good writer on mathematics so I’m not in any way suggesting he’s got things wrong but I can’t see any need to use Bayes’ theorem or even Bayesian rather than frequentist theory to get this conditional probability result. The message of course is that for uncommon conditions a test must have low false positive rates for the results to have much value (other than to say “we need to do some more tests”). What’s worrying is that doctors may not realise this.
Sally Clark’s case was a horrifying miscarriage of justice which can be blamed on everyone in the court’s failure to understand probability theory (especially the fact that two events may not be independent) but is mostly down to the “expert” witness. Even his 1 in 8,500 figure was highly dubious as he’d upped the actual population rate of 1 in 1,300 to allow for the family’s prosperous background but failed to adjust it down for the sex of the children.
The actual case was more complicated than the OP’s report as it also involved a pathologist suppressing evidence (for which he was later found guilt of gross professional misconduct).
I’m reminded of the anecdote about the guy who learned that there was a (numbers made up) one in ten thousand chance of a bomb being on a plane. He decided that was too risky, so now, whenever he flies, he carries a bomb with him, because there’s only a one in a hundred million chance of their being two bombs on a plane.