You tested positive. Now what?

Most people say “about 99%”. The test is 99% accurate, after all. The real answer is about 9%. More than nine out of ten people who get that positive result are perfectly healthy.

That sounds like a trick, but it isn’t. It is what happens whenever a test looks for something rare, and it catches out experts too. In 1978, researchers at four Harvard Medical School teaching hospitals put a similar question to 60 people — 20 fourth-year medical students, 20 junior doctors and 20 senior physicians: a disease found in 1 in 1,000 people, a test with a 5% false-positive rate. Assuming the test never misses a sick person, the right answer is about 2%. Only 11 of the 60 gave it; the most common answer, from 27 of them, was 95%. When the question was repeated in 2014, the results were much the same.

The way out is not a cleverer formula. It is to stop thinking about percentages of a test and start counting people.

A town of 10,000

Imagine testing everyone in a town of 10,000 people. One in 1,000 of them has the disease, so about 10 people are sick. They are the red circles below. The test finds 99% of them, so nearly all 10 test positive (filled red).

Now the other 9,990 people, who are healthy. The test wrongly flags 1% of them: about 100 people (gold). One percent of a big number is still a lot of people. Look at the picture: the positive results are mostly gold.

So a positive result takes you from 1 in 1,000 (0.1%) to about 1 in 11 (9%). That is a big jump — ninety times more likely than before — but it is nowhere near 99%. The “99%” describes what the test does to sick people. It says nothing, by itself, about how many of the positives are sick. That depends on how many sick people there were to find in the first place.

Three dials

Three numbers decide everything here, and it helps to name them carefully.

• Prevalence: the share of the people being tested who actually have the disease. Here, 1 in 1,000.
• Sensitivity: of the people who have the disease, the share the test flags as positive. Here, 99%. The ones it misses are false negatives.
• Specificity: of the people who don’t have it, the share the test correctly clears. Here, 99%. The other 1% — the false-positive rate — are false positives.

The number you actually want after a positive result is different from all three. It is the positive predictive value: of everyone who tests positive, the share who are sick. After a negative result, you want to know the reverse: of everyone who tests negative, how many are sick anyway? (Doctors usually quote the opposite, the negative predictive value: the share of negatives who are truly healthy.)

The diagram under the dots is a natural-frequency tree: it splits the town first by whether people are sick, then by their test result. To find the chance you are sick after a positive, read the two “test positive” boxes and ask: out of all of those, how many are sick?

Here is the arithmetic the tree is doing, in one line:

chance sick after a positive = true positives ÷ (true positives + false positives)

With the default dials that is 9.9 ÷ (9.9 + 99.9) ≈ 9.0%. The false positives win because they are a small slice (1%) of a huge group (9,990 people), while the true positives are a big slice (99%) of a tiny one (10 people).

Counting like this is not just a teaching trick. In a 1995 study, the psychologists Gerd Gigerenzer and Ulrich Hoffrage gave people problems like this one. When the facts were stated as probabilities (“the probability is 1% that…”), 16% of the answers used correct reasoning. When the same facts were stated as counts (“10 out of every 1,000 women…”), that rose to 46%, with no teaching at all.

Who gets tested matters

Prevalence is not a fixed property of a disease. It is a property of the group of people being tested. Test everyone in a town, and the sick are 1 in 1,000. Test only people who walked into a clinic with the right symptoms, and they might be 1 in 10. Doctors call this the prior, or pre-test probability: how likely the disease was before the result came back.

The same test, with the same 99% and 99%, means very different things in these two groups. In the figure below, drag along the chart (or focus it and use the arrow keys) to change who is being tested. The red curve is the chance you are sick after a positive result; the green curve is the chance you are sick after a negative one.

Among people with symptoms (1 in 10), a positive result means about a 92% chance of disease; in whole-town screening (1 in 1,000), 9%. Notice the other curve too. In screening, a negative result is extremely reassuring. Among people who were already quite likely to be sick (1 in 2), a negative still leaves about a 1% chance — worth taking seriously if the disease is dangerous.

This is why doctors usually don’t screen everyone for rare conditions, and why a positive screening result is normally followed by a second, different test rather than by treatment.

Test again

So what should you do with your positive result? Test again. Take the 110 people who tested positive in our town, and test them a second time. Of the 10 who are sick, about 9.9 test positive again. Of the 100 healthy ones, only 1% do: about 1 person. Now the positives are roughly 10 sick people and 1 healthy one — about a 91% chance of disease.

There is a compact way to do this that works for any number of tests. Instead of a probability, use odds: the number of sick people for each healthy one. Before any test, the odds are 1 to 999. Each positive result multiplies the odds by the same number, the likelihood ratio: how much more often the test says “positive” to a sick person than to a healthy one.

odds after = odds before × likelihood ratio,  where likelihood ratio = sensitivity ÷ false-positive rate

For our test that ratio is 99% ÷ 1% = 99. One positive: 1 to 999 becomes 99 to 999, about 1 to 10 — the same 9% as before. Two positives: 9,801 to 999, about 10 to 1, or 91%. This is Bayes’ rule, written in the form that is easiest to use in your head.

That multiplication has a catch, and the checkbox is it. It assumes that the second test makes its mistakes independently of the first: a healthy person who fooled the first test is no more likely than anyone else to fool the second. That is roughly true for a different kind of test. It is false if you simply run the same test on the same sample again, or if something about you — another condition, a medicine, a cross-reacting antibody — causes the false positive. Then the second result just repeats the first, and it tells you almost nothing new. Untick the box to see that case: the hop disappears.

So the habit to take away is a question: out of how many? When you hear that a test is 99% accurate, ask how many people being tested actually have the thing — and count the false alarms among everyone who doesn’t.