“If a test to detect a disease whose prevalence is one in a thousand has a false positive rate of 5% (95% accurate), what is the chance that a person with a positive result actually has the disease?”

Put another way:

1/1,000 people have cancer. A 95% accurate test for cancer is given to a random person. What is the chance he has cancer if the test says he cancer?

Only 18% of them were able to answer correctly.

ANSWER:

About two percent.

EXPLANATION:

Assume that the test is performed on everyone regardless of symptoms of the disease. Out of every thousand people who receive the test, one has the disease and 999 do not. One out of every thousand tests are true positives. The remaining 999 should be negative results, but the 5% false positive rate means that 50 (49.95) of these people will receive false positives. Out of our 1000 tests, 51 return positive results, but only one of these actually has the disease, so the chance that a positive test identified someone who actually has the disease is 1/51 or about 2%.”

This result has interesting implications. Most of us are intuitively unable to assess probability correctly, and require training to understand it at all; and most highly intelligent people with relevant training in medicine still fail to understand probability. Importantly, some people diagnosed with diseases or conditions are likely to be treated unnecessarily, sometimes with risky treatments.

In deciding to treat, a doctor should first weight the risks of treatment against the probability that the patient actually has the disease. If the doctor is unable to accurately assess the accuracy of the tests, how can he make a reasonable decision about which treatment is best?

This gives us a reasonable explanation for spontaneous remission-- the patients recovering may not have had the disease in the first place!

80% of Harvard Medical School doesn't understand what test results mean! Jesus! What the fuck!?