Skip to content
Tutorials Center
My learning

Topics

Lesson 22 of 37 ยท Compound Events and Conditional Probability

Bayes' Theorem

Reverse a conditional probability with Bayes' theorem, use natural frequencies, and see why a positive test for a rare condition is often a false alarm.

You often know P(evidence | cause) but need P(cause | evidence). A medical test comes with known error rates, yet a patient wants to know how likely they are to be sick given a positive result. Bayes' theorem converts one into the other.

It uses ideas from conditional probability and the law of total probability in the tree-diagram lesson.

The formula

Text
P(A | B) = P(B | A) ร— P(A) / P(B)

with the denominator expanded by total probability:

Text
P(B) = P(B | A) ร— P(A) + P(B | not A) ร— P(not A)

P(A) is called the prior (what you believed before the evidence) and P(A | B) is the posterior (after).

A worked example: a medical test

A condition affects 1% of people. A test is positive for 95% of people who have it (its sensitivity) and, wrongly, for 5% of people who do not (the false positive rate). You test positive. What is the chance you have the condition?

  • Prior: P(sick) = 0.01.
  • P(+ | sick) = 0.95, P(+ | healthy) = 0.05.
  • P(+) = 0.01 ร— 0.95 + 0.99 ร— 0.05 = 0.0095 + 0.0495 = 0.059.
  • P(sick | +) = 0.0095 / 0.059 โ‰ˆ 0.161.

Only about 16%, despite a test that is right 95% of the time in each direction. The reason is that healthy people are so much more numerous.

The same thing with natural frequencies

Imagine 10,000 people:

Group People Test positive
Sick (1%) 100 95
Healthy (99%) 9,900 495
Total 10,000 590

Of the 590 positives, 95 are truly sick: 95 / 590 โ‰ˆ 0.161. Counting people instead of multiplying decimals makes the logic easy to see.

Python A Bayes calculator

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Why the prior matters

Change only how common the condition is and watch the answer move:

Python How prevalence changes the posterior

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The posterior climbs as the condition becomes more common. This is why doctors do not screen everyone for every rare disease: most positive results would be false alarms.

Updating twice

Posteriors can become priors. If a second, independent test is also positive, use the 0.161 as the new prior:

Text
P(sick | two positives) = 0.95 ร— 0.161 / (0.95 ร— 0.161 + 0.05 ร— 0.839) โ‰ˆ 0.785

Two positive tests raise the probability far more than one. (Independence of the two tests, given the true state, is an assumption.)

flowchart LR
  P["Prior<br/>1%"] --> T1["Test 1 positive"] --> Q["Posterior<br/>16%"]
  Q --> T2["Test 2 positive"] --> R["Posterior<br/>78%"]

Each piece of evidence updates the belief.

This lesson uses made-up numbers for a made-up condition. Real tests have published sensitivity and specificity; always use those, not these.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In Bayes' theorem, what is the 'prior'?

    The prior P(A) is what you believe before the new evidence; the posterior P(A | B) is what you believe after.
  2. 2. 1% of people have a condition, sensitivity is 95%, false positive rate is 5%. Roughly what is P(sick | positive)?

    0.0095 / (0.0095 + 0.0495) โ‰ˆ 0.161. Most positives come from the large healthy group.
  3. 3. If the condition were much more common (say 30%), with the same test, P(sick | positive) would...

    A higher prior means a larger share of the positives come from truly sick people.
  4. 4. Which denominator is correct in P(sick | +) = P(+ | sick) P(sick) / ?

    P(+) is found by the law of total probability over both groups.

Summary

  • P(A | B) = P(B | A) P(A) / P(B), with P(B) found by total probability.
  • Natural frequencies (counts out of 10,000, say) make the calculation intuitive.
  • Rare conditions give many false positives, even with accurate tests.
  • Posteriors can be used as priors for the next piece of evidence.

Next: Discrete Random Variables and Expected Value