Skip to content
Tutorials Center
My learning

Topics

Lesson 15 of 57 ยท Probability Foundations

Conditional Probability and Independence

Learn conditional probability, the multiplication theorem, tree diagrams, the law of total probability and how to test whether two events are independent, with worked Python checks.

Probabilities change when you learn something new. The chance that a random person owns an umbrella is low, but the chance that they own one given that they are carrying it is 100%. Conditional probability is the mathematics of updating a probability in the light of information. It is the bridge to Bayes' theorem and to nearly every statistical model.

Definition

The probability of A given B is written P(A | B). It restricts attention to the outcomes where B occurred and asks how many of them also have A:

P(A | B) = P(A and B) / P(B), defined when P(B) > 0.

Roll a fair die. Let B = "the number is even" = {2, 4, 6} and A = "the number is greater than 3" = {4, 5, 6}. Given B, there are three equally likely outcomes, and two of them (4 and 6) are in A, so P(A | B) = 2/3. The formula agrees: P(A and B) = 2/6, P(B) = 3/6, ratio 2/3. Without the information, P(A) = 3/6 = 1/2.

Conditioning shrinks the sample space to B.

Python Conditional probability by counting

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Here P(A | B) and P(B | A) happen to match, but they are generally different: for D = "greater than 4" the last line shows 1/3 against 1/2. Confusing them is the prosecutor's fallacy: "the chance of this evidence if he is innocent is tiny" is not "the chance he is innocent given this evidence".

Two-way tables

A table of counts makes conditional probabilities easy: restrict to a row or column.

Passed Failed Total
Attended review 45 5 50
Did not attend 30 20 50
Total 75 25 100

P(pass) = 75/100 = 0.75. P(pass | attended) = 45/50 = 0.90. P(pass | did not attend) = 30/50 = 0.60. The difference suggests attendance matters (association), though not necessarily that it causes success.

The multiplication theorem

Rearranging the definition gives the general multiplication rule:

P(A and B) = P(B) * P(A | B) = P(A) * P(B | A)

It extends to chains: P(A and B and C) = P(A) * P(B | A) * P(C | A and B).

Draw three cards without replacement. The chance of three hearts is 13/52 * 12/51 * 11/50 โ‰ˆ 0.0129.

Tree diagrams

A tree diagram lays out sequential events, with probabilities on the branches. Multiply along a path to get the chance of that scenario; add the paths that make up the event you want.

An urn has 3 red and 2 blue balls. Draw two without replacement.

flowchart LR
    S["Start"] -->|"R 3/5"| R1["R"]
    S -->|"B 2/5"| B1["B"]
    R1 -->|"R 2/4"| RR["RR: 3/5 x 2/4 = 0.30"]
    R1 -->|"B 2/4"| RB["RB: 3/5 x 2/4 = 0.30"]
    B1 -->|"R 3/4"| BR["BR: 2/5 x 3/4 = 0.30"]
    B1 -->|"B 1/4"| BB["BB: 2/5 x 1/4 = 0.10"]

The four leaf probabilities add up to 1.0. P(one of each colour) = 0.30 + 0.30 = 0.60.

The law of total probability

If the events B1, B2, ..., Bk split the sample space into non-overlapping parts (a partition), then any event A can be found by summing over the parts:

P(A) = P(A | B1) * P(B1) + P(A | B2) * P(B2) + ... + P(A | Bk) * P(Bk)

A factory has two machines. Machine 1 makes 60% of the items and 2% of them are defective; machine 2 makes 40% and 5% are defective. A random item is defective with probability 0.02 * 0.6 + 0.05 * 0.4 = 0.012 + 0.020 = 0.032.

Python Total probability with two machines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last lines go backwards from the observed result (defective) to its cause, which is exactly what Bayes' theorem formalizes.

Independence

Events A and B are independent if learning that one occurred does not change the probability of the other:

P(A | B) = P(A), equivalently P(A and B) = P(A) * P(B).

For the exam table: P(pass) = 0.75 but P(pass | attended) = 0.90, so passing and attending are not independent. For two flips of a fair coin, P(second heads | first heads) = 0.5 = P(second heads), so they are independent.

Independence is checked with numbers, not intuition. Events with no obvious link may be dependent, and events that seem related may be independent.

Python Test independence numerically

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A and B (two different dice) are independent. A and C are not: knowing the first die is 6 raises the chance of a large sum from 1/6 to 1/2.

Conditional independence

Two events can be dependent overall yet independent once a third fact is known. The number of ice creams sold and the number of sunburns are correlated, but given the temperature they may be independent. This idea underlies causal reasoning and models such as naive Bayes.

Common mistakes

  • Treating P(A | B) as equal to P(B | A).
  • Assuming independence without justification; multiplying for dependent events gives wrong answers.
  • Ignoring the base rate (the share of each group) when combining conditional probabilities.
  • Mixing up "independent" with "mutually exclusive".

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. P(A and B) = 0.12 and P(B) = 0.4. What is P(A | B)?

    0.12 / 0.4 = 0.30.
  2. 2. Two cards are drawn without replacement from a standard deck. P(both are kings)?

    After one king is removed, 3 kings remain among 51 cards.
  3. 3. If A and B are independent, P(A | B) equals...

    Knowing B does not change the probability of A.
  4. 4. A bag: 30% of items come from supplier X with 10% defects, 70% from Y with 4% defects. P(defect)?

    0.3 * 0.10 + 0.7 * 0.04 = 0.03 + 0.028 = 0.058.

Summary

  • P(A | B) = P(A and B) / P(B) restricts the sample space to B.
  • The multiplication rule: P(A and B) = P(B) * P(A | B).
  • Tree diagrams and two-way tables organize conditional calculations.
  • The law of total probability sums over a partition.
  • Events are independent when conditioning does not change their probability.

Next: Bayes' Theorem