Skip to content
Tutorials Center
My learning

Topics

Lesson 45 of 86 ยท Permutations, Combinations and Probability

Conditional Probability and Independent Events

Learn the and/or rules, independent events, conditional probability, Bayes basics, draws with and without replacement, and expected value.

Single-step probability is just counting. Real questions chain steps together: two cards drawn in a row, a test result that depends on which machine made the part, a game with prizes. To handle them you need three ideas: how to combine events with and and or, how information changes a probability (conditional probability), and how to average over outcomes (expected value).

This lesson builds on Probability. The notation is light: P(A) means the probability of event A, P(A and B) that both happen, P(A or B) that at least one happens, and P(A | B) the probability of A given that B has already happened.

The "or" rule

For any two events,

P(A or B) = P(A) + P(B) โˆ’ P(A and B)

The subtraction stops outcomes that belong to both events from being counted twice. If A and B can never happen together (they are mutually exclusive), P(A and B) = 0 and the rule simplifies to P(A) + P(B).

Example 1. One card from a pack: P(red or king). P(red) = 26/52, P(king) = 4/52, and P(red and king) = 2/52 (the two red kings). So P = 26/52 + 4/52 โˆ’ 2/52 = 28/52 = 7/13.

Example 2. One die: P(even or prime). Even: {2, 4, 6}, prime: {2, 3, 5}, both: {2}. P = 3/6 + 3/6 โˆ’ 1/6 = 5/6.

The "and" rule and independence

Two events are independent when one has no effect on the other, for example a coin toss and a die roll, or two separate people's actions. For independent events,

P(A and B) = P(A) ร— P(B)

Example 3. A coin shows heads and a die shows 6: 1/2 ร— 1/6 = 1/12.

Example 4. Three archers hit a target with probabilities 1/2, 1/3 and 1/4, each independently. What is the probability that at least one hits? Use the complement: P(all miss) = 1/2 ร— 2/3 ร— 3/4 = 1/4, so P(at least one hit) = 3/4.

If the events are not independent, use the general rule below.

Conditional probability

The probability of A given B is

P(A | B) = P(A and B) รท P(B)

and rearranged, P(A and B) = P(B) ร— P(A | B). Think of it as shrinking the sample space: once you know B has happened, only outcomes in B remain possible.

Example 5. Two fair dice are rolled and the sum is 8. What is the probability that both dice show even numbers? The sum-8 outcomes are (2,6), (3,5), (4,4), (5,3), (6,2): five of them. The ones with both even are (2,6), (4,4), (6,2). So the answer is 3/5.

Independence can be described with this rule: A and B are independent exactly when P(A | B) = P(A), meaning that learning B tells you nothing about A.

flowchart LR
    S["All outcomes"] --> B["Event B happened<br/>new sample space"]
    B --> AB["A and B<br/>inside B"]
    AB --> R["P(A | B) = P(A and B) / P(B)"]

Conditioning on B restricts attention to B; A is then judged only within it.

Drawing with and without replacement

Take a bag with 5 red and 3 blue balls and draw two, one after the other.

  • With replacement (the first ball goes back): the draws are independent, so P(both red) = 5/8 ร— 5/8 = 25/64.
  • Without replacement: the second draw depends on the first. P(both red) = 5/8 ร— 4/7 = 5/14.

Without replacement, the denominator drops by one and the count of the drawn color drops by one. The other cases follow the same pattern: P(both blue) = 3/8 ร— 2/7 = 3/28, and P(one of each) = 2 ร— (5/8 ร— 3/7) = 15/28 (red-then-blue plus blue-then-red). All three together give 10/28 + 3/28 + 15/28 = 1, as they must.

Drawing two balls "together" gives the same answers as drawing without replacement, which you can also compute with combinations: 5C2 รท 8C2 = 10/28 = 5/14.

JS Exact draws from a bag

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Total probability and Bayes basics

Sometimes an outcome can arrive by several routes. A factory example: machine A makes 60% of the bolts and 2% of its bolts are defective; machine B makes 40% and 5% of its bolts are defective.

Total probability. A bolt is defective by one of two routes (from A, or from B), so

P(defective) = 0.6 ร— 0.02 + 0.4 ร— 0.05 = 0.012 + 0.020 = 0.032.

Bayes' rule reverses the condition. A random bolt is found defective: what is the chance it came from A?

P(A | defective) = P(A and defective) รท P(defective) = 0.012 รท 0.032 = 0.375 (3/8).

So even though A makes most of the bolts, B is more likely to be the source of a faulty one, with probability 0.625. A tree diagram keeps the routes visible.

flowchart LR
    S["Bolt"] -->|"0.6"| A["Machine A"]
    S -->|"0.4"| B["Machine B"]
    A -->|"0.02"| AD["Defective: 0.012"]
    A -->|"0.98"| AO["Good: 0.588"]
    B -->|"0.05"| BD["Defective: 0.020"]
    B -->|"0.95"| BO["Good: 0.380"]

Multiply along a branch to get a route's probability; add the routes that lead to the same result.

Expected value

The expected value of a random quantity is the average you would get over many repetitions: multiply each possible value by its probability and add.

E = ฮฃ (value ร— probability)

Example 6. A fair die: E = (1 + 2 + 3 + 4 + 5 + 6) ร— 1/6 = 3.5. You can never roll 3.5, but it is the long-run average.

Example 7. A lottery ticket costs 10, and wins 500 (a net gain of 490) with probability 1/100. The expected net gain is 490 ร— 1/100 + (โˆ’10) ร— 99/100 = 4.9 โˆ’ 9.9 = โˆ’5. On average the buyer loses 5 per ticket, so the game is unfair.

Expected values add up even when the events are not independent, which makes them a useful shortcut: the expected number of heads in 10 fair tosses is simply 10 ร— 1/2 = 5.

JS Bayes and expected value

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Common mistakes

  • Multiplying probabilities for events that are not independent (draws without replacement). Update the numbers after each draw.
  • Adding P(A) and P(B) without subtracting the overlap when A and B can both happen.
  • Confusing P(A | B) with P(B | A): "a defective bolt came from A" and "a bolt from A is defective" are different probabilities (0.375 versus 0.02).
  • Assuming "mutually exclusive" means "independent". Exclusive events with positive probabilities are in fact dependent: if one happens, the other cannot.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A fair coin is tossed and a fair die is rolled. What is the probability of a head and a 6?

    The events are independent, so multiply: 1/2 x 1/6 = 1/12.
  2. 2. One card is drawn from a standard pack. What is the probability that it is red or a king?

    P = 26/52 + 4/52 - 2/52 = 28/52 = 7/13. The two red kings are subtracted once because they were counted in both groups.
  3. 3. A bag has 5 red and 3 blue balls. Two are drawn one after the other without replacement. What is the probability that both are red?

    5/8 x 4/7 = 20/56 = 5/14. Without replacement the second draw has 4 red balls among 7.
  4. 4. Machine A makes 60% of bolts, 2% defective; machine B makes 40%, 5% defective. A bolt is found defective. What is the probability it came from A?

    P(defective and A) = 0.012 and P(defective) = 0.032, so the answer is 0.012 / 0.032 = 3/8.
  5. 5. A ticket costs 10 and pays 500 in total with probability 1/100 (nothing otherwise). What is the expected net gain per ticket?

    Expected winnings are 500 x 1/100 = 5. Subtract the price of 10 to get an expected net gain of -5.

Summary

  • P(A or B) = P(A) + P(B) โˆ’ P(A and B); for mutually exclusive events the last term is 0.
  • For independent events, P(A and B) = P(A) ร— P(B); "at least one" is easiest through the complement.
  • P(A | B) = P(A and B) รท P(B): knowing B shrinks the sample space to B.
  • Without replacement the draws are dependent, so update the counts after each draw; with replacement they are independent.
  • Total probability adds the routes, Bayes' rule reverses the condition, and expected value is ฮฃ value ร— probability.

Next: Table and Caselet Data Interpretation, the start of the data interpretation chapter.