Skip to content
Tutorials Center
My learning

Topics

Lesson 11 of 36 ยท Conjugate Models

The Beta-Binomial Model

Learn the Beta-Binomial conjugate model: why Beta times Binomial gives a Beta posterior, sequential updating, posterior predictive checks and an A/B test, in Python.

When the prior and likelihood fit together just right, the posterior has the same form as the prior with updated parameters, and no computation is needed. This is called conjugacy. The Beta-Binomial pair is the classic example and the most useful model in the whole subject: it estimates any probability from successes and failures.

The model

You observe k successes in n independent trials with unknown success probability ฮธ.

Text
k ~ Binomial(n, ฮธ)
ฮธ ~ Beta(a, b)

The Beta distribution lives on [0, 1], so it is a natural prior for a probability. Its mean is a / (a + b), and larger a + b means a more concentrated prior.

Why the posterior is Beta

Prior density โˆ ฮธ^(a-1) (1-ฮธ)^(b-1). Likelihood โˆ ฮธ^k (1-ฮธ)^(n-k). Multiply:

posterior โˆ ฮธ^(a+k-1) (1-ฮธ)^(b+n-k-1)

That is the shape of a Beta(a + k, b + n โˆ’ k) distribution. So the update rule is simply:

  • add the number of successes to a
  • add the number of failures to b

Beta-Binomial in one slide

Prior Beta(a, b)
Data k successes, n-k failures
Posterior Beta(a+k, b+n-k)

Just add the counts.

1 / 2

Updating in code

A factory tests a new part: 3 failures in 40 units. Prior Beta(1, 19) expresses "failure rates are usually around 5%".

Python Conjugate update

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The posterior is Beta(4, 56), mean 4/60 = 0.0667. The prior (worth 20 pseudo-observations) and the data (40 observations) are blended in proportion to their weight: the posterior mean is a weighted average of the prior mean 0.05 and the sample proportion 0.075, with weights 20 and 40.

Sequential updating

Because the posterior is again a Beta, you can update in batches and get the same answer as updating all at once.

Python Batch vs all-at-once

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both routes end at Beta(4, 56). Exchangeable data order does not matter, and yesterday's posterior is today's prior, literally.

Posterior predictive distribution

What will the next 100 units show? The posterior predictive averages over uncertainty in ฮธ: draw a ฮธ from the posterior, then simulate data from the Binomial with that ฮธ.

Python Posterior predictive for the next batch

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The posterior predictive is wider than the plug-in prediction because it includes uncertainty about ฮธ itself. Ignoring that gives overconfident forecasts.

A/B testing with Beta-Binomial

Two page designs: A had 120 conversions out of 2,400 visitors, B had 150 out of 2,500. Fit each with a flat prior and compare.

Python A/B test

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The output speaks directly about the decision: the probability B is better, the expected improvement, and its uncertainty.

Beyond Beta(1, 1)

  • Beta(1, 1): uniform.
  • Beta(0.5, 0.5) (Jeffreys): puts more mass near 0 and 1; invariant under reparameterization.
  • Beta(2, 2): gently favors the middle.
  • Strong prior from history: pick a and b so the mean matches and a + b equals the number of earlier trials you trust.

A mean m and a strength s give a = m s and b = (1 โˆ’ m) s.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A Beta(2, 8) prior meets 6 successes and 14 failures. What is the posterior?

    Add successes to a and failures to b: Beta(2 + 6, 8 + 14) = Beta(8, 22).
  2. 2. What does a + b represent in a Beta(a, b) prior?

    Larger a + b means more concentration; it acts like that many earlier trials.
  3. 3. Why is the posterior predictive wider than a plug-in prediction using the posterior mean?

    Averaging over the posterior propagates parameter uncertainty into the prediction.
  4. 4. Does updating one batch at a time give a different answer than updating all the data at once?

    Because each posterior becomes the next prior, the final Beta has the same total counts.

Summary

  • Beta prior + Binomial likelihood gives a Beta posterior: add successes to a, failures to b.
  • a + b is the prior's strength in pseudo-observations; the posterior mean blends prior and data by weight.
  • Updating in batches equals updating all at once.
  • The posterior predictive averages over ฮธ and is wider than a plug-in forecast.
  • Comparing two Beta posteriors by sampling gives P(B > A) and the distribution of lift.

Next: The Gamma-Poisson Model