Skip to content
Tutorials Center
My learning

Topics

Lesson 43 of 57 · Bayesian Statistics and Simulation

Bayesian Inference with the Beta-Binomial Model

Learn Bayesian updating with the Beta-Binomial model: choose a prior, update with data, get credible intervals and compare with frequentist confidence intervals.

Everything you did in the hypothesis-testing chapters was frequentist: probability meant long-run frequency, parameters were fixed unknowns, and a p-value described how surprising the data would be if the null were true. Bayesian statistics takes a different view. A parameter such as "the conversion rate of this web page" is something you are uncertain about, and uncertainty is described by a probability distribution. Data updates that distribution.

The engine is Bayes' theorem applied to a parameter θ:

posterior ∝ likelihood × prior

  • Prior p(θ): what you believe before seeing the data.
  • Likelihood p(data | θ): how probable the observed data are for each value of θ.
  • Posterior p(θ | data): your updated belief after seeing the data.

The symbol ∝ means "proportional to": the posterior is the product, rescaled so it integrates to 1.

flowchart LR
  A["Prior belief<br/>p(θ)"] --> C["Bayes' theorem"]
  B["Data via likelihood<br/>p(data | θ)"] --> C
  C --> D["Posterior<br/>p(θ | data)"]
  D -->|"becomes the next prior"| A

The Beta distribution as a prior for a proportion

Suppose θ is an unknown proportion (a coin's chance of heads, a click-through rate, the share of voters supporting a measure). It lives between 0 and 1, which is exactly the range of the Beta(α, β) distribution. Think of α − 1 as prior "successes" and β − 1 as prior "failures".

Prior Meaning
Beta(1, 1) flat: every proportion equally likely
Beta(2, 2) mild belief that θ is near 0.5
Beta(20, 20) strong belief that θ is near 0.5
Beta(2, 8) belief that θ is probably low, around 0.2

The mean is α / (α + β). Larger α + β means a narrower, more confident prior.

Conjugacy: the update is just addition

If the data are k successes in n Bernoulli trials (a Binomial likelihood) and the prior is Beta(α, β), the posterior is exactly

Beta(α + k, β + n − k)

This is called a conjugate prior: prior and posterior belong to the same family, so no integration is needed. You just add the successes to α and the failures to β. The posterior mean (α + k)/(α + β + n) is a compromise between the prior mean and the observed proportion k/n; with more data it moves towards k/n.

Python Updating a Beta prior with data

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Credible intervals vs confidence intervals

A 95% credible interval is an interval that contains θ with 95% posterior probability. You may literally say "given the data and the prior, there is a 95% probability that θ is in this range". A frequentist 95% confidence interval cannot be read that way: its 95% refers to the procedure over repeated samples, not to this one interval. Bayesian answers also come directly as probabilities of hypotheses: P(θ > 0.5 | data) is one line of code.

The prior matters less as data grows

With little data the prior has a large influence; with lots of data the likelihood dominates and different reasonable priors lead to nearly the same posterior. This is why Bayesians do a sensitivity analysis: try a few priors and see whether conclusions change. A prior is not a way to "make up" results; it must be justified (from earlier studies, expert knowledge, or deliberately weak).

Python Different priors, growing data

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With 10 observations the three priors disagree noticeably. With 1,000 they nearly agree: the data have swamped the prior.

Sequential updating

Because the posterior from one batch of data is a valid prior for the next, you can update as data arrive: a day at a time, one visitor at a time. The final answer is identical whether you process the data in one go or in pieces. This is natural for A/B testing dashboards and spam filters.

Comparing two proportions

For two versions of a page, give each its own Beta posterior and then compare them. A question like "what is the probability that B beats A?" has no clean frequentist counterpart, but the Bayesian answer is simply the fraction of posterior draws where θ_B > θ_A. Drawing samples from a distribution is the idea behind the next lesson, Monte Carlo Simulation.

Python Probability that B beats A

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Other conjugate pairs and beyond

The same trick works for other models: a Gamma prior with a Poisson likelihood (counts per time period), and a Normal prior with a Normal likelihood (means). For models without conjugacy, which includes most realistic ones, computers approximate the posterior with simulation methods such as MCMC (Markov chain Monte Carlo), implemented in tools like Stan and PyMC.

Check your understanding

Quick check · 4 questions

of 4 answered You scored / 4 · perfect!

  1. 1. A Beta(3, 5) prior is updated with 6 successes and 4 failures. What is the posterior?

    Add successes to alpha and failures to beta: Beta(3 + 6, 5 + 4) = Beta(9, 9).
  2. 2. How is a 95% credible interval correctly read?

    A credible interval is a statement about the posterior distribution of the parameter itself.
  3. 3. What happens to the influence of the prior as the sample size grows?

    With much data, different reasonable priors converge to nearly the same posterior.
  4. 4. Why is the Beta distribution a convenient prior for a coin's probability of heads?

    A proportion lives in [0, 1], and Beta times Binomial gives a Beta again, so updating is just addition.

Summary

  • Bayesian inference updates a prior with the likelihood to get a posterior distribution.
  • For a proportion, Beta(α, β) is a conjugate prior: the posterior is Beta(α + k, β + n − k).
  • Credible intervals give direct probability statements about the parameter.
  • More data reduces the prior's influence; test sensitivity to the prior.
  • Posterior draws make questions like "P(B beats A)" easy.

Next: Monte Carlo Simulation