Skip to content
Tutorials Center
My learning

Topics

Lesson 20 of 36 ยท Hierarchical Models

Partial Pooling and Shrinkage with Beta-Binomial

See shrinkage in action with a beta-binomial hierarchy: estimate many success rates at once, learn the shared prior from the data, and beat no-pooling estimates.

The previous lesson used a normal model. Success rates, such as conversion rates, hit rates or defect rates, call for a beta-binomial hierarchy. Each group j has n_j trials and k_j successes. Every group has its own rate p_j, and the rates come from a shared Beta distribution:

TEXT
k_j | p_j  ~  Binomial(n_j, p_j)
p_j        ~  Beta(a, b)

If a and b were known, each posterior would be Beta(a + k_j, b + n_j - k_j): a conjugate update you already know from conjugate models. The new part is that a and b are learned from all groups together.

Simulating a league

We build a fake league of 200 players whose true hit rates come from Beta(30, 90) (mean 0.25). Each player has a different number of at-bats, from 5 to 400. Because we know the truth, we can score each method.

Python No pooling versus shrinkage

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The shrunk estimates have a clearly smaller error. The marginal likelihood of the data given (a, b) is a product of beta-binomial terms, so scipy.optimize can find the best (a, b) directly. This shortcut is called empirical Bayes: it plugs in the best hyperparameters instead of averaging over them. A full Bayesian analysis would put a prior on (a, b) and sample; the shrinkage pattern is the same.

Where the improvement comes from

Shrinkage helps most for groups with little data. A player with 5 at-bats and 3 hits has a raw rate of 0.60, which is not believable. The posterior mean (3 + a) / (5 + a + b) sits near the league average because the prior carries about a + b "pseudo-observations" and the player contributes only 5.

Python Who gets shrunk the most?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The table shows two things. First, the weight on a group's own data rises with n, so large groups are barely changed. Second, the error reduction is biggest for the smallest groups. This is the bias-variance trade-off in Bayesian clothing: shrinkage adds a little bias but removes a lot of variance.

Shrinkage is not about the extremes only

A common misunderstanding is that shrinkage "fixes outliers". It pulls every estimate toward the group mean in proportion to its uncertainty. Moderate estimates from small groups are pulled too. And a truly exceptional group with lots of data keeps its extreme value.

flowchart LR
    A["Small group, extreme raw rate"] --> B["Strong pull to the mean"]
    C["Large group, extreme raw rate"] --> D["Little pull: believed"]

The data decide how much to believe an extreme value: sample size is the evidence.

Ranking with uncertainty

If you rank groups by raw rate, tiny samples crowd the top and bottom of the list. Ranking by the posterior mean is better, and ranking by a lower credible bound (say the 5th percentile of Beta(a + k, b + n - k)) is a cautious, "how sure are we it is good" ranking. The ratings project uses this idea.

Mistakes to avoid

  • Pooling unlike groups. Shrinkage assumes exchangeability. If two sub-populations differ systematically (rookies versus veterans), add a predictor or a separate level.
  • Treating the empirical Bayes (a, b) as exact. With few groups it is uncertain; the full Bayesian treatment (Gibbs for a hierarchy) propagates that uncertainty.
  • Forgetting the scale. Rates near 0 or 1 behave better on the logit scale; the normal hierarchy on log-odds is a common alternative.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In the beta-binomial hierarchy, what does the Beta(a, b) distribution describe?

    It is the group-level (population) distribution of the rates p_j.
  2. 2. A group has n = 4 trials. Compared with a group with n = 400 trials, its estimate is...

    The weight on own data is n / (n + a + b), which is small for small n.
  3. 3. What is empirical Bayes?

    Empirical Bayes estimates the prior from the data and plugs it in, which ignores uncertainty in the hyperparameters.
  4. 4. Which situation breaks the logic of partial pooling?

    Pooling borrows strength only when the groups can be considered draws from one population.

Summary

  • A beta-binomial hierarchy learns a shared Beta prior from all groups and shrinks each rate toward it.
  • The weight on a group's own data is n / (n + a + b): small groups are shrunk the most.
  • Shrinkage lowers the overall error by trading a little bias for much less variance.
  • Rank by posterior mean or a lower credible bound, not by raw rates.

Next: Gibbs Sampling for a Hierarchical Model