Skip to content
Tutorials Center
My learning

Topics

Lesson 19 of 36 ยท Hierarchical Models

The Idea of a Hierarchical Model

Learn why hierarchical Bayesian models treat groups as related, how a hyperprior ties them together, and how the normal-normal model borrows strength across groups.

Real data usually comes in groups: patients in hospitals, students in schools, products in shops, players in teams. Each group has its own parameter, such as a mean outcome or a success rate. You then face a choice that looks harmless but changes your answers a lot. Do you analyze every group on its own, or do you lump everything together?

A hierarchical model (also called a multilevel or partial-pooling model) takes a third road. It says the group parameters are different but related: they are draws from a shared population distribution whose own parameters you also learn from the data. Groups with little data lean on the others, and groups with a lot of data speak for themselves.

Two extreme answers

Suppose you measure the average improvement in six clinics. Clinic j reports a sample mean y_j with a known standard error s_j.

  • No pooling: estimate each clinic by its own y_j. Unbiased, but a clinic with 8 patients gives a wild estimate.
  • Complete pooling: pretend all clinics share one true mean and use the overall average. Stable, but it ignores real differences.

Both are special cases of one model. No pooling means "the groups have nothing in common", complete pooling means "the groups are identical". Reality is almost always in between.

flowchart TD
    H["Hyperparameters: mu, tau"] --> T1["theta 1"]
    H --> T2["theta 2"]
    H --> T3["theta 3"]
    T1 --> D1["data y 1"]
    T2 --> D2["data y 2"]
    T3 --> D3["data y 3"]

Hyperparameters generate group parameters, group parameters generate data. Learning flows in both directions.

The normal-normal model

The classic model has two levels:

TEXT
y_j | theta_j   ~  Normal(theta_j, s_j^2)     (data level, s_j known)
theta_j | mu, tau ~ Normal(mu, tau^2)          (group level)

Here mu is the typical group effect and tau is how much groups really differ. If tau = 0, every theta_j equals mu (complete pooling). If tau is huge, the groups are unrelated (no pooling). The data tells us where tau actually sits.

For a fixed mu and tau, the posterior for one group is a standard conjugate result: a precision-weighted average.

TEXT
posterior mean of theta_j = (y_j / s_j^2 + mu / tau^2) / (1 / s_j^2 + 1 / tau^2)
posterior sd of theta_j   = sqrt(1 / (1 / s_j^2 + 1 / tau^2))

The weight on the group's own data is tau^2 / (tau^2 + s_j^2). Noisy groups (large s_j) get pulled strongly toward mu.

Python Shrinkage as tau changes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With tau = 0.01 all estimates collapse to mu (complete pooling). With tau = 50 they are almost the raw y_j (no pooling). The values in between are partial pooling. Notice how the clinic with the extreme value of 28 and a large standard error is pulled in the most.

Learning tau from the data

In a full Bayesian analysis tau is unknown, so it gets its own prior and its own posterior. That is the point of the hierarchy: the spread of the observed y_j around their center, compared with the noise s_j, tells us about tau. If the observed values vary no more than the noise alone would produce, the data support a small tau and strong pooling.

We can compute the posterior of tau on a grid. With a flat prior on mu it can be integrated out, which gives p(tau | y) proportional to a product of normal densities with variance s_j^2 + tau^2, times a correction factor V^(1/2).

Python Posterior of tau on a grid

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The posterior of tau is wide and leans toward small values: six noisy clinics cannot tell you precisely how different the clinics are. That uncertainty flows into every theta_j when you average over it, which is why hierarchical posteriors are honest about how much pooling is justified.

When to use a hierarchical model

Reach for one when you have:

  • Many groups, some with little data.
  • A reason to believe groups are exchangeable (you cannot tell them apart before seeing data).
  • A wish to predict for a new, unseen group: draw a fresh theta from Normal(mu, tau).

Be careful with very few groups (two or three). The data cannot estimate tau, so the prior on tau matters a lot; see prior sensitivity.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does the parameter tau control in the normal-normal hierarchical model?

    tau is the standard deviation of the group-level distribution. tau = 0 means identical groups, a large tau means unrelated groups.
  2. 2. Which statement about complete pooling is correct?

    Complete pooling is the tau = 0 limit: one shared parameter for all groups.
  3. 3. A group has a very noisy estimate (large standard error). In partial pooling its estimate is...

    The weight on the group's own data is tau^2 / (tau^2 + s^2), which is small when s is large.
  4. 4. Why give tau its own posterior instead of fixing it?

    Fixing tau hides uncertainty. Averaging over its posterior gives honest intervals.

Summary

  • A hierarchical model treats group parameters as draws from a shared population distribution.
  • No pooling and complete pooling are the two extremes (tau large and tau = 0); partial pooling sits between.
  • Group estimates are precision-weighted averages of the group's data and the overall mean.
  • tau is learned from the data, and its posterior is often wide when there are few groups.

Next: Partial Pooling and Shrinkage