Skip to content
Tutorials Center
My learning

Topics

Lesson 22 of 36 ยท Hierarchical Models

Non-Centered Parameterization and the Funnel

Understand the funnel geometry of hierarchical models, why centered samplers struggle, and how the non-centered parameterization restores efficient sampling.

Hierarchical models are powerful, but they have a famous geometric problem called Neal's funnel. When the group-level spread tau is small, the group effects theta_j are squeezed into a narrow neck. When tau is large, they spread into a wide mouth. A sampler that takes steps of one fixed size cannot handle both: steps that suit the mouth are far too large for the neck, and steps that suit the neck crawl through the mouth.

The centered parameterization

The version we have used so far is centered: the group effects are drawn around the mean.

TEXT
theta_j ~ Normal(mu, tau^2)

The posterior over (tau, theta_j) is funnel shaped, because the width of theta_j depends directly on tau.

The non-centered parameterization

Write theta_j as a shift and scale of a standard normal variable instead:

TEXT
z_j     ~ Normal(0, 1)
theta_j = mu + tau * z_j

The model is mathematically identical, but now the sampler works with z_j and tau, which are independent in the prior. The funnel turns into a round blob. Data then reintroduce some dependence, but much less severe when groups have little information, which is precisely when the centered version fails.

flowchart LR
    A["Centered: theta depends on tau"] --> B["Funnel"]
    C["Non-centered: z independent of tau"] --> D["Round blob"]

Same model, different coordinates for the sampler.

A funnel you can sample

Take a toy funnel: v ~ Normal(0, 3) and x | v ~ Normal(0, exp(v/2)). Here v plays the role of log tau and x the group effect. The centered sampler works in (v, x). The non-centered one works in (v, z) with x = exp(v/2) * z. We run a random-walk Metropolis sampler with the same step size on both and ask how well each explores the neck, where v < -3. The exact probability of that region is about 0.159, since v is Normal(0, 3) and -3 is one standard deviation below the mean.

Python Centered versus non-centered sampler

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The non-centered run accepts more proposals and has much lower autocorrelation in v (about 0.24 at lag 50 against about 0.61 for the centered run), so each draw carries more information. Both estimates of the neck probability are noisy (the exact value is 0.159); with a funnel, a correlated chain simply needs far more iterations to be trusted. The exact numbers depend on the seed and the step size, but the contrast in autocorrelation is the point.

A hierarchical model on a grid

Why is the neck important? In the six-clinic model the posterior of tau puts real mass near zero. We can see how much with the exact grid from earlier lessons. A chain that cannot enter the neck would underestimate the probability of strong pooling.

Python Mass near tau = 0

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

When the data are consistent with tau = 0, the posterior piles up near the funnel's neck, and a sampler must handle it. Modern samplers such as Hamiltonian Monte Carlo diagnose this with "divergent transitions", warnings that the sampler could not follow the neck.

When to use which

  • Centered works best when the groups have a lot of data each (the likelihood dominates and decouples theta_j from tau).
  • Non-centered works best when groups have little data or tau may be near zero.
  • Mixed designs are possible: center the data-rich groups and non-center the data-poor ones.
  • Gibbs samplers can use a non-centered form too, but the main benefit is for Metropolis and HMC samplers that use one step size.

To recover theta_j in post-processing, compute mu + tau * z_j for every draw; nothing is lost.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the non-centered parameterization of theta_j ~ Normal(mu, tau^2)?

    The sampler works with the standard normal z_j, and theta_j is rebuilt as a shift and scale.
  2. 2. When does the centered parameterization tend to fail?

    Weak data plus small tau creates the funnel neck that a fixed step size cannot explore.
  3. 3. Does the non-centered form change the model?

    Both forms describe the same joint distribution; only the sampler's coordinates differ.
  4. 4. What warning do HMC samplers raise when they cannot follow a narrow funnel?

    Divergences signal regions of very high curvature, a classic symptom of a centered funnel.

Summary

  • Hierarchical posteriors can form a funnel: narrow where tau is small, wide where it is large.
  • Fixed-step samplers explore the neck badly in the centered parameterization.
  • The non-centered form samples independent z_j and rebuilds theta_j = mu + tau * z_j.
  • Use non-centered when groups have little data, centered when they have a lot.
  • Always check chains (and divergences) near small tau.

Next: Posterior Predictive Checks