Skip to content
Tutorials Center
My learning

Topics

Lesson 33 of 36 ยท PyMC-style Workflows

PyMC and ArviZ: The Real Tools

See what a real PyMC and ArviZ analysis looks like, from model block to summary, and learn R-hat, ESS and HDI by computing them yourself in NumPy.

You now know what a probabilistic program does. In practice, most people use PyMC (Python) or Stan with an interface such as CmdStanPy, plus ArviZ for diagnostics and plots. This lesson shows the shape of real PyMC code and explains the output, and it teaches the diagnostics by computing them yourself.

An honest note first. PyMC and ArviZ cannot run on this site: the Python runner in your browser does not include them. The PyMC code below is read-only. Every runnable block on this page is a plain NumPy stand-in that computes the same quantities, and is labelled as such. Run the PyMC code on your own machine after pip install pymc arviz.

A hierarchical model in PyMC (read-only)

Here is the six-clinic model from the hierarchical chapter in non-centered form:

Python
import pymc as pm
import arviz as az
import numpy as np

y = np.array([28.0, 8.0, -3.0, 7.0, -1.0, 1.0])
s = np.array([15.0, 10.0, 16.0, 11.0, 9.0, 11.0])

with pm.Model() as clinics:
    mu = pm.Normal("mu", 0, 10)
    tau = pm.HalfNormal("tau", 5)
    z = pm.Normal("z", 0, 1, shape=len(y))             # non-centered
    theta = pm.Deterministic("theta", mu + tau * z)
    pm.Normal("obs", mu=theta, sigma=s, observed=y)

    idata = pm.sample(2000, tune=1000, chains=4, target_accept=0.95, random_seed=1)
    idata.extend(pm.sample_posterior_predictive(idata))

az.summary(idata, var_names=["mu", "tau", "theta"])
az.plot_trace(idata, var_names=["mu", "tau"])

Compare with the model class from writing a model as code: the same ingredients (priors, likelihood, observed data) with real machinery behind them. pm.sample runs the NUTS algorithm on four chains, tunes the step size during the tune iterations, and returns an InferenceData object that holds the draws, the sampler statistics and (after the second call) posterior predictive samples.

flowchart LR
    A["pm.Model: priors, likelihood"] --> B["pm.sample: NUTS, 4 chains"]
    B --> C["InferenceData: posterior, sample_stats"]
    C --> D["az.summary: mean, hdi, ess, r_hat"]
    C --> E["az.plot_trace / plot_rank"]
    C --> F["az.loo, az.compare"]

Real workflow: model, sample, then everything flows from one InferenceData object.

Reading az.summary

az.summary returns one row per parameter with columns such as mean, sd, hdi_3%, hdi_97%, ess_bulk, ess_tail and r_hat. You already know the ideas:

  • HDI is the highest density interval: the narrowest interval containing a given probability (94% by default in ArviZ).
  • r_hat compares chains; values above about 1.01 are a warning.
  • ess_bulk and ess_tail estimate effective sample sizes for the center and the tails of the posterior; a rule of thumb is to want several hundred for each.
  • Divergences (in sample_stats) signal regions the sampler could not follow, as in the funnel lesson.

HDI, R-hat and ESS in NumPy (stand-in)

To make these diagnostics concrete, here is a stand-in that computes them from an array of draws shaped (chains, draws). It is simpler than ArviZ's rank-normalized versions, but captures the idea. We feed it draws from four independent exact samplers, so R-hat should be about 1.

Python Diagnostics from draws (stand-in for ArviZ)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The sticky chains have an autocorrelation of 0.95, so their effective sample size is a small fraction of the 4000 draws, even though R-hat could look fine. This is why you need both: R-hat tells you whether chains agree, ESS tells you how much each chain is worth.

A bad R-hat example

Chains that explore different regions give a large R-hat. Here two of four chains are stuck in a different mode.

Python Detecting chains that disagree

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

An R-hat far above 1.01 says "do not trust this posterior". Typical causes: a multimodal posterior, too few warm-up iterations, or a model that needs reparameterization.

Model comparison in ArviZ (read-only)

Python
loo_a = az.loo(idata_a)           # PSIS-LOO for model A
loo_b = az.loo(idata_b)
az.compare({"A": idata_a, "B": idata_b})   # elpd_loo, se, weights, differences

This is the PSIS-LOO machinery described in WAIC and LOO; az.compare also reports stacking weights. Both models need pointwise log likelihoods stored in their InferenceData.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does an ArviZ r_hat of 1.3 indicate?

    Values well above 1.01 mean that between-chain variation exceeds within-chain variation.
  2. 2. What is a highest density interval (HDI)?

    For skewed posteriors the HDI differs from the central (equal-tailed) interval.
  3. 3. Why look at ESS as well as R-hat?

    Agreement between chains says nothing about how much independent information each one carries.
  4. 4. On this site, PyMC examples are...

    PyMC is not part of the in-browser Python runner, so the real code is read-only and the stand-ins are labelled.

Summary

  • PyMC models read like the math: priors, likelihood, observed data, then pm.sample.
  • InferenceData holds draws and sampler statistics; ArviZ summarizes and compares them.
  • Read r_hat, ess_bulk, ess_tail, divergences and HDIs before trusting any result.
  • The PyMC code here is read-only; the NumPy stand-ins show the same diagnostics.
  • Next you apply everything to full projects.

Next: Project: A Bayesian A/B Test