Skip to content
Tutorials Center
My learning

Topics

Lesson 12 of 36 · Conjugate Models

The Gamma-Poisson Model

Estimate event rates with the conjugate Gamma-Poisson model: update shape and rate, handle different exposure times, predict future counts with the negative binomial, in Python.

Counts of events in a fixed window, such as support tickets per day, goals per match, or defects per meter of cable, are usually modeled with a Poisson distribution. Its single parameter λ is the average rate. The Gamma distribution is the conjugate prior for λ, which gives a closed-form posterior just as Beta did for probabilities.

The model

Text
yᵢ ~ Poisson(λ)     for i = 1..n   (counts)
λ  ~ Gamma(α, β)    (shape α, rate β)

The Gamma lives on positive numbers, with mean α/β and variance α/β². Remember the SciPy trap from the setup lesson: SciPy takes scale = 1/β.

The update rule

The Poisson likelihood of n counts is proportional to λ^(Σy) e^(−nλ). The Gamma prior is proportional to λ^(α−1) e^(−βλ). Multiply:

posterior ∝ λ^(α + Σy − 1) e^(−(β + n)λ)

That is Gamma(α + Σy, β + n):

  • add the total count to the shape
  • add the number of observations to the rate

So β plays the role of "prior observations" and α "prior total count".

Worked example: tickets per hour

A help desk saw these ticket counts in 8 hours: [3, 5, 2, 4, 6, 3, 4, 5] (total 32). A weakly informative prior Gamma(2, 0.5) says "about 4 tickets per hour, with lots of uncertainty".

Python Conjugate Gamma-Poisson update

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The posterior mean (2 + 32)/(0.5 + 8) = 4.0 sits between the prior mean 4 and the sample mean 4. Since both agree here, the interval simply gets narrower than the prior's.

Different exposure times

Real data rarely has equal windows. If observation i covers tᵢ hours, then yᵢ ~ Poisson(λ tᵢ) where λ is the rate per hour. The update becomes:

Gamma(α + Σy, β + Σt)

You add total exposure instead of the number of observations.

Python Unequal exposure

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A common mistake is to average the per-window rates (4, 4.5, 3.75), which gives the short windows as much weight as long ones. Adding counts and exposures weighs by information.

Posterior predictive: the negative binomial

How many tickets will arrive in the next hour? Averaging Poisson counts over a Gamma posterior gives a negative binomial distribution. It is more spread out than a plain Poisson at the posterior mean, because it includes uncertainty in λ.

Python Predicting the next hour

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The variance of the prediction (about 4.5) exceeds the Poisson variance (about 4.0): the extra is parameter uncertainty.

Checking the Poisson assumption

A Poisson distribution has mean equal to variance. Real counts are often overdispersed, with variance larger than the mean, caused by clustering or varying rates. A quick check: compare the sample variance with the sample mean.

Python Dispersion check

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Here the ratio is 0.43: a little under 1, which with only 8 values is well within chance. A ratio near 1 is consistent with Poisson; a ratio of 2 or more (as in the second sample) warns you to use a negative binomial likelihood or a hierarchical model for the rates (later chapters).

Common priors for rates

  • Gamma(1, small) is vague but not strictly flat; a flat prior on λ ∈ (0, ∞) is improper.
  • To encode "about 10 events, give or take", pick mean α/β = 10 and a standard deviation √α/β = 3, so β = 10/9 ≈ 1.11 and α ≈ 11.1.
  • Jeffreys prior for the Poisson is Gamma(0.5, 0), improper but sometimes used.

Check your understanding

Quick check · 4 questions

of 4 answered You scored / 4 · perfect!

  1. 1. Prior Gamma(3, 1) and counts [2, 4, 3] (n = 3). What is the posterior?

    Shape = 3 + (2+4+3) = 12; rate = 1 + 3 = 4.
  2. 2. With different exposure times, what do you add to the rate parameter?

    The Poisson rate applies per unit of exposure, so the Gamma rate accumulates total exposure.
  3. 3. What distribution describes future counts when averaging over a Gamma posterior?

    The Gamma-Poisson mixture is the negative binomial, which is more dispersed than a Poisson.
  4. 4. A Poisson variable has...

    Mean equals variance; a larger variance signals overdispersion.

Summary

  • Poisson counts with a Gamma(α, β) prior on the rate give a Gamma(α + Σy, β + n) posterior.
  • With varying windows, add total exposure to the rate parameter instead of n.
  • Remember that SciPy's scale is 1/β.
  • The posterior predictive is negative binomial and wider than a plug-in Poisson.
  • Check overdispersion: variance much larger than the mean means Poisson is too simple.

Next: The Normal-Normal Model