Skip to content
Tutorials Center
My learning

Topics

Lesson 9 of 36 ยท Bayes Theorem and Priors

Choosing Priors

Learn how to choose priors: informative, weakly informative and flat priors, prior predictive simulation, conjugate convenience, and sensitivity checks, with runnable Python.

The prior is the most debated part of Bayesian statistics, and the most misunderstood. It is not a way to sneak in opinions; it is an explicit, checkable statement of what values of a parameter are plausible before you look at the data. A prior can be wrong, and with enough data a reasonable prior stops mattering. Your job is to choose one that is honest and to show it does not decide the answer by accident.

What a prior encodes

A prior is a probability distribution over parameter values. It answers: "before the data, how plausible is each value?" Priors come from:

  • Earlier studies or domain knowledge (informative).
  • General constraints, like "a rate is between 0 and 1" or "heights are not 10 meters" (weakly informative).
  • A deliberate attempt to say very little (flat or vague).

Three kinds of prior

Kind Idea Example for a click rate
Informative Real knowledge, strong Beta(20, 180): about 10%, from past campaigns
Weakly informative Rules out the absurd, otherwise lets data speak Beta(2, 8)
Flat / uniform Every value equally likely Beta(1, 1)

Weakly informative priors are the modern default. They regularize (keep estimates sane in small data) without forcing a strong opinion.

Priors as prior data

For a Beta prior on a probability, Beta(a, b) behaves like a - 1 prior successes and b - 1 prior failures (or a and b as pseudo-counts). So Beta(20, 180) is worth about 200 earlier observations, while Beta(1, 1) is worth almost none. This makes the prior's strength concrete: compare it with your real sample size.

Python Prior strength vs sample size

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With 20 observations the strong prior (centered at 0.30 as well) mostly agrees but narrows the interval; the weak prior pulls toward 0.2. As n grows, flat and strong posteriors converge: data overwhelms the prior unless the prior is extreme.

Priors on other parameters

  • Means and effects: Normal(0, s) with s set so the plausible range fits. For a standardized variable, Normal(0, 1) or Normal(0, 2.5) is typical.
  • Scales (standard deviations): must be positive. Use HalfNormal, Exponential or HalfCauchy. Do not use a flat prior on a variance; it can produce extreme values.
  • Rates and counts: Gamma on a Poisson rate.
  • Probabilities: Beta.

Prior predictive simulation

The best test of a prior is to simulate the data it implies, before seeing any data. If prior-simulated data looks absurd, the prior is wrong, even if its parameters look innocent.

Say we model human height in cm as Normal(ฮผ, ฯƒ) with ฮผ ~ Normal(170, 20) and ฯƒ ~ HalfNormal(15).

Python Prior predictive check

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Almost no simulated person is shorter than 0 cm, and about 0.3% are taller than 250 cm, so this prior is a little loose but mostly sensible. If the share were 30%, you would tighten the priors. Chapter 6 develops predictive checks further.

Flat priors are not neutral

A "flat" prior on one scale is not flat on another. A uniform prior on a probability p is not uniform on the odds p/(1-p) or the log-odds. And a flat prior over an unbounded range, such as Uniform(-โˆž, โˆž), is not even a proper distribution. Flatness depends on how you parameterize the problem, so no prior is purely "objective".

Conjugate priors for convenience

Some priors combine with a likelihood to give a posterior in the same family, with no computation. The Beta prior for a Binomial likelihood is the first example. Conjugacy is a convenience, not a requirement: choose a prior for its meaning first, and use conjugacy when it happens to fit (the next chapter).

Sensitivity analysis

Re-run the analysis with a few defensible priors and report whether the conclusion changes. This is cheap and persuasive.

Python Does the conclusion depend on the prior?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

If every reasonable prior says "probably above 30%" the conclusion is robust. Here flat and weak priors say about 75-78%, but a skeptical prior says 6% and an optimistic one almost 100%, so 40 observations cannot settle the question: report that the answer depends on the prior, or collect more data.

A checklist

  1. Write the prior next to the likelihood in your report.
  2. Use weakly informative defaults on a sensible scale (standardize predictors).
  3. Simulate from the prior to see if the implied data is plausible.
  4. Check sensitivity with at least one alternative prior.
  5. Avoid exact zeros and improper flat priors on scales.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is a weakly informative prior?

    Weakly informative priors regularize and exclude nonsense without dominating the data.
  2. 2. A Beta(20, 180) prior is roughly equivalent to how many earlier observations?

    The two shape parameters act as pseudo-counts of successes and failures, so the prior carries about 200 observations of weight.
  3. 3. How do you check that a prior is sensible before seeing the data?

    Simulating data from the prior shows whether it implies plausible observations.
  4. 4. Why is a flat prior not automatically neutral?

    A prior that is flat on one scale (probability) is not flat on another (log-odds).

Summary

  • A prior states what is plausible before the data; it is explicit and checkable.
  • Prefer weakly informative priors; use informative ones when you have real knowledge.
  • Beta shape parameters behave like pseudo-counts; compare prior strength to your sample size.
  • Use positive-valued priors (HalfNormal, Exponential) for scales.
  • Simulate from the prior and run sensitivity checks; data overwhelms reasonable priors as n grows.

Next: Posterior Summaries and Credible Intervals