Skip to content
Tutorials Center
My learning

Topics

Lesson 45 of 76 ยท Statistics for Data Science

Bayesian Thinking for Analysts

A practical intro to Bayesian thinking: priors, likelihood, posterior updating with the beta-binomial model, grid approximation and credible intervals, in live Python.

The statistics in the last few lessons answered questions like "if there were no effect, how surprising is this data?" Many people actually want to ask the opposite: "given this data, how likely is it that B is better than A?" Bayesian statistics answers that question directly. It treats unknown quantities, such as a true conversion rate, as things you hold beliefs about, and it uses data to update those beliefs.

You do not need heavy software to use the idea. This lesson builds the logic from one formula and a few lines of NumPy and SciPy: a prior belief, a likelihood from the data, and a posterior that combines them. We compare the results with the intervals from confidence intervals and use them in an A/B-style comparison like the one in A/B testing. Tools like PyMC and Stan automate the same logic for bigger models; they are not needed to understand it.

Bayes' rule in one line

Bayes' rule says how to update a belief:

posterior โˆ prior ร— likelihood

  • The prior is what you believe about the parameter before seeing the data, written as a probability distribution. "Our signup rate is probably near 10%, but I am not sure."
  • The likelihood is how probable the observed data are for each possible parameter value. "9 signups out of 50 visitors would be unlikely if the rate were 1%, and quite likely if it were 18%."
  • The posterior is the updated belief after seeing the data. The symbol โˆ means "proportional to": you rescale so the total probability is 1.
flowchart LR
    P["Prior<br/>belief before data"] --> M(("ร—"))
    L["Likelihood<br/>what the data say"] --> M
    M --> Q["Posterior<br/>belief after data"]
    Q -. "tomorrow's prior" .-> P

Beliefs are updated by data; today's posterior can serve as tomorrow's prior when more data arrive.

With little data the prior matters; with a lot of data the likelihood dominates and reasonable priors lead to nearly the same posterior.

A conjugate example: the beta-binomial model

For a yes/no outcome (signup or not), a convenient prior for the rate is the Beta distribution, with two parameters a and b. Think of them as pseudo-counts of earlier successes and failures. Beta(2, 18) behaves like "I have seen 2 signups in 20 visitors" and has mean 2 / 20 = 0.10. Beta(1, 1) is flat: every rate from 0 to 1 is equally plausible.

The special property of this pair is called conjugacy: after observing s successes and f failures, the posterior is again a Beta distribution with updated parameters:

Beta(a, b) โ†’ Beta(a + s, b + f)

No integration or sampling is needed; you just add counts.

Python Updating a signup-rate belief with new data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The raw data rate is 18% (9 of 50), the prior mean was 10%, and the posterior mean lands between them at about 15.7%, pulled toward the prior because 50 visitors is not much data. The shaded area is a 95% credible interval, and it supports direct statements: given the model and prior, there is a 95% probability that the rate lies between about 8% and 25%, and a 54% probability that it exceeds 15%. Notice too that updating in two batches gives the same result as one update, which is what makes Bayesian methods natural for data that arrive over time.

Credible intervals versus confidence intervals

The two intervals look alike but answer different questions:

Confidence interval Credible interval
Parameter A fixed unknown number Has a probability distribution
Probability statement About the procedure over repeated samples About the parameter given this data
"95% chance the true value is inside" Not valid Valid (given model and prior)
Needs a prior No Yes

The Bayesian interval says what most people want a 95% interval to say. The price is that you must supply a prior, and different priors give different answers, especially with small samples. With plenty of data and a mild prior, the two intervals are nearly identical, as the comparison below shows.

Choosing a prior

Priors raise the most objections, but they are less mysterious than they sound:

  • Weakly informative priors (like Beta(1, 1), or Beta(2, 2)) say little and let the data speak. A good default when you have no strong knowledge.
  • Informative priors use real knowledge: the past conversion rate of similar pages, a physical limit, a result from earlier experiments. Used honestly they sharpen conclusions from small samples.
  • Check sensitivity. Fit with two or three reasonable priors. If the conclusion changes, say so: the data alone do not settle the question.

A strong prior with little data can dominate. The last lines of the final example below show it: after 3 successes out of 10, a flat prior says about 33% while a prior that is firmly centered on 10% barely moves.

Grid approximation: Bayes with arithmetic

Conjugate formulas exist only for special pairs. For anything else, the most transparent method is grid approximation: list many possible parameter values on a grid, compute prior times likelihood at each, and normalize. It needs no calculus and works for any prior shape, which makes it a great way to see the mechanism.

Here the prior is bimodal: "the rate is probably either about 5% (old design) or about 20% (new design), rarely in between." No Beta distribution has that shape.

Python Grid approximation with an unusual prior

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The data (15% of 40 visitors) sit between the two prior peaks, and the posterior reflects that: most of its mass is between 10% and 20%, but it leans toward the 20% hump, the one the data are closer to. Grid approximation works nicely for one or two parameters; with many parameters the grid explodes, which is why software such as PyMC uses MCMC sampling instead.

Comparing two variants: probability that B is better

The most useful Bayesian output for decisions is a plain probability. Draw many samples from each variant's posterior, subtract, and count.

Python Probability that B beats A

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The two intervals are almost the same, as expected with 400 visitors per group and a flat prior. A frequentist would say the interval includes zero and the result is "not significant at 5%". The Bayesian output says something more decision-friendly: B is better with probability about 96%, and the chance that it is better by at least a full point is about 90%. Whether 96% is enough depends on the cost of being wrong, which is a business choice, not a statistical one.

Bayesian methods do not remove the discipline of good design. A biased sample is still biased, and the model (here, independent yes/no outcomes with a constant rate) must be reasonable. Also, the "peeking" problem looks different in this framework, but it does not disappear for free: decision rules based on the posterior still need to be checked by simulation to see how often they would pick the wrong variant.

Tip: Report the posterior as a statement a stakeholder can act on ("B is better with 96% probability; the likely gain is 0 to 8 points"), and always say which prior you used.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In 'posterior โˆ prior ร— likelihood', what does the likelihood represent?

    The likelihood scores each parameter value by how well it explains the observed data. Multiplying by the prior and rescaling gives the posterior.
  2. 2. You use a Beta(3, 7) prior and then observe 5 successes and 15 failures. What is the posterior?

    With a conjugate beta-binomial model you add successes to a and failures to b, giving Beta(3 + 5, 7 + 15) = Beta(8, 22).
  3. 3. Which statement is valid for a 95% credible interval (given the model and prior)?

    A credible interval is a statement about the parameter's posterior distribution. The long-run coverage reading belongs to confidence intervals.
  4. 4. With only 10 observations, two analysts use very different priors and get different posteriors. What is the best response?

    When conclusions depend on the prior, the data are not decisive. Showing a prior sensitivity check is honest and informative, and more data would make the priors matter less.

Summary

  • Bayesian updating combines a prior belief with the likelihood of the data to give a posterior: posterior โˆ prior ร— likelihood.
  • For rates, the beta-binomial model updates by simply adding successes and failures to the Beta parameters.
  • A credible interval is a direct probability statement about the parameter; a confidence interval describes a long-run procedure.
  • Choose weakly informative priors by default, use informative ones only with real knowledge, and check how sensitive the answer is.
  • Grid approximation and posterior sampling let you handle unusual priors and compute decision-ready quantities such as P(B is better than A).

Next: Models, Learning and the Modelling Loop starts the modelling chapter.