Skip to content
Tutorials Center
My learning

Topics

Lesson 1 of 36 ยท Getting Started

What Is Bayesian Statistics?

Learn what Bayesian statistics is: probability as degrees of belief, priors, likelihoods and posteriors, and how data updates beliefs, with runnable Python examples.

Bayesian statistics is a way of reasoning about uncertain things by combining what you believed before you saw data with what the data tells you. The result is not a single guess but a full distribution of plausible values, so you always know how sure you are.

You already reason this way in daily life. If a friend who is usually on time is ten minutes late, you think "traffic" before "they forgot". If three more friends are also late, you update: maybe the train line is down. Bayesian statistics is that habit, made precise with probability.

This course is hands-on. Every idea comes with Python you can run in your browser, using only NumPy and SciPy for the first chapters, so nothing is a black box.

Probability as a degree of belief

In the Bayesian view, a probability describes how strongly you believe a statement, given what you know. "The probability that this coin lands heads is 0.5" means "I have no reason to prefer heads or tails". That works for one-off events too: "the probability that it rains tomorrow is 30%" has no meaningful "repeat tomorrow a thousand times" reading, but it is a perfectly good belief.

Beliefs change as evidence arrives. The rule for changing them correctly is Bayes' theorem, the subject of the next chapter.

The three ingredients

Every Bayesian analysis has the same parts:

Part Symbol Meaning
Prior p(ฮธ) What you believe about the unknown parameter ฮธ before the data
Likelihood `p(data ฮธ)`
Posterior `p(ฮธ data)`

The posterior is proportional to prior times likelihood. That one sentence is most of the subject.

flowchart LR
    A["Prior<br/>what I believed"] --> C["Bayes' theorem"]
    B["Likelihood<br/>what the data says"] --> C
    C --> D["Posterior<br/>what I believe now"]
    D -->|"new data arrives"| A

The posterior of today becomes the prior of tomorrow.

A first example: is the coin fair?

Suppose you flip a coin 10 times and get 7 heads. Is the coin biased? Let ฮธ be the probability of heads. We will list many candidate values of ฮธ, give each a prior weight, multiply by the likelihood of the data, and normalize. This brute-force method is called a grid approximation (we study it properly in Grid Approximation).

Python Coin posterior on a grid

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The most plausible value is 0.7, but the posterior mean is a bit lower (about 0.667), because a flat prior spreads some belief toward 0.5. And we can answer a question frequentist methods find awkward: the probability that the coin favors heads is about 0.88, a direct statement about ฮธ.

What changes when you get more data

With a few flips the posterior is wide. With many flips it narrows around the truth. Watch the same coin (true bias 0.7) with growing sample sizes:

Python More data, narrower posterior

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The standard deviation of the posterior shrinks as data accumulates (roughly with the square root of n). Uncertainty is not a failure to hide; it is the output.

Why people choose Bayesian methods

  • Direct answers. "There is a 95% probability the effect is between 1.2 and 3.4" is what most people think a confidence interval says.
  • Prior knowledge is allowed. Expert knowledge and earlier studies enter through the prior, openly and checkably.
  • Small data. A sensible prior stabilizes estimates when you have few observations.
  • Flexible models. Hierarchical, missing-data and custom models use the same recipe: prior, likelihood, posterior.
  • Sequential learning. New data updates the old posterior; you never start over.

Costs and honest caveats

  • You must choose a prior. A reasonable prior rarely matters with lots of data, but it can with little data, so you check sensitivity (Choosing Priors).
  • Complex models need computation (MCMC), covered in chapter 4.
  • Results are only as good as the model; we learn to test that in the later chapters.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In Bayesian statistics, what does a probability express?

    Bayesian probability measures how strongly we believe a statement, given what we know, which lets us attach probabilities to parameters.
  2. 2. Which pair of ingredients is multiplied (then normalized) to get the posterior?

    Posterior is proportional to prior times likelihood.
  3. 3. What usually happens to the posterior as you collect more data?

    More evidence reduces uncertainty, so the posterior concentrates.
  4. 4. Which of these is a genuine cost of Bayesian analysis?

    Priors and computation are the main practical costs; both can be handled with care.

Summary

  • Bayesian statistics updates beliefs with data using probability.
  • Prior, likelihood and posterior are the three ingredients; posterior is proportional to prior times likelihood.
  • The result is a distribution, so uncertainty is always visible.
  • More data narrows the posterior; yesterday's posterior is tomorrow's prior.
  • Costs: choosing priors and computation.

Next: Bayesian vs Frequentist Thinking