Bayesian Statistics Tutorial
Learn Bayesian Statistics step by step: 36 lessons in 9 chapters with examples, quizzes, diagrams and projects.
Start learning →
of lessons completed
Getting Started
- What Is Bayesian Statistics? Learn what Bayesian statistics is: probability as degrees of belief, priors, likelihoods and posteriors, and how data updates beliefs, with runnable Python examples.
- Bayesian vs Frequentist Thinking Compare Bayesian and frequentist statistics: what is random, how confidence intervals differ from credible intervals, and when each approach is the better tool.
- A Probability Refresher for Bayesians Review the probability you need for Bayesian statistics: joint, marginal and conditional probability, independence, the law of total probability and distributions, with Python.
- The Bayesian Workflow Learn the steps of a Bayesian analysis: define the question, build a model, choose priors, compute the posterior, check the model and communicate results, with a runnable example.
- Setting Up Python for Bayesian Work Meet the Python tools used in this course: NumPy random generators, SciPy distributions, helper functions for posterior summaries, and how PyMC and ArviZ fit in.
Bayes Theorem and Priors
- Bayes' Theorem Understand Bayes' theorem from the product rule, apply it to a medical test and spam filter, use natural frequencies and odds form, and update beliefs sequentially in Python.
- The Likelihood Function Learn what the likelihood function is, how it differs from probability, how to build one for binomial, Poisson and normal data, why we use logs, and how MLE relates to Bayes.
- Grid Approximation Compute posteriors with grid approximation: one-parameter and two-parameter examples for coins and normal data, sampling from the grid, and why it fails in high dimensions.
- Choosing Priors Learn how to choose priors: informative, weakly informative and flat priors, prior predictive simulation, conjugate convenience, and sensitivity checks, with runnable Python.
- Posterior Summaries and Credible Intervals Summarize a posterior with means, medians, modes, equal-tailed credible intervals and HDIs, compute probabilities of hypotheses, and use ROPE and loss for decisions.
Conjugate Models
- The Beta-Binomial Model Learn the Beta-Binomial conjugate model: why Beta times Binomial gives a Beta posterior, sequential updating, posterior predictive checks and an A/B test, in Python.
- The Gamma-Poisson Model Estimate event rates with the conjugate Gamma-Poisson model: update shape and rate, handle different exposure times, predict future counts with the negative binomial, in Python.
- The Normal-Normal Model Estimate a mean from normal data with a normal prior: precision-weighted updates, shrinkage toward the prior, known versus unknown variance, and posterior predictive intervals in Python.
- The Dirichlet-Multinomial Model Learn the Dirichlet-Multinomial conjugate model for categorical data: pseudo-counts, posterior updates, sampling proportions, comparing categories and posterior predictive probabilities.
MCMC and Sampling
- Why MCMC? Sampling from Posteriors Understand why Bayesian models need MCMC: Monte Carlo integration, rejection sampling, the curse of dimensionality and Markov chains with a stationary distribution, with Python.
- Metropolis-Hastings Write the Metropolis-Hastings sampler from scratch in Python: proposals, acceptance ratios, tuning the step size, burn-in and sampling a two-parameter posterior.
- Gibbs Sampling Learn Gibbs sampling: update one parameter at a time from its full conditional distribution, with a bivariate normal example and a normal model with unknown mean and variance.
- MCMC Diagnostics Check whether an MCMC run can be trusted: multiple chains, trace behavior, R-hat, autocorrelation, effective sample size, Monte Carlo error and stuck chains, with Python code.
Hierarchical Models
- The Idea of a Hierarchical Model Learn why hierarchical Bayesian models treat groups as related, how a hyperprior ties them together, and how the normal-normal model borrows strength across groups.
- Partial Pooling and Shrinkage with Beta-Binomial See shrinkage in action with a beta-binomial hierarchy: estimate many success rates at once, learn the shared prior from the data, and beat no-pooling estimates.
- Gibbs Sampling for a Hierarchical Model Write a Gibbs sampler for the hierarchical normal model with full conditionals, run it in NumPy, summarize group effects and the between-group spread tau.
- Non-Centered Parameterization and the Funnel Understand the funnel geometry of hierarchical models, why centered samplers struggle, and how the non-centered parameterization restores efficient sampling.
Model Checking and Comparison
- Posterior Predictive Checks Learn how to simulate replicated data from the posterior, compare it with the real data using test statistics, and spot overdispersion and other model failures.
- Marginal Likelihood and Bayes Factors Compute the marginal likelihood of a model, form Bayes factors to compare hypotheses, and see why Bayes factors depend strongly on the width of the prior.
- WAIC and Leave-One-Out Cross-Validation Compare Bayesian models by predictive accuracy: compute the log pointwise predictive density, WAIC and exact leave-one-out scores for polynomial regressions in NumPy.
- Prior Sensitivity and Simulation-Based Calibration Test how much your conclusions depend on the prior and verify that your inference code is correct with simulation-based calibration and rank histograms.
Bayesian Regression
- Bayesian Linear Regression Fit a Bayesian linear regression with a conjugate normal-inverse-gamma prior, get posterior draws for coefficients and noise, and make predictive intervals.
- Metropolis Sampling for Regression Write a random-walk Metropolis sampler for a regression with a Student-t likelihood, run several chains, check R-hat and effective sample size, and read the posterior.
- Bayesian Logistic Regression Fit a Bayesian logistic regression with a Gaussian prior, approximate the posterior with the Laplace method, check it against MCMC, and get calibrated predicted probabilities.
- Shrinkage Priors: Ridge and Lasso as Bayesian Models See ridge regression as a Gaussian prior and lasso as a Laplace prior, choose the prior scale with cross-validation, and compare posterior means with maximum likelihood.
PyMC-style Workflows
- Writing a Bayesian Model as Code Build a tiny probabilistic-programming layer in NumPy and SciPy: declare priors and a likelihood, simulate from the prior, and sample the posterior with Metropolis.
- A Full Bayesian Workflow: From Prior Predictive to Checks Follow a complete Bayesian workflow on response-time data: prior predictive checks, a grid fit, fake-data recovery tests, posterior predictive checks and sensitivity analysis.
- PyMC and ArviZ: The Real Tools See what a real PyMC and ArviZ analysis looks like, from model block to summary, and learn R-hat, ESS and HDI by computing them yourself in NumPy.
Projects and Practice
- Project: A Bayesian A/B Test Analyze a conversion experiment end to end: Beta posteriors, probability that B beats A, expected loss, a region of practical equivalence, and the danger of peeking.
- Project: Ranking Products with Hierarchical Ratings Rank 40 products with very different numbers of reviews using a hierarchical beta-binomial model, a fully Bayesian grid over hyperparameters, and top-k probabilities.
- Practice: Bayesian Problems with Solutions Six practice problems on conjugate updates, shrinkage, Bayes factors, posterior predictive checks, regression and decisions, each with a runnable solution you can compare to yours.