Skip to content
Tutorials Center
My learning

Topics

Lesson 37 of 76 Β· Statistics for Data Science

More Probability Distributions - Bernoulli, Exponential, Gamma, Beta, Chi-Squared

Learn the distributions behind tests and models: Bernoulli, uniform, exponential, gamma, beta, chi-squared, t and F, with formulas, Python code and where each one is used.

Probability and Distributions introduced the normal, binomial, Poisson and log-normal families. This lesson goes one level deeper into the distributions that sit underneath tests, waiting-time models and Bayesian estimates: Bernoulli, uniform, exponential, gamma, beta, chi-squared, Student t and F. For each one you will see what it models, its key numbers, and how to compute with it in SciPy.

Do not try to memorize formulas. Learn what real situation each one describes, and remember that every SciPy distribution offers the same methods: pmf or pdf, cdf, sf, ppf and rvs.

Bernoulli: one yes/no trial

A Bernoulli trial is a single experiment with exactly two outcomes, usually called success (1) and failure (0). A visitor buys or does not; a patient responds or does not; a transaction is fraud or is not. A trial is Bernoulli when:

  • there are only two outcomes,
  • the probability of success p is the same every time you could run it,
  • trials do not influence each other.

The Bernoulli distribution is the distribution of that one 0/1 outcome. Its probability mass function is P(X = 1) = p and P(X = 0) = 1 - p. Its mean is p, its variance is p * (1 - p) and its standard deviation is the square root of that. The variance is largest at p = 0.5 (0.25) and shrinks toward 0 as p approaches 0 or 1: a coin that almost always lands heads is predictable.

Add up n independent Bernoulli trials and you get the binomial distribution from the previous lesson. A binomial with n = 1 is exactly a Bernoulli. Logistic regression and every yes/no classifier are, at heart, models of a Bernoulli probability that changes from row to row.

Python Bernoulli: pmf, moments and simulation

Press Run (or Ctrl + Enter) to see the result here.

Program input Β· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Uniform: every value in a range is equally likely

A continuous uniform distribution on [a, b] has a flat density 1 / (b - a). Its mean is the midpoint (a + b) / 2 and its variance is (b - a)² / 12. Use it when you only know the limits: the minute within an hour a bus arrives when you ignore the timetable, a random number generator's output, rounding error in a measurement. It is also the engine of simulation: if U is uniform on 0 to 1, then F⁻¹(U) has any distribution whose inverse CDF F⁻¹ you know. That is how rvs works for many families.

In SciPy the parameters are loc (the start) and scale (the width): stats.uniform(loc=a, scale=b - a). For lotteries and dice you want the discrete version, where each of k outcomes has probability 1/k.

Exponential: waiting time until the next event

If events happen at a steady average rate Ξ» (lambda) per unit of time and independently of each other, the count in a period is Poisson and the waiting time between events is exponential. Its density is Ξ» e^(-Ξ»x) for x β‰₯ 0, its mean is 1 / Ξ» and its variance is 1 / λ². Typical uses: minutes between customers at a bank counter, time between server failures, call durations, the lifetime of a component with no wear-out.

The exponential has a famous quirk, the memoryless property: P(T > s + t | T > s) = P(T > t). If you have already waited 10 minutes for a bus whose waiting time is exponential, the chance of waiting 5 more minutes is exactly what it was when you arrived. The past does not matter. That is realistic for random failures but wrong for anything that wears out or follows a timetable.

SciPy parametrizes it with scale = 1 / Ξ».

Python Exponential waiting times and the memoryless property

Press Run (or Ctrl + Enter) to see the result here.

Program input Β· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In queueing models (call centres, checkout lines, server capacity planning) the exponential describes both inter-arrival times and service times, which is why a surprising amount of capacity planning begins with it.

Gamma: the total waiting time for several events

The gamma distribution generalizes the exponential. If a single waiting time is exponential, the time until the k-th event is gamma with shape k (also called alpha). With shape 1 it is the exponential. With larger shapes it becomes a rounder, more symmetric hump, and for a large shape it looks almost normal. It has mean shape Γ— scale and variance shape Γ— scaleΒ².

Because it is positive and flexible, gamma also models insurance claim sizes, rainfall totals, and repair costs. In SciPy it is stats.gamma(a=shape, scale=scale). Do not confuse the scale with the rate: scale = 1 / rate.

Beta: uncertainty about a probability

The beta distribution lives on the interval 0 to 1, so it is the natural distribution for a probability or a proportion you are unsure about. It has two shape parameters, a and b. Think of a as "successes plus one" and b as "failures plus one". The mean is a / (a + b). Beta(1, 1) is the uniform distribution (no knowledge); Beta(2, 8) leans toward low values; Beta(50, 50) is a tight peak at 0.5; values of a and b below 1 give a U shape piled at both ends.

Its main job is in Bayesian statistics, covered in Bayesian Thinking: beta is the conjugate prior for a Bernoulli or binomial likelihood. Start with Beta(a, b), observe s successes and f failures, and the new belief is simply Beta(a + s, b + f). No integration required.

Chi-squared: sums of squared normals

If Z₁ … Z_k are independent standard normal variables, then Z₁² + … + Z_kΒ² follows a chi-squared distribution with k degrees of freedom. Properties worth remembering:

  • It is never negative, because it is a sum of squares.
  • It is right-skewed, strongly so for small k, and approaches a bell shape as k grows.
  • Its mean is k and its variance is 2k.
  • It is a special case of the gamma distribution (shape k/2, scale 2).
  • Its moment-generating function is (1 - 2t)^(-k/2) for t < 1/2; you will rarely need it, but it is how sums of independent chi-squared variables are shown to be chi-squared again with added degrees of freedom.

Its uses are tests: the sample variance of normal data, goodness-of-fit and independence tests on counts (see Chi-Square Tests). Variants include the non-central chi-squared (the normals have non-zero means, used for power calculations; stats.ncx2) and the generalized chi-squared (a weighted sum of squared normals, which appears in advanced test theory).

SciPy's four workhorse functions map to the ones you may know from R's dchisq, pchisq, qchisq and rchisq: pdf, cdf, ppf and rvs.

Python Gamma, beta and chi-squared in SciPy

Press Run (or Ctrl + Enter) to see the result here.

Program input Β· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Student t and F: the distributions of test statistics

The t distribution describes the standardized sample mean when the population standard deviation is estimated from a small sample. It looks like a normal curve with heavier tails, controlled by its degrees of freedom; with 30 or more it is very close to the normal curve. The F distribution is the ratio of two independent chi-squared variables, each divided by its degrees of freedom, so it is the distribution of a ratio of two variances. It has two degrees-of-freedom parameters. You will use t in Confidence Intervals and t-tests, and F in ANOVA and F-Tests. Both are used almost exclusively to look up critical values and p-values.

Log-normal, briefly

The log-normal distribution is a variable whose logarithm is normal. It is the usual model for incomes, prices and sizes, where effects multiply rather than add. The previous lesson showed how a log transform tames it.

Where each distribution is used

Distribution Typical applications
Bernoulli A single conversion, click, churn or fraud flag; the target of every binary classifier
Uniform Lotteries and dice, random number generation, rounding error, "only the bounds are known" priors
Exponential Queues in banks and call centres, time between telecom calls or server failures, reliability
Gamma Insurance claims, rainfall, total service time for several tasks
Beta Click-through and conversion rates, risk probabilities, Bayesian priors
Chi-squared Goodness-of-fit and independence tests, confidence intervals for a variance
t and F Small-sample means, regression coefficients, ANOVA, comparing variances
Log-normal Incomes, prices, financial returns over time, supply chain lead times

Check your understanding

Quick check Β· 5 questions

of 5 answered You scored / 5 Β· perfect!

  1. 1. A Bernoulli trial has success probability 0.2. What is its variance?

    The variance is p * (1 - p) = 0.2 * 0.8 = 0.16.
  2. 2. Which statement is the memoryless property of the exponential distribution?

    P(T > s + t | T > s) equals P(T > t). The time already waited carries no information.
  3. 3. Which distribution is the natural prior for an unknown conversion rate?

    Beta lives on 0 to 1 and updates simply when combined with Bernoulli or binomial data.
  4. 4. A chi-squared variable with 6 degrees of freedom has which mean?

    The mean equals the degrees of freedom, and the variance is twice that.
  5. 5. A gamma distribution with shape 1 is the same as which distribution?

    The time until the first event is just one exponential waiting time.

Summary

  • Bernoulli: one 0/1 trial, mean p, variance p(1 - p); summed, it gives the binomial.
  • Uniform: equal density on a range; mean (a + b)/2, variance (b - a)Β²/12.
  • Exponential: waiting time at a constant rate, mean 1/Ξ», memoryless; gamma is the wait for the k-th event.
  • Beta: a distribution over probabilities and the conjugate prior for yes/no data.
  • Chi-squared (sum of squared normals), t and F are the distributions of test statistics.
  • In SciPy, use pdf/pmf, cdf, sf, ppf and rvs on all of them.

Next: Confidence Intervals, where the t distribution gives you the critical values.