Skip to content
Tutorials Center
My learning

Topics

Lesson 56 of 68 ยท Probability and Statistics for Engineers

Gamma, Chi-Square and Student's t Distributions

Learn the continuous distributions behind waiting times and statistical tests: gamma, chi-square and Student's t, with PDFs, means, variances, how they connect, and simulations.

Probability for Engineers introduced the density (PDF) and cumulative distribution (CDF) of a continuous random variable, and then the normal and exponential distributions. Three more continuous families appear constantly in engineering and statistics:

  • the gamma distribution, for total waiting times and for positive, right-skewed quantities such as rainfall or repair times;
  • the chi-square distribution, for sums of squared errors and tests of fit and variance;
  • the Student's t distribution, for estimates of a mean when the spread is itself estimated from a small sample.

They are close relatives: chi-square is a special gamma, and t is built from a normal and a chi-square. Seeing the family tree makes all three easier to remember.

A reminder: density, moments and the gamma function

A continuous random variable X has a density f(x) >= 0 with total area 1. Its mean is E[X] = integral of x f(x) dx and its variance is E[(X - mean)**2]. More generally the k-th moment is E[X**k] and the moment generating function E[e**(tX)] packs all moments into one function. The gamma function Gamma(a) = integral from 0 to infinity of u**(a-1) e**(-u) du (see Gamma and Beta Functions) satisfies Gamma(n) = (n - 1)! for integers and is the normalizing constant for the next two densities.

The gamma distribution

X ~ Gamma(k, theta) with shape k > 0 and scale theta > 0 has density

f(x) = x**(k-1) * e**(-x/theta) / (Gamma(k) * theta**k) for x > 0.

Its mean is k * theta and its variance is k * theta**2.

What makes it useful:

  • With k = 1 it is the exponential distribution with mean theta.
  • For integer k it is the Erlang distribution: the sum of k independent exponential waiting times, so it describes the time until the k-th event of a Poisson process.
  • The shape controls skewness: small k gives a sharp spike near zero, large k approaches a bell shape (by the central limit theorem).
  • Sums of independent gammas with the same scale are gamma: Gamma(k1, theta) + Gamma(k2, theta) = Gamma(k1 + k2, theta).
Python Gamma: formula, moments and the sum of exponentials

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The chi-square distribution

If Z1, ..., Zv are independent standard normal variables, then

X = Z1**2 + Z2**2 + ... + Zv**2

follows the chi-square distribution with v degrees of freedom, written X ~ chi2(v). It is the gamma distribution with shape v/2 and scale 2, so its density is f(x) = x**(v/2 - 1) e**(-x/2) / (2**(v/2) Gamma(v/2)) for x > 0, its mean is v, and its variance is 2v.

Why you care:

  • Variance of normal data: if s**2 is the sample variance of n normal observations with variance sigma**2, then (n - 1) s**2 / sigma**2 ~ chi2(n - 1). This builds confidence intervals for a variance.
  • Sums of squared standardized errors (least squares, goodness-of-fit) follow chi-square when the model is right. The chi-square test compares observed and expected counts with sum of (O - E)**2 / E.
  • Additivity: independent chi2(a) and chi2(b) add to chi2(a + b).
Python Chi-square from squared normals, and the sample variance

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Student's t distribution

Suppose you measure n values from a normal population and compute the sample mean xbar and sample standard deviation s. If you knew the true sigma, the statistic (xbar - mu)/(sigma/sqrt(n)) would be standard normal. But you only have s, an estimate. Replacing sigma by s makes the statistic a little more spread out, because s is itself random. William Gosset, writing as "Student", found its exact distribution.

If Z ~ N(0, 1) and V ~ chi2(v) are independent, then

T = Z / sqrt(V / v)

follows the t distribution with v degrees of freedom. For a sample of size n, (xbar - mu)/(s/sqrt(n)) has v = n - 1 degrees of freedom.

Key facts:

  • It is bell-shaped and symmetric about 0, but with heavier tails than the normal: extreme values are more likely, especially for small v.
  • Mean 0 (for v > 1) and variance v / (v - 2) (for v > 2); with v = 1 it is the Cauchy distribution, which has no mean.
  • As v grows it approaches the standard normal; beyond about v = 30 the two are close.
  • The heavier tails mean wider confidence intervals for small samples: the critical value for a 95% two-sided interval is 4.30 for v = 2, 2.78 for v = 4, 2.23 for v = 10, and only 1.96 in the normal limit.
Python The t distribution: construction and heavy tails

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The family tree

flowchart TB
    N["Standard normal Z"] -->|"sum of v squares"| C["Chi-square(v)"]
    C -->|"same as"| G["Gamma(v/2, scale 2)"]
    E["Exponential"] -->|"sum of k"| G2["Gamma(k, scale)"]
    N -->|"Z / sqrt(chi2(v)/v)"| T["Student's t(v)"]
    C --> T

Normals, exponentials, gammas, chi-squares and t distributions are all built from each other.

Distribution Typical use Mean Variance
Gamma(k, theta) waiting for the k-th event, positive skewed data k theta k theta**2
Chi-square(v) variance tests, goodness of fit, sums of squared errors v 2v
Student's t(v) mean of small normal samples, regression coefficients 0 v/(v-2)

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which distribution describes the time until the 4th event of a Poisson process (rate 1 per hour)?

    The k-th arrival time is the sum of k independent exponential gaps, which is a gamma (Erlang) distribution with shape k.
  2. 2. What are the mean and variance of a chi-square distribution with v degrees of freedom?

    Chi-square(v) is Gamma(v/2, scale 2): mean (v/2) x 2 = v and variance (v/2) x 4 = 2v.
  3. 3. If Z1, Z2, Z3 are independent standard normals, Z1^2 + Z2^2 + Z3^2 follows:

    That sum is the definition of the chi-square distribution with 3 degrees of freedom.
  4. 4. How does the t distribution compare with the standard normal?

    Using an estimated standard deviation adds uncertainty, so extreme values are more likely. With many degrees of freedom the estimate becomes accurate and t becomes normal.
  5. 5. Which special case of the gamma distribution is the exponential distribution?

    With k = 1 the density becomes e^(-x/theta)/theta, the exponential density with mean theta.

Summary

  • Gamma(k, theta) models sums of exponential waits and positive skewed data; mean k theta, variance k theta^2; k = 1 is exponential.
  • Chi-square(v) is the sum of v squared standard normals, equal to Gamma(v/2, scale 2), with mean v and variance 2v; it underlies variance and goodness-of-fit tests.
  • Student's t(v) is Z / sqrt(V/v): symmetric, heavy-tailed and tending to the normal as v grows.
  • (n - 1) s^2 / sigma^2 follows chi-square and (xbar - mu)/(s/sqrt(n)) follows t, both with n - 1 degrees of freedom, for normal data.

Next: Statistics and Curve Fitting, then Inferential Statistics.