Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 57 ยท Estimation

Degrees of Freedom and the t Distribution

Understand degrees of freedom and why small samples need the Student t distribution: heavier tails, t critical values, comparison with the normal, and Python examples.

When we standardize a sample mean we write (x-bar - mu) / (sigma / sqrt(n)), which needs the population standard deviation sigma. But in practice sigma is unknown, so we replace it by the sample standard deviation s. That small substitution adds extra randomness, because s itself varies from sample to sample, and the result is no longer exactly normal. The correct distribution is the Student t distribution, and its shape depends on a number called the degrees of freedom.

What are degrees of freedom?

Degrees of freedom (df) is the number of values in a calculation that are free to vary once certain constraints are fixed. Take 5 numbers whose mean must be 10. You can choose the first 4 freely, but the fifth is then forced, so there are only 5 - 1 = 4 degrees of freedom. When we compute the sample variance we measure deviations from x-bar, which is calculated from the same data, using up one degree of freedom. This is also the reason for dividing by n - 1.

TEXT
one-sample mean:        df = n - 1
two-sample (pooled):    df = n1 + n2 - 2
regression with k predictors: df = n - k - 1
Python Degrees of freedom in a tiny example

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The t distribution

If the data come from a normal population, then

TEXT
t = (x-bar - mu) / (s / sqrt(n))

has a t distribution with n - 1 degrees of freedom. Compared with the standard normal:

  • it is symmetric around 0 and bell shaped,
  • it has heavier tails (more probability far from the center), reflecting the extra uncertainty from estimating sigma,
  • as df grows, it approaches the standard normal. With df above about 30 the two are close; with df = infinity they are identical.
Python t versus normal

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For a 95% interval the normal multiplier is 1.96, but with 5 degrees of freedom it is 2.571. Using the normal value for a small sample would make intervals too narrow, giving false confidence.

Seeing the heavier tails by simulation

You can watch the t distribution appear. Draw many small samples from a normal population, compute t from each (using s), and check how often |t| exceeds the normal cut-off 1.96.

Python Why the normal is wrong for small samples

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Where the normal promises 5% outside, small samples really give much more. The t distribution predicts the simulated numbers closely.

flowchart LR
    A["Known sigma"] --> B["Standard normal (z)"]
    C["Unknown sigma, estimate with s"] --> D["t with n-1 df"]
    D -. "large n" .-> B

The t distribution replaces z when sigma is estimated.

Reading t tables and functions

A t table lists critical values: the t* for which a given probability lies beyond. With a function, use t.ppf(probability, df). For a 95% two-sided interval you need the 97.5th percentile (2.5% in each tail); for 99%, the 99.5th.

Python t critical values

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The t distribution assumes that the population is roughly normal. For moderate or large samples the CLT protects you; for very small samples with strongly skewed data, consider a bootstrap or a nonparametric method.

Other distributions with degrees of freedom

The same idea of degrees of freedom appears in the chi-square distribution (sums of squared standard normals; used for variances and for chi-square tests) and the F distribution (ratio of two scaled chi-squares; used in ANOVA). In each case df tells you how many independent pieces of information stand behind the statistic.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. For a sample of 12 observations, how many degrees of freedom does the one-sample t statistic have?

    df = n - 1 = 11.
  2. 2. Compared with the standard normal, the t distribution has:

    Estimating sigma adds uncertainty, producing heavier tails.
  3. 3. As df increases, the t distribution:

    With many degrees of freedom the estimate s is very accurate and t is nearly normal.
  4. 4. Why use a t critical value instead of 1.96 for n = 6?

    With df = 5 the 97.5th percentile is 2.571, so intervals widen to reflect the extra uncertainty.

Summary

  • Degrees of freedom count independent pieces of information; one-sample mean has n - 1.
  • When sigma is unknown, (x-bar - mu) / (s / sqrt(n)) follows a t distribution with n - 1 df (for normal data).
  • The t has heavier tails than the normal and approaches it as df grows.
  • Use t.ppf for critical values.

Next: Confidence Intervals for Means