Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 57 ยท Sampling and Experiments

Sampling Distributions and the Central Limit Theorem

Understand sampling distributions, the standard error and the Central Limit Theorem: why sample means look normal, with simulations of skewed data and proportions in Python.

A statistic such as the sample mean changes from sample to sample. If you could repeat the sampling process many times, you would collect a whole collection of sample means, and that collection has a shape, a center and a spread. Its distribution is called the sampling distribution of the statistic. It is the single most important idea behind inference: it tells you how far a sample result is likely to be from the truth.

The surprise is that this shape is almost always the same bell curve, whatever the population looks like. That result is the Central Limit Theorem (CLT).

The sampling distribution of the mean

Take a population with mean mu and standard deviation sigma. Draw a random sample of size n, compute the mean x-bar, and repeat many times. Then:

TEXT
mean of x-bar   = mu
sd of x-bar     = sigma / sqrt(n)     (the standard error)

The first line says the sample mean is unbiased: on average it hits the target. The second says its spread shrinks with the square root of n.

Python Building a sampling distribution

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The Central Limit Theorem

Statement. If X1, ..., Xn are independent observations from any population with finite mean mu and finite standard deviation sigma, then for large n the sample mean is approximately normal:

TEXT
x-bar  ~  N(mu, sigma / sqrt(n))

The population does not have to be normal. It can be skewed, lumpy, or even two-valued, and the averages still pile up in a bell shape. A frequently used guideline is n >= 30, but it depends on the shape: nearly symmetric data need fewer, heavy skew needs more.

flowchart LR
    A["Any population shape"] --> B["Draw samples of size n"]
    B --> C["Compute each sample mean"]
    C --> D["Means form an approximate bell curve"]

The CLT turns almost any population into a normal sampling distribution.

Watching the CLT happen

The best way to believe the CLT is to simulate it. Start with a strongly skewed population (exponential) and look at the skewness of the sample means as n grows. Skewness is 0 for a symmetric distribution; the exponential has skewness 2.

Python Skewed data, normal means

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

As n grows, the skewness falls toward 0 and the share of means more than two standard errors above mu approaches the normal value of 0.023 (an exponential population has a much fatter right tail at first).

Using the CLT to compute probabilities

Suppose delivery times have mean 30 minutes and standard deviation 10 minutes (shape unknown). What is the chance that the average of 49 deliveries exceeds 32 minutes? The standard error is 10 / 7 = 1.43, so z = (32 - 30) / 1.43 = 1.4.

Python A probability about an average

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The sampling distribution of a proportion

A proportion is a mean of 0/1 values, so the CLT applies. If the population proportion is p, the sample proportion p-hat is approximately

TEXT
p-hat  ~  N(p, sqrt(p (1 - p) / n))

as long as n p >= 10 and n (1 - p) >= 10.

Python Sample proportions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

When the CLT struggles

  • Tiny samples from skewed data: n = 5 from an exponential is still skewed.
  • Heavy tails: distributions with infinite variance (such as the Cauchy) do not obey the theorem; their averages never settle down.
  • Dependent data: time series with strong autocorrelation have much less information than their size suggests.

The CLT is about the distribution of the mean (and sums, proportions). It does not make the raw data normal. A histogram of individual values stays skewed no matter how many you collect.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A population has sigma = 20. What is the standard error of the mean for n = 100?

    SE = 20 / sqrt(100) = 2.
  2. 2. The Central Limit Theorem says that for large n the sample mean is approximately normal:

    The theorem does not require a normal population, only independent draws and a finite variance.
  3. 3. The mean of the sampling distribution of x-bar equals:

    The sample mean is an unbiased estimator of mu.
  4. 4. For p-hat, which condition is a typical guideline for using the normal approximation?

    Both the expected successes and expected failures should be at least about 10.

Summary

  • A sampling distribution describes how a statistic varies from sample to sample.
  • The sample mean has mean mu and standard error sigma / sqrt(n).
  • By the CLT, x-bar is approximately normal for large n regardless of the population's shape.
  • Sample proportions have standard error sqrt(p (1 - p) / n).
  • The CLT is about averages, not individual values.

Next: Experiments and Sources of Bias