Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 57 ยท Sampling and Experiments

Populations, Samples and Sampling Frames

Go deeper on populations, samples and sampling frames: parameters versus statistics, why samples vary, sampling error, sample size and population versus sample variance.

Statistics exists because we almost never get to measure everyone. A company wants to know how satisfied all its customers are, a health agency wants the average blood pressure of a country, a factory wants the defect rate of a whole production run. Measuring every unit is too slow, too costly, or impossible (testing a battery until it dies destroys it). So we measure a sample and use it to learn about the population. This chapter is about doing that carefully.

You have already met the basic words in the introduction. Here we make them precise, and focus on the part that beginners miss: the gap between what you know (the sample) and what you want (the population), and how to quantify it.

Population, sample and frame

  • The population is the complete group you want to learn about. It is defined by the question, for example "all adult residents of Canada" or "all bulbs from this week's production".
  • The sample is the subset you actually observe.
  • The sampling frame is the concrete list or mechanism from which you draw the sample, such as a customer database, a phone-number list or a street map. The frame is where theory meets reality.

If the frame does not match the population, you have coverage error. A phone survey drawn from landline numbers misses everyone with only a mobile phone, however large the sample is.

flowchart LR
    A["Target population"] --> B["Sampling frame"]
    B --> C["Sample"]
    C --> D["Statistic"]
    D -. "estimates" .-> E["Parameter"]
    A --> E

The sample is drawn from the frame, but we want to learn about the population.

Parameters and statistics

A number describing a population is a parameter; a number computed from a sample is a statistic. We use Greek letters for parameters and Latin letters for statistics.

Quantity Parameter (population) Statistic (sample)
Mean mu x-bar
Standard deviation sigma s
Proportion p p-hat
Size N n

The parameter is a fixed number (usually unknown). The statistic changes from sample to sample, which means it is itself a random variable.

Sample variance versus population variance

When you have the whole population, the variance divides by N:

TEXT
sigma^2 = sum((x - mu)^2) / N

When you only have a sample, the same formula divided by n tends to come out too small, because the sample mean is fitted to the same data and sits closer to those points than the true mean does. Dividing by n - 1 corrects this (Bessel's correction):

TEXT
s^2 = sum((x - x-bar)^2) / (n - 1)
Python Why divide by n - 1?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The divide-by-n estimates average noticeably below the truth, the n - 1 version lands on it. We say the second one is an unbiased estimator. (More in Point Estimation.)

Samples vary: sampling error

Draw two different samples from the same population and you get two different means. This variation is called sampling error. It is not a mistake by the researcher; it is the unavoidable price of looking at part of the data. What we can do is measure how large it typically is, and shrink it by using bigger samples.

Python Sampling error shrinks with sample size

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Each time the sample size quadruples, the typical error roughly halves. In symbols, the standard deviation of the sample mean is sigma / sqrt(n), called the standard error.

Sampling error versus bias

Two different things can make a sample miss the truth:

  • Sampling error is random. It averages out and shrinks as n grows.
  • Bias is systematic. It pushes the result the same way every time, and a bigger sample does not fix it. Examples: a biased frame, people refusing to answer (nonresponse), leading questions.

A huge biased sample is worse than a small fair one, because it gives a very precise wrong answer. We look at the main sources of bias in Experiments and Bias.

A famous failure: a 1936 magazine poll mailed to millions of people (drawn from phone and car-owner lists) wrongly predicted the US presidential winner, while a much smaller scientific sample got it right. Size cannot repair a bad frame.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. The mean income of ALL households in a city is 52,000. This number is a:

    A numerical summary of the whole population is a parameter.
  2. 2. Which divisor gives an unbiased estimate of the population variance from a sample of size n?

    Dividing the sum of squared deviations by n - 1 (Bessel's correction) removes the downward bias.
  3. 3. A survey uses a list of landline numbers to study all households. This mainly creates:

    The frame leaves out households without landlines, so it does not cover the population.
  4. 4. If you quadruple the sample size, the standard error of the mean becomes about:

    SE = sigma / sqrt(n); quadrupling n halves it.

Summary

  • The population is what you care about, the frame is what you can actually sample from, the sample is what you observe.
  • Parameters describe populations; statistics describe samples and vary from sample to sample.
  • Divide by n - 1 for the sample variance.
  • Sampling error is random and shrinks like 1 / sqrt(n); bias is systematic and does not.

Next: Sampling Methods