Skip to content
Tutorials Center
My learning

Topics

Lesson 3 of 57 ยท Getting Started

Population vs Sample

Learn the difference between a population and a sample, parameters and statistics, why we sample, and how sampling bias and sampling error distort conclusions.

You almost never get to measure everyone. A factory cannot test every light bulb until it burns out, and a pollster cannot ask every voter. So we measure a part and use it to learn about the whole. This lesson names the two sides of that idea and shows what can go wrong.

Population and sample

A population is the complete group you want to know about: all adult residents of a country, every bulb a factory will produce this year. A sample is the subset you actually measure. The size of the population is written N, the size of the sample n.

A census measures the entire population. It is expensive, slow, sometimes impossible (it can be destructive: you cannot crash-test every car) and often unnecessary, because a well-chosen sample answers the question.

Parameters and statistics

A number that describes a population is a parameter. A number computed from a sample is a statistic. We use statistics to estimate parameters.

Quantity Population parameter Sample statistic
Mean mu (ฮผ) xฬ„ (x-bar)
Standard deviation sigma (ฯƒ) s
Proportion p pฬ‚ (p-hat)
Size N n

Parameters are fixed but usually unknown. Statistics are known but change from sample to sample. That last point is the key idea of inference: a statistic is itself a random quantity.

Python A statistic changes with the sample

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Each sample mean lands near the population mean but never exactly on it. The difference is sampling error, and it is unavoidable. Bigger samples shrink it.

Why sample size matters

Sampling error shrinks roughly like 1 / sqrt(n). Quadrupling the sample size halves the typical error. You will derive this properly in the lessons on the central limit theorem and confidence intervals.

Python Bigger samples, smaller error

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Each time n grows by a factor of 4, the typical error drops to about half. This is why a poll of 1,000 people can be surprisingly accurate, and why going from 1,000 to 2,000 helps less than you might expect.

Sampling bias: the bigger danger

Sampling error is random and shrinks with size. Bias is systematic and does not. A biased sample leaves out or over-represents part of the population, so the estimate misses the target in a consistent direction, however many people you ask.

Classic examples:

  • Selection bias: surveying only people who answer a landline phone misses younger households.
  • Voluntary response bias: online polls attract people with strong opinions.
  • Survivorship bias: studying only the companies that still exist ignores those that failed.
  • Non-response bias: if the unhappy customers never reply to a survey, the average looks too rosy.

A huge biased sample is worse than a small fair one. In 1936 a magazine polled over two million readers and wrongly predicted the US election; its sample came from phone and car owners, who were not representative of voters in that Depression year.

flowchart LR
    P["Population<br/>(N people)"] -->|"sampling method"| S["Sample<br/>(n people)"]
    S -->|"compute"| T["Statistic"]
    T -->|"estimates"| PA["Parameter"]

Inference moves from a sample statistic to a population parameter. The sampling method decides how trustworthy that step is.

Simple random sampling

The gold standard is a simple random sample (SRS): every individual, and every group of size n, has the same chance of being chosen. Random selection protects against hidden bias because it does not depend on any characteristic. The Sampling and Experiments chapter covers stratified, cluster and systematic designs.

Seeing bias in action

This example builds a population of incomes, where high earners are easier to reach. A sample made of the easy-to-reach people overestimates the mean, while a random sample of the same size does not.

Python Biased vs random sample

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The biased mean is far above the truth, and making it larger than 200 would not help. Only a better sampling method fixes bias.

Finite populations and the 10% rule

When you sample without replacement from a small population, observations are not quite independent. Many methods assume they are, which is acceptable when the sample is less than about 10% of the population. If you sample 500 of 2,000, take extra care.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. The average height of all 18-year-olds in a country is 171 cm. This number is a...

    It describes the entire population, so it is a parameter.
  2. 2. What does a larger random sample mainly reduce?

    Sampling error shrinks as n grows. Bias caused by a flawed method does not.
  3. 3. An online poll invites anyone to vote. Which problem is most likely?

    People with strong opinions are more likely to respond, so the sample is unrepresentative.
  4. 4. Which symbol usually denotes the sample mean?

    xฬ„ is the sample mean; ฮผ is the population mean.

Summary

  • A population is everyone of interest; a sample is the part you measure.
  • Parameters describe populations; statistics describe samples and estimate parameters.
  • A statistic varies from sample to sample (sampling error), and the error shrinks as n grows.
  • Bias is systematic error from a poor sampling method and is not cured by a bigger sample.
  • Simple random sampling gives every member an equal chance of selection.

Next: Statistics with Python