Skip to content
Tutorials Center
My learning

Topics

Lesson 5 of 37 ยท Collecting and Displaying Data

Collecting Data: Surveys, Sampling and Bias

Learn how to design fair survey questions and choose a sample: simple random, systematic, stratified and convenience sampling, plus common sources of bias.

Good analysis cannot rescue bad data. This lesson is about the first, most important step: collecting data so that they truly represent what you want to study. You will learn where data come from, how to choose a sample, and how to word questions fairly.

Sources of data

  • Primary data are collected by you for your own question: a survey, an experiment, a measurement.
  • Secondary data already exist: government statistics, published tables, earlier studies. They are cheap to get but may not match your question exactly.

Common ways to collect primary data are questionnaires, interviews, observation (counting cars at a junction) and experiments (changing one thing and measuring the effect).

Writing good survey questions

A poorly worded question produces poor data.

Problem Bad question Better question
Leading "Don't you agree the cafeteria food is terrible?" "How would you rate the cafeteria food?"
Two questions in one "Is the library quiet and well stocked?" Ask about quietness and stock separately.
Vague "Do you exercise often?" "How many days in the last week did you exercise for 20 minutes or more?"
Overlapping options "0-5, 5-10, 10-15 hours" "0 to under 5, 5 to under 10, 10 to under 15 hours"
Missing options "Which sport do you like? Football, Tennis" Add "Other" and "None".

Give response options that do not overlap and that cover every possible answer. Test the questionnaire on a few people before you use it.

Sampling methods

How you pick the sample decides whether it is representative.

Random methods

  • Simple random sample: every member of the population has the same chance of being picked, and every group of that size is equally likely. Example: draw 30 names from a hat containing all 600 names.
  • Systematic sample: pick every k-th member after a random start. To choose 30 from 600, use k = 600 / 30 = 20, pick a random start between 1 and 20, then take every 20th name.
  • Stratified sample: split the population into groups (strata) such as year levels, then take a random sample from each group in proportion to its size. Example: if 40% of the school is in Year 7, then 40% of the sample comes from Year 7.
  • Cluster sample: randomly choose whole groups (say, 3 classes) and survey everyone in them.

Non-random methods

  • Convenience sample: ask whoever is easy to reach, such as the first 20 people you meet. It is quick but often biased.
  • Voluntary response: people choose to respond, as in an online poll. Those with strong opinions are over-represented.
  • Quota sample: interviewers fill quotas (for example, 50 women and 50 men) but pick people themselves.
Python Random, systematic and stratified samples

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The stratified sample always matches the year proportions (12, 10, 8), while a simple random sample only matches them approximately, by chance (a systematic sample over an ordered list often matches closely too, but only because the list is sorted by year).

Bias

Bias means the sample or method systematically favors some answers.

  • Selection bias: some members have no chance (or a smaller chance) of being chosen, such as surveying only the morning shift.
  • Non-response bias: the people who do not reply differ from those who do.
  • Response bias: people answer inaccurately, for example to look good ("How much do you study?").
  • Measurement bias: a faulty instrument, such as a scale that reads 200 g too high.

A bigger sample does not fix bias. It only gives a more precise estimate of the wrong thing.

Python Bias beats size

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The biased sample is forty times larger, yet it is far off. This is why how you collect matters more than how many.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which sampling method gives every member of the population an equal chance of selection?

    A simple random sample is chosen by chance so that each member is equally likely to be picked.
  2. 2. A school has 300 boys and 200 girls. A stratified sample of 50 students by gender should contain

    Boys are 60% of the school, so 60% of 50 is 30. Girls are 40%, so 20.
  3. 3. Which question is leading?

    The wording pushes the respondent toward a particular answer.
  4. 4. An online poll that anyone can answer is an example of

    People choose themselves, and those with strong opinions respond more often, which causes bias.

Summary

  • Primary data are collected by you; secondary data already exist.
  • Fair questions are neutral, single-barreled, specific and have non-overlapping options.
  • Random, systematic, stratified and cluster sampling use chance; convenience and voluntary samples are prone to bias.
  • A larger sample cannot fix a biased method.

Next: Frequency Tables and Tally Charts