Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 44 ยท Descriptive and Correlational Studies

Surveys and Questionnaires

How to design surveys: question wording, response formats, Likert scales, reverse-coded items, response biases like social desirability and acquiescence, and how to score a scale.

Much of psychology depends on asking people. How do you feel? How often do you exercise? Do you agree with this statement? A survey lets a researcher reach hundreds or thousands of people and measure things that cannot be seen, such as attitudes, beliefs and experiences. But a survey does not record reality. It records what people choose to report, in response to the exact words used.

This lesson covers how to design questions, scales and surveys that give usable data, and how to score a multi-item scale including reverse-coded items.

A questionnaire with a five-point agreement scale, one reverse-coded item flagged, and a score being added up.
A questionnaire is a measuring instrument: the wording, the response scale and the scoring rules all shape the number you get.

What you will learn

  • Common survey formats: closed, open-ended, rating scales, and the Likert scale.
  • Rules for writing clear, unbiased questions.
  • Response biases: social desirability, acquiescence, extreme responding and others.
  • How to score a scale, including reverse-coded items.

Key terms

Closed question: respondents choose from provided options.

Open question: respondents answer in their own words.

Likert scale: a set of statements rated on an agreement scale, usually 5 or 7 points.

Reverse-coded item: an item worded in the opposite direction, whose score is flipped before totalling.

Social desirability bias: answering in a way that looks good to others.

Question types

Closed questions (multiple choice, yes/no, rating) are easy to score and compare. Open questions give rich detail but need coding. Many surveys combine both.

A Likert scale, named after Rensis Likert who described it in 1932, presents statements such as "I often feel rushed," with options from "strongly disagree" (1) to "strongly agree" (5). Responses to several statements about one topic are added to make a scale score. Scales with 5 to 7 points are common. An odd number provides a neutral midpoint, while an even number forces a lean.

Writing good questions

Problem Poor question Better question
Double-barreled (two questions in one) "Do you enjoy school and your teachers?" Ask two separate questions
Leading "Don't you agree that exams are stressful?" "How stressful do you find exams?"
Loaded words "Do you support wasteful spending on ...?" Use neutral wording
Vague terms "Do you often exercise?" "In the past week, on how many days did you exercise for 20 minutes or more?"
Double negatives "I do not disagree that ..." State the idea positively
Memory demands "How many hours did you sleep last year?" Ask about last night or a typical night

Wording can change answers a lot. In a classic 1974 study by Elizabeth Loftus and John Palmer, people who watched films of car crashes estimated higher speeds when the question used the verb "smashed" than when it used "hit." The finding concerned memory, but it illustrates that a single word in a question can move numbers. Question order matters too, because earlier questions can prime later answers.

Pilot testing on a few people from the target group reveals confusing items before the main study.

Response biases

  • Social desirability. People under-report socially disapproved behavior (drinking, prejudice) and over-report approved behavior (voting, exercise). Anonymity and neutral wording help. Crowne and Marlowe developed a well-known scale in 1960 to measure this tendency.
  • Acquiescence. Some people tend to agree with statements whatever they say. Mixing positively and negatively worded items helps to detect and balance it.
  • Extreme or midpoint responding. Individuals and cultures differ in how readily they use the ends of a scale.
  • Recall errors. Memory is reconstructive, and details fade.
  • Non-response. People who do not respond may differ from those who do.

Scoring a scale: reverse-coded items

Imagine a 4-item scale of "study confidence" with 1 (strongly disagree) to 5 (strongly agree).

  1. I feel ready for tests.
  2. I can learn hard material if I work at it.
  3. I often feel I will fail no matter what. (reverse-coded)
  4. I believe my effort pays off.

For item 3, agreement means low confidence. Before adding, flip it: reversed score = (max + min) - score = 6 minus the answer on a 1 to 5 scale. A 5 becomes 1, a 4 becomes 2, and so on. Then total the items (range 4 to 20) or average them.

Scores from such a scale are only useful if the scale is reliable and valid, as covered in the lesson on measures. A scale score is not a diagnosis, and the demonstrations here are only practice in scoring.

Administering surveys

Surveys can be on paper, online, by phone or face to face. Online surveys are cheap and fast but reach only those with access, and attention can be low. Face-to-face interviews allow clarification but can heighten social desirability. Whichever mode you choose, report it, since mode can affect answers.

Common misconceptions

"Anonymous surveys are always honest." They reduce social desirability but do not remove memory errors or misunderstanding.

"Likert items prove a trait is high or low." They give relative standing on a particular measure, not a diagnosis.

"A neutral midpoint is always best." It can attract people who are unsure or unwilling to commit, so the choice depends on the aim.

Try it

Likert scoring with a reverse-coded item (demonstration)
Python Response biases shift the average (simulation)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. 'Do you enjoy school and your teachers?' is a problem because it is

    It asks two things at once, so one answer cannot be interpreted.
  2. 2. On a 1 to 5 scale, the reversed score of an answer of 4 is

    Reversed score is 6 minus the answer, so 4 becomes 2.
  3. 3. Which bias describes answering to look good to others?

    Social desirability bias leads people to over-report approved behavior and under-report disapproved behavior.
  4. 4. Why mix positively and negatively worded items?

    If a person agrees with everything, mixed wording shows up as inconsistency and averages out.
  5. 5. Loftus and Palmer found that wording like 'smashed' versus 'hit' changed

    Speed estimates were higher with 'smashed,' illustrating that wording shifts answers.

Summary

  • Surveys use closed and open questions; Likert scales add item ratings into a scale score.
  • Avoid double-barreled, leading, loaded, vague and negated questions, and pilot test.
  • Response biases such as social desirability and acquiescence shift results.
  • Reverse-coded items are flipped (max + min minus score) before totalling.

Next: Correlation and Causation