Skip to content
Tutorials Center
My learning

Topics

Lesson 27 of 36 ยท Usability Testing

Planning a Usability Test

Plan a usability test step by step: set goals, write realistic tasks, recruit the right participants, choose moderated or unmoderated, and prepare a test script.

A usability test watches real people try to complete real tasks with your design while you observe where they struggle. It is the most reliable way to find problems, because opinions about a design ("I think it is clear") are often wrong and watching someone get lost is hard to argue with.

Testing works on anything from a paper sketch to a live product. The quality of the result depends mostly on planning: a clear goal, good tasks and the right participants. This lesson covers that plan.

Step 1: Define the goal and questions

Start with what you need to learn, not with the method. Write 2 to 4 research questions:

  • Can new users find and start a return without help?
  • Do people understand the difference between the two plans on the pricing page?
  • Where in checkout do people hesitate or abandon?

Vague goals ("see if people like it") produce vague results. Specific questions shape tasks, metrics and who you invite.

Step 2: Choose the method

Choice Options When to pick
Moderation Moderated (a facilitator guides) vs unmoderated (participants work alone, recorded) Moderated for exploring why; unmoderated for scale and speed
Location In person vs remote Remote for reach and lower cost; in person for hardware or rich observation
Stimulus Paper, prototype, live product Match the fidelity to the question
Style Think aloud, or retrospective Think aloud is standard for qualitative tests

Step 3: Recruit participants

Test with people who resemble your real users, using a short screener questionnaire. A screener asks about behavior ("How often do you order groceries online?") rather than "Are you a frequent shopper?", and hides which answer qualifies someone.

How many? For finding problems in a qualitative test, five participants per distinct user group uncover most of the common issues, and running several small rounds (test, fix, test again) beats one big round. Quantitative studies that compare numbers need many more, often 20 or more. Plan for no-shows by recruiting a few extra, and consider fair incentives.

Python How many problems do n users find?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The curve flattens quickly: with p = 0.31 five users reveal about 84 percent. When issues are rarer (lower p) you need more people, which is one reason to test repeatedly with small groups. Treat these as model estimates, not guarantees.

Step 4: Write good tasks

A task is a realistic goal given as a scenario, not as instructions. Compare:

  • Bad: "Click Menu, then Orders, then Return item."
  • Good: "You ordered running shoes last week and they are too small. Start a return."

Rules for tasks:

  1. Describe the goal and the context, not the steps or the interface words ("Find the cheapest flight" is fine; "Click the Sort button" is a leak).
  2. Avoid labels that appear on screen, or you test word matching instead of understanding.
  3. Make the task achievable and give any needed data (a fake account, a card number to use).
  4. Order tasks from easy to hard and randomize if tasks influence each other.
  5. Define success beforehand: what counts as completed, completed with help, or failed.
  6. Keep a session to 45 to 60 minutes with 5 to 7 tasks.

Step 5: Decide what to measure

Combine observations and a few simple measures:

  • Task success (completed, partly, failed) and errors.
  • Time on task (only if speed matters).
  • Satisfaction after each task (the Single Ease Question, "How easy or difficult was this task?" on a 1 to 7 scale) and at the end (the System Usability Scale, see UX metrics).
  • Quotes and observed behavior (these explain the numbers).

Step 6: Prepare the script and logistics

A test script keeps sessions consistent. It contains: welcome and consent, the "we test the design, not you" statement, background questions, tasks, follow-up questions, and thanks. Get informed consent for recording, protect privacy, and tell participants they can stop at any time.

Python Test plan skeleton

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A written plan like this can be shared with the team so everyone agrees on what you are testing before the first session. Always run a pilot with a colleague first to catch confusing tasks and technical problems.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which task wording is better?

    A scenario gives a goal and context without revealing the interface steps.
  2. 2. Why recruit with a screener?

    Testing with the wrong people gives misleading results.
  3. 3. What is a pilot test?

    A pilot tests the script, tasks and tools before real participants arrive.
  4. 4. Why run several small rounds instead of one big test?

    Iterating, test, fix, test, finds more and confirms improvements.

Summary

  • Start with specific research questions, then pick the method.
  • Recruit participants who match real users with a behavior-based screener.
  • Write scenario tasks that do not leak interface labels; define success beforehand.
  • Measure success, errors, satisfaction and observed behavior; run a pilot.

Next: Moderating Usability Sessions