Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 36 ยท UX in Teams and Metrics

UX Metrics: SUS and Task Success

Measure user experience with the System Usability Scale, task success rate with confidence intervals, time on task and the HEART framework, using runnable calculators.

Teams argue less and decide faster when they can point to numbers. UX metrics turn "the checkout feels clunky" into "62 percent of participants completed checkout, and the satisfaction score is below average". Numbers do not replace the qualitative insight from observation, but together they make a convincing case and show whether changes help.

This lesson covers behavioral and attitudinal metrics, how to calculate the System Usability Scale (SUS), how to treat small samples honestly, and the HEART framework for product teams.

Two kinds of measures

  • Behavioral (what people do): task success, errors, time on task, clicks, conversion, retention.
  • Attitudinal (what people say): satisfaction, perceived ease, trust, likelihood to recommend.

Another split: qualitative (why) versus quantitative (how many). Use both: numbers show where the problem is, observation explains it.

Task success rate

The simplest and most useful usability metric: of the people who tried a task, how many completed it? Define success before testing (see test planning). Report binary success (yes/no) or levels (success, success with difficulty, failure).

With small samples you must show uncertainty. If 8 of 10 participants succeeded, the true rate is not exactly 80 percent. The adjusted Wald interval is a good small-sample confidence interval for a success rate.

Python Success rate with a confidence interval

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice how wide the interval is for 10 people (roughly 48 to 95 percent) and how it narrows for 100. This is why small tests should find problems, not claim precise rates.

Time on task

Measure the time from start to success for successful attempts only. Task times are usually skewed (a few very long attempts), so report the median or the geometric mean instead of the plain average, and avoid timing during concurrent think-aloud.

Python Median vs mean time

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

One outlier drags the mean to about 63 seconds while the median stays at 40, which better describes a typical user.

System Usability Scale (SUS)

SUS is a 10-statement questionnaire answered on a 1 (strongly disagree) to 5 (strongly agree) scale. Odd statements are positive ("I thought the system was easy to use"), even ones negative ("I found the system unnecessarily complex"). The score runs 0 to 100. To compute it:

  • Odd items: contribution = answer minus 1.
  • Even items: contribution = 5 minus answer.
  • Sum the ten contributions and multiply by 2.5.

A rough benchmark: the average SUS across many studies is about 68, so below 68 is below average and above 80 is good. It is a relative score, not a percentage.

Python Compute SUS

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Other attitudinal measures

  • Single Ease Question (SEQ): after each task, "Overall, how difficult or easy was the task?" (1 to 7).
  • Net Promoter Score (NPS): "How likely are you to recommend...?" (0 to 10); promoters (9 to 10) minus detractors (0 to 6). Popular, but not a usability diagnostic.
  • CSAT: satisfaction rating after an interaction.
  • Customer Effort Score: how much effort it took to get something done.

Product analytics and HEART

Live products also give behavioral data: funnels, drop-off, search with no results, support tickets, rage clicks. Google's HEART framework helps choose metrics that represent UX:

HEART, five lenses

Happiness Satisfaction, NPS, ratings.
Engagement How often and how deeply people use it.
Adoption New users or new feature uptake.
Retention Do people come back?

Fifth lens, Task success: completion, errors, time.

1 / 2

For each lens you set a goal (what we want), signals (what shows it) and a metric (the number we track). Without goals, metrics become vanity numbers.

Good metric habits

  • Tie each metric to a decision it will inform.
  • Track change over time and against a baseline, not a single reading.
  • Combine numbers with observed reasons.
  • Beware proxies: time on page can mean engagement or confusion.
  • Respect privacy: collect only what you need.

Next you can test changes with experiments: see A/B testing for UX decisions.

Check your understanding

Quick check ยท 3 questions

of 3 answered You scored / 3 ยท perfect!

  1. 1. How is a SUS score computed?

    Odd items minus 1, even items 5 minus answer, then the sum times 2.5 for 0 to 100.
  2. 2. Why use the median for task time?

    A few very long attempts inflate the mean; the median reflects a typical user.
  3. 3. What does HEART help with?

    HEART covers Happiness, Engagement, Adoption, Retention and Task success.

Summary

  • Use behavioral and attitudinal measures, quantitative and qualitative together.
  • Report task success with a confidence interval and time with the median.
  • SUS gives a 0 to 100 score; about 68 is average.
  • HEART turns goals into signals and metrics; avoid vanity metrics.

Next: A/B Testing for UX Decisions