Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 37 ยท Analysis and Reporting

Apdex Scores and Error Analysis in JMeter

Compute the Apdex satisfaction score from response times, configure thresholds in JMeter, and classify errors by status, message and time to find the real cause.

Two questions follow every test run: "Were users satisfied?" and "What failed, and why?" The first is answered by a satisfaction index such as Apdex, which JMeter's HTML dashboard calculates for you. The second needs systematic error analysis: grouping failures so you can see patterns instead of scrolling through thousands of red lines. JMeter cannot run on this site; the code below is a runnable Python stand-in for both calculations.

Apdex in one formula

Apdex (Application Performance Index) turns response times into one number from 0 to 1. You pick a tolerating threshold T. Every request falls into a class:

  • Satisfied: response time <= T
  • Tolerating: T < response time <= 4T
  • Frustrated: response time > 4T (and every failed request)
TEXT
Apdex = (Satisfied + Tolerating / 2) / Total samples

A score of 0.94 to 1.00 is usually called excellent, 0.85 to 0.93 good, 0.70 to 0.84 fair, 0.50 to 0.69 poor and below 0.50 unacceptable. The thresholds are a convention, so agree on them with your team.

Python Stand-in: calculate Apdex with T = 500 ms

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In JMeter you set thresholds in reportgenerator.properties (or user.properties) with the property jmeter.reportgenerator.apdex_satisfied_threshold and jmeter.reportgenerator.apdex_tolerated_threshold (milliseconds), and you can override them per label:

PROPERTIES
jmeter.reportgenerator.apdex_satisfied_threshold=500
jmeter.reportgenerator.apdex_tolerated_threshold=1500
jmeter.reportgenerator.apdex_per_transaction=Search:300|1200;Checkout:1500|6000

A search page and a report-export page deserve different thresholds, and the dashboard's Apdex table then shows a score per transaction.

Error analysis: start with the numbers

The error rate is the first signal, but the kind of error points to the cause:

Response code or message Typical cause
401, 403 expired or missing token, broken correlation, rate limiting
404 wrong URL, missing test data, an id that does not exist
429 rate limit reached (a good sign that your load is realistic)
500, 502, 503, 504 server crash, overloaded upstream, pool exhaustion, timeouts at a gateway
Non HTTP response code: java.net.SocketTimeoutException the response timeout fired
Non HTTP response code: java.net.ConnectException server unreachable or its listen queue is full
Non HTTP response code: org.apache.http.NoHttpResponseException connection dropped by server or proxy
Assertion failure a 200 whose content is wrong

Two golden rules: first remove test errors (broken correlation, missing data) before you blame the application, and second look at errors over time. Errors that start when the thread count passes a threshold point to a capacity limit. Errors that start at a fixed second may be a deployment, a cron job, or a database failover.

Group errors with a small script

The dashboard has an Errors table, but in practice you often want to slice a .jtl file yourself.

Python Stand-in: classify failures by code, label and time

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Read the result like a detective. Two timeouts on search at 20 to 25 s suggest a slow query that exceeds the 30 s limit; the 401s on login look like a data problem (wrong password in the CSV) rather than a capacity issue; the 503s on checkout may mean an overloaded downstream service. Each hypothesis then needs a server-side check: logs, database slow-query log, application metrics.

Time-outs and "slow errors"

A request that times out after 30 seconds counts as one failed sample but also inflates every average that includes it. Set timeouts explicitly in the HTTP Request (Advanced > Timeouts: connect and response, in ms) so stalled requests end in a predictable time. Keep the values close to what a real client would wait.

Expected errors

Sometimes a test intentionally gets an error: a 404 for an unknown product, a 401 for a wrong password, a 429 for a throttled call. Mark these as successes with a Response Assertion (response code equals 404) so the error rate stays meaningful. Never hide real failures by setting assertions too loosely.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. With T = 500 ms, a request taking 1,800 ms counts as...

    Tolerating is between T (500) and 4T (2000); above 2000 ms or failed means frustrated.
  2. 2. Apdex = (Satisfied + Tolerating / 2) / Total. 6 satisfied, 2 tolerating, 2 frustrated: what is it?

    (6 + 1) / 10 = 0.70.
  3. 3. Many 401 errors appear for one login sampler right at the start. What do you check first?

    Authorization errors from the first moment usually come from broken test data, not capacity.
  4. 4. Why set explicit timeouts on HTTP requests?

    Without timeouts a stalled request can wait very long and distort results.

Summary

  • Apdex turns response times into a 0 to 1 satisfaction score using a threshold T and 4T.
  • Configure Apdex thresholds in the report generator properties, per label if needed.
  • Classify errors by label, code, message and time before guessing at causes.
  • Remove test errors first; then correlate real failures with server-side metrics.
  • Use explicit timeouts and mark expected errors as successes.

Next: HTML Dashboard and Custom Reports