Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 37 ยท CI and Best Practices

Non-GUI Mode and Command-Line Options

Run JMeter tests from the command line: -n, -t, -l, -J, -G, -e -o, logging options, exit codes, properties and a stand-in that builds the command and checks thresholds.

The JMeter GUI is for building and debugging test plans. Real load tests run in non-GUI mode (also called CLI mode): it uses less memory and CPU, can run on servers without a display, and fits into scripts and continuous integration (CI). Every serious test run, from a laptop to a build server, should use it. JMeter cannot run on this site, so this lesson shows the real commands, and the runnable code is a stand-in that builds command lines and checks results the way a pipeline would.

The basic command

Terminal
jmeter -n -t plans/checkout.jmx -l results/run1.jtl -j results/run1.log
  • -n: non-GUI mode.
  • -t: path to the .jmx test plan.
  • -l: results file (JTL).
  • -j: JMeter log file for this run (default is jmeter.log in the working folder).

Start the command from a folder that you control, because relative paths such as CSV files resolve against the test plan's folder or the working directory.

Frequently used options

Option Purpose
-J name=value define a JMeter property for this run (read with ${__P(name)})
-G name=value send a property to remote agents (see setting up remote agents)
-q file.properties load an extra properties file
-p file.properties use another main properties file
-e and -o dir generate the HTML dashboard at the end into an empty folder
-g file.jtl generate a dashboard from an existing JTL file
-r / -R host,host run on remote agents
-X shut the agents down at the end
-H host -P port use an HTTP proxy
-L category=LEVEL set a log level, such as -Ljmeter.engine=DEBUG
-f force deletion of existing result and report files
-v / --version print the version

A typical complete CI invocation:

Terminal
jmeter -n -t plans/api.jmx \
  -Jhost=staging.shop.test -Jthreads=50 -Jrampup=60 -Jduration=600 \
  -l out/results.jtl -j out/jmeter.log \
  -e -o out/report

For this to work, the test plan reads ${__P(host,shop.test)}, ${__P(threads,10)}, ${__P(rampup,30)} and ${__P(duration,300)} in its Thread Group and defaults (see variables and functions). The plan stays the same for every environment; only the properties change.

Exit codes and failing the build

JMeter exits with code 0 when the run finishes, even if requests failed. A plain jmeter -n will therefore never fail your build on a high error rate. You have three options:

  1. Post-process the JTL with a small script that computes the error rate or a percentile and exits non-zero when a threshold is broken (the stand-in below shows the logic).
  2. Use a plugin that adds assertions on a whole test, such as the Taurus wrapper (its passfail module) or Maven/Gradle plugins with failIfErrors style settings.
  3. Add a Duration Assertion per request for individual limits, and a post-run check for totals.

Exit code 1 or other non-zero values do appear when JMeter cannot start (a missing .jmx, invalid properties, a Java error). Handle those separately from "the test ran but the application is too slow".

Stopping a run

  • Press Ctrl+C: stops gracefully, waiting for samples to finish.
  • Send shutdown or stoptest with the shutdown.cmd/shutdown.sh (or stoptest) script in bin, which writes to the control port 4445. shutdown is graceful, stoptest is immediate.
  • Use a Test Action sampler or an Action to be taken after a Sampler error (Stop Test, Stop Test Now) to stop automatically when an error appears.
  • Set a scheduler on the Thread Group (duration) so the test ends on its own, which is what pipelines need.

Useful properties in user.properties

PROPERTIES
jmeter.save.saveservice.output_format=csv
jmeter.save.saveservice.response_data=false
jmeter.save.saveservice.samplerData=false
jmeter.save.saveservice.requestHeaders=false
jmeter.save.saveservice.responseHeaders=false
summariser.interval=30
summariser.out=true
httpclient4.retrycount=0

summariser.interval prints a progress summary to the console every 30 seconds, which is handy in CI logs. Do not save response data or headers in the result file during load: it multiplies the file size and slows the run.

A pipeline gate (stand-in)

This stand-in reads a JTL text, checks pass/fail rules, and returns an exit code, as a CI step would do after jmeter -n finishes. It also prints the command it would run.

Python Stand-in: build a command line and gate on thresholds

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In a real pipeline the last line becomes sys.exit(1 if failed else 0) in a script run after JMeter, so the stage turns red.

Practical advice

  • Keep test plans, data, properties and the wrapper script in version control next to the application.
  • Always write results into a fresh folder per run (use the build number), and publish the report folder as an artifact.
  • Run on an agent that is not shared with other heavy jobs; a noisy neighbour spoils measurements.
  • Keep the JMeter version pinned (a Docker image tag or a checked-in apache-jmeter-x.y version number) so results are comparable.
  • Use the same Java version as the one JMeter was tested with.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does -n do?

    -n is non-GUI mode, the way real tests should be run.
  2. 2. A run has 40 percent errors, but the build is green. Why?

    You need a gate that reads the results and fails the step on a broken threshold.
  3. 3. How do you pass the host name to the test plan from the command line?

    -J defines a property that functions such as __P can read.
  4. 4. Why not store response bodies in the JTL during a load test?

    Saving bodies multiplies I/O and memory; enable it only while debugging.

Summary

  • Build in the GUI, run in non-GUI mode with -n -t -l and optionally -e -o.
  • Use -J properties to keep one plan for all environments.
  • JMeter's exit code does not reflect response-time or error thresholds; add a gate.
  • Keep results out of the repository but plans and scripts in it, and pin the JMeter version.

Next: JMeter in CI/CD Pipelines