Skip to content
Tutorials Center
My learning

Topics

Lesson 17 of 30 ยท Running at Scale

Newman, the CLI and CI/CD

Run Postman collections from the command line with Newman, pick reporters, read exit codes, and wire the run into GitHub Actions or GitLab CI.

The Collection Runner is a fine place to run tests while you sit in front of Postman. It is a poor place to run them at 3 a.m., or after every pull request, or on a build server that has no screen. For that you need the same collection to run from a command line, print a report, and tell the machine whether everything passed. That is the job of Newman.

Newman is Postman's command-line collection runner. It is a Node.js program: you give it an exported collection (and optionally an environment and a data file), it sends the requests, runs the scripts and tests, prints the results and exits with a code. A CI system (continuous integration: a server that builds and tests your code on every change) only needs that exit code to turn a build red or green.

Newman cannot run on this site (it needs Node packages, a network and a collection file), so its commands appear in read-only blocks, and the runnable examples are stand-ins written in plain JavaScript: a tiny command parser, a miniature runner, a few reporters and an exit-code rule. They show how a CLI runner works inside, honestly labeled as models.

Installing and running Newman

Newman is installed from npm, either globally or as a project dependency:

Terminal
# Global install (read-only example)
npm install -g newman

# Or as a development dependency of your project
npm install --save-dev newman
npx newman --version

Before you can run anything, you need the collection as a file. In Postman you can export a collection as JSON (the v2.1 format is the standard one), and likewise export an environment. Save both in your repository, for example under a postman/ folder, so they are versioned with the code.

Terminal
newman run postman/shop.collection.json

That alone runs every request in the collection and prints the results to the terminal. In practice you add options.

The options you will use most

Terminal
newman run postman/shop.collection.json \
  --environment postman/staging.environment.json \
  --iteration-data postman/signup-cases.csv \
  --iteration-count 1 \
  --folder "Order life cycle" \
  --delay-request 200 \
  --timeout-request 10000 \
  --env-var "baseUrl=https://staging.shop.example" \
  --bail \
  --reporters cli,junit \
  --reporter-junit-export results/newman.xml

Several options have short forms (-e for --environment, -d for --iteration-data, -n for --iteration-count, -r for --reporters). Check newman run --help for the list in the version you installed, because option names are the part you should not trust to memory.

Choosing what to run

Collection The exported JSON file.
Environment Base URL and other values.
Data file One row per iteration.
Folder Run just one folder.

Same ideas as the Collection Runner, written as flags.

1 / 2

A few of these deserve a closer look.

  • --env-var overrides a single variable without editing the environment file. It is the clean way to inject a secret such as an API key from CI, so the secret never has to be written into a file in the repository.
  • --bail stops the run at the first failure. That saves time on a broken build, at the price of not seeing the other failures.
  • --folder limits the run to one folder. Teams often keep a small "smoke" folder that runs on every change and a larger one that runs nightly.
  • --timeout-request sets how long Newman waits for one response before treating it as a failure.

Never commit real secrets. Exported environments contain the current values of variables, which may include tokens and passwords. Review the file before you commit it, remove secret values, and pass them with --env-var or your CI system's secret storage.

Reporters

A reporter decides what Newman writes when the run finishes. You choose one or more with --reporters (comma separated), and file-based reporters get an export path.

  • cli prints a readable summary to the terminal. This is the default and what you read in a build log.
  • json writes the complete run data as JSON, ideal for custom processing.
  • junit writes JUnit-style XML, a format that many CI systems know how to display as a test report with names, durations and failures.
  • progress shows a progress bar while the run goes.
  • Community reporters such as htmlextra produce a styled HTML report. They are separate npm packages that you install next to Newman, so check each one's own documentation.
Terminal
newman run postman/shop.collection.json \
  -r cli,json,junit \
  --reporter-json-export results/newman.json \
  --reporter-junit-export results/newman.xml

The CLI reporter is for people; JUnit and JSON are for machines. Using more than one at the same time is normal.

Exit codes: how CI knows

Every command-line program ends with an exit code, a small number it hands back to whoever started it. By convention 0 means success and anything else means failure. CI systems read exactly that number to decide whether a step passed.

Newman exits with 0 when the run finished and every test passed. If any assertion fails, or a request cannot be completed, it exits with a non-zero code, and the CI job fails. There is also a flag to suppress the failing exit code (--suppress-exit-code), which is useful when you want to publish a report from a run that you know has failures, but dangerous as a habit, because it hides failures from the build.

Here is the mental model of a whole pipeline step:

flowchart LR
  A["Code change pushed"] --> B["CI starts a job"]
  B --> C["newman run ..."]
  C --> D{"Exit code"}
  D -->|"0"| E["Job passes"]
  D -->|"non-zero"| F["Job fails, report uploaded"]

Newman's only message to the pipeline is the exit code. The reports are for humans to read afterwards.

A stand-in command parser and runner

The first example is the front half of a CLI runner. It parses an argument list like the one you type after newman, applies the options (--folder, --bail, --env-var) to a small built-in collection, and prints a simple result per request. The collection uses pre-baked responses instead of real HTTP, so the output is the same every time. It runs on the server-side Node runner.

Node.js Parsing options and running a fake collection

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The first run shows the failing "Get order" and an exit code of 1, while "Delete order" still runs afterwards. The second run uses --bail and stops at the failure; --env-var replaced the base URL for that run only. The third run executes only the Auth folder and passes. Real Newman has many more options and real HTTP, but the shape (arguments in, results out, one exit code) is exactly this.

A stand-in for reporters and the exit code

The second example takes a finished list of results and renders them the way reporters do: a human summary, a JSON document and JUnit XML. It also applies the exit-code rule. Notice how the XML is just nested text, which is why every CI system can read it, and how the failure message travels in an attribute.

Node.js Rendering a CLI summary, JSON and JUnit XML

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The same failure appears in all three formats. The last two lines show why --suppress-exit-code is risky: the report is the same, yet the build would stay green.

What is real and what is stand-in: the options, reporter names, JUnit as a CI-friendly format and the exit-code convention describe Newman and CI honestly. The parser, the pre-baked responses and the XML writer are written for this lesson and do not reproduce Newman's exact output.

Running Newman in CI

CI configuration lives in a file in your repository. The commands are the same ones you ran on your laptop. Below is a GitHub Actions workflow (read-only example, so check the current action versions when you copy it) that runs the collection on every push to main, on pull requests and every morning, and uploads the report even when the run fails:

YAML
name: API regression

on:
  push:
    branches: [main]
  pull_request:
  schedule:
    - cron: "0 5 * * *"

jobs:
  api-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - name: Install Newman
        run: npm install -g newman
      - name: Run the collection
        run: >
          newman run postman/shop.collection.json
          --environment postman/staging.environment.json
          --env-var "apiKey=${{ secrets.STAGING_API_KEY }}"
          --reporters cli,junit
          --reporter-junit-export results/newman.xml
      - name: Keep the report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: newman-report
          path: results/

The same idea in GitLab CI looks like this:

YAML
api-tests:
  image: node:22
  script:
    - npm install -g newman
    - newman run postman/shop.collection.json -e postman/staging.environment.json --env-var "apiKey=$STAGING_API_KEY" -r cli,junit --reporter-junit-export results/newman.xml
  artifacts:
    when: always
    reports:
      junit: results/newman.xml

Notice three points that matter in any CI system:

  1. The environment file comes from the repository, but the secret comes from the CI system's secret storage and is injected with --env-var.
  2. The report is saved even when the tests fail (if: always() in GitHub, when: always in GitLab), because failing runs are the ones you want to read.
  3. The target must be reachable from the CI machine. A collection that points at localhost on your laptop will not work on a build server; use a staging URL or start the service inside the pipeline.

Postman also offers its own newer command-line tool, the Postman CLI, which can run collections stored in your Postman workspace. Newman remains the simplest choice for collections kept as files in a repository, which is what this tutorial uses. The CI project later in this course puts the whole pipeline together.

Common problems

  • Works in Postman, fails in Newman. Usually a variable lives only in your local Postman (a current value that was never exported), or the script uses a feature that the installed Newman version lacks. Update Newman and compare the environments.
  • Everything times out on CI. The server may not be reachable from the runner, or may be slow when cold. Raise --timeout-request slightly and check the network path.
  • Tests pass but nothing is verified. A collection without assertions exits with 0. Make sure each request has tests, as in Writing tests with pm.test and Chai.
  • Flaky builds. If a run fails at random, find the cause (shared test data, ordering, rate limits) instead of adding retries that hide it.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a CI system look at to decide whether the Newman step passed?

    CI steps pass or fail by exit code. Newman exits with a non-zero code when tests fail, which turns the job red.
  2. 2. Which reporter produces a format that most CI servers can display as a test report?

    JUnit-style XML is widely understood by CI systems, which can show test names, durations and failures from it.
  3. 3. How should an API key reach Newman in a CI job?

    Secrets belong in the CI system's secret storage and are passed on the command line at run time, so they never sit in a file in the repository.
  4. 4. Why do pipelines upload the report with 'always' (if: always() or when: always)?

    By default a step after a failed step is skipped. Running the upload step always keeps the report available for debugging failures.

Summary

  • Newman runs an exported Postman collection from the command line: newman run collection.json.
  • Common options: -e environment, -d data file, -n iterations, --folder, --delay-request, --timeout-request, --env-var, --bail.
  • Reporters choose the output: cli for people, json and junit for machines; community reporters add HTML.
  • The exit code is what CI reads: 0 for a pass, non-zero for failures. Avoid hiding it with --suppress-exit-code.
  • In CI, keep the collection and environment in the repository, inject secrets from secret storage and save reports even on failure.
  • The runnable examples are plain-JavaScript stand-ins, not Newman.

Next: Monitors, performance and load basics.