Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 36 ยท Projects and Practice

Project: A Performance Gate for Pull Requests

Build a performance gate: a k6 smoke test exports a summary, a small Node script compares it with a stored baseline and budgets, posts a Markdown table and fails the build on regressions.

Thresholds inside the k6 script answer "is it fast enough in absolute terms?". A performance gate adds a second question that teams care about just as much: "did this change make things worse than before?". In this project you build the whole loop: a smoke test, a summary export, a comparison script with a baseline and a pull request comment.

Honesty note. The k6 script and the CI workflow are read-only examples (neither k6 nor a CI runner is available here). The comparison script is plain Node and runs in the editors below on made-up summary data, so you can develop and test the gate logic without any pipeline.

The design

flowchart LR
    A["Pull request"] --> B["Deploy preview environment"]
    B --> C["k6 smoke test<br/>--summary-export"]
    C --> D["gate.mjs compares<br/>summary vs baseline"]
    D -->|within budget| E["Green check + Markdown table"]
    D -->|regression| F["Red check + table of what regressed"]
    G["Main branch nightly"] --> H["Update baseline.json"]
    H --> D

The gate combines absolute thresholds (inside k6) with relative budgets (against a baseline).

Three pieces of data are involved:

  1. summary.json from the current run (k6 run --summary-export=summary.json smoke.js).
  2. baseline.json from a known-good run on the main branch, stored in the repo or as a CI artifact.
  3. Budgets: how much worse than the baseline is acceptable (for example +15 % on p(95), +0.5 percentage points on errors).

Step 1: the smoke script

JavaScript
import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  vus: 5,
  duration: '45s',
  thresholds: {
    http_req_failed: ['rate<0.02'],       // absolute guard rails
    http_req_duration: ['p(95)<800'],
    checks: ['rate>0.98'],
  },
};

export default function () {
  const res = http.get(`${__ENV.BASE_URL}/api/products`, { tags: { name: 'ListProducts' } });
  check(res, { 'status 200': (r) => r.status === 200 });
  sleep(1);
}

Smoke tests on shared CI infrastructure are noisy, so keep the absolute thresholds generous and let the relative comparison catch regressions with a margin that exceeds the noise.

Step 2: the comparison script

The k6 summary export shape depends on the version: some versions put trend statistics directly on the metric (metrics.http_req_duration['p(95)']), others nest them under values. The helper below accepts both, which is a good habit when you read machine output.

Node.js gate.mjs: compare a summary with a baseline and budgets

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

In CI the script reads the two JSON files (fs.readFileSync), prints the table (which you can post as a pull request comment or write to the job summary) and calls process.exit(1) on regressions. The p(95) change of +20 % in the example exceeds the +15 % budget, so the gate blocks the merge.

Step 3: the workflow

YAML
name: perf-gate
on: pull_request

jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 24 }
      - uses: grafana/setup-k6-action@v1

      - name: Run smoke test
        run: k6 run --summary-export=summary.json tests/smoke.js
        env:
          BASE_URL: ${{ vars.PREVIEW_URL }}

      - name: Compare with baseline
        if: always()
        run: node tests/gate.mjs summary.json tests/baseline.json | tee gate.md >> "$GITHUB_STEP_SUMMARY"

      - name: Comment on the PR
        if: always() && github.event.pull_request
        uses: marocchino/sticky-pull-request-comment@v2
        with:
          path: gate.md

$GITHUB_STEP_SUMMARY shows the table on the workflow run page; the sticky-comment step keeps one updated comment on the pull request instead of one per push. The if: always() lines make sure you still get the table when the k6 step itself fails its thresholds.

Step 4: managing the baseline

A baseline that never changes becomes wrong; one that changes after every run hides slow drift. Common policy:

  • Refresh it only from the main branch, nightly, after the build passed.
  • Use a rolling median of the last N runs, which ignores single noisy runs.
  • Store it somewhere versioned (a file in a perf branch or artifact storage).
  • Never update it from a pull request's own run.
Node.js Rolling-median baseline resists one noisy run

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Handling noise

  • Run the smoke test twice and compare the better (or the median) of both before failing, or re-run automatically on failure once.
  • Use a dedicated or isolated preview environment, not a shared staging that others are deploying to.
  • Compare ratios, not absolute milliseconds, between runs on possibly different CI hardware.
  • Keep the generator small and the test short: a gate that takes 20 minutes will be bypassed.
  • Review budgets quarterly: too tight causes flaky blocks, too loose lets real regressions through.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why combine absolute thresholds with a comparison against a baseline?

    A change can stay under an absolute limit while being 30 percent slower than before; the relative budget catches that.
  2. 2. Why does gate.mjs read values from either m.values or m directly?

    Summary layouts changed over time; being tolerant keeps the gate working after an upgrade.
  3. 3. Which baseline policy is recommended?

    Main-branch only avoids contaminating the baseline with unreviewed code; the median tolerates single noisy runs.
  4. 4. What does if: always() on the comparison step ensure?

    Without it, a failed k6 step would skip the comparison and you would lose the explanatory table.

Summary

  • A performance gate pairs k6 thresholds with a relative comparison against a baseline.
  • Export the summary, compare with budgets, print a Markdown table and set the exit code.
  • Refresh the baseline from the main branch using a rolling median.
  • Reduce flakiness with isolated environments, short tests and sensible budgets.

Next: Practice: k6 Exercises