Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 35 ยท CI and Scale

Parallel Runs, Sharding and Speed

Make browser test suites finish sooner: Playwright workers, fullyParallel and --shard, Cypress parallelization, balancing by duration and cutting test time.

A suite of 30 end-to-end tests that takes 5 minutes is fine. A suite of 600 tests that takes two hours is not: developers stop waiting for it, merge without it, or re-run it until it passes. Speed is a feature of a test suite. This lesson covers the two ways to shorten the wall-clock time, running tests in parallel and splitting the suite across machines (sharding), then the cheaper alternative: making each test faster.

Real parallel runs need browsers and multiple processes, so the Playwright and Cypress code below is read-only. The runnable examples are stand-ins in plain Node.js: a scheduler that balances tests across workers, and a comparison of naive and duration-aware splitting. They model the arithmetic of scheduling, not real browser workers.

Two levels of parallelism

flowchart TD
  S["Test suite: 120 tests"] --> M1["Machine 1 (shard 1/3)"]
  S --> M2["Machine 2 (shard 2/3)"]
  S --> M3["Machine 3 (shard 3/3)"]
  M1 --> W1["Worker A"]
  M1 --> W2["Worker B"]
  M2 --> W3["Worker A"]
  M2 --> W4["Worker B"]
  M3 --> W5["Worker A"]
  M3 --> W6["Worker B"]

Sharding splits the suite across machines; workers run tests in parallel inside one machine.

Workers are processes on one machine. They are limited by CPU cores and memory, because every worker runs its own browser. Shards are separate machines (or CI jobs), each running a slice of the suite. Together they multiply: 3 shards with 2 workers each run 6 tests at the same moment.

Playwright: workers and fullyParallel

Playwright runs test files in parallel by default, using several worker processes. Within one file, tests run in order in the same worker unless you change that. You control it in the config:

TypeScript
// playwright.config.ts (read-only)
import { defineConfig } from '@playwright/test';

export default defineConfig({
  fullyParallel: true,                       // also run tests inside one file in parallel
  workers: process.env.CI ? 2 : undefined,   // undefined = a default based on CPU cores
});

You can override it on the command line:

Terminal
# read-only
npx playwright test --workers=4
npx playwright test --workers=1        # sequential, useful for debugging

Without fullyParallel, one huge file with 50 tests runs on a single worker while the others sit idle. Turning it on spreads those tests out. The price is that tests must be independent: no test may rely on data created by another, and no two tests may fight over the same record, such as two tests renaming the same user. Isolation tools from earlier lessons (a fresh browser context for every test, API-created data per test, see Test Data and Logging In Fast) are what make this safe. A group of tests that truly must run in order can be marked with test.describe.configure({ mode: 'serial' }), but that is a last resort, because one failure then skips the rest of the group.

Each worker gets an index, available as testInfo.parallelIndex (and workerIndex), which is handy for giving every worker its own test account or database schema.

Playwright: sharding

When one machine is not enough, split the run:

Terminal
# read-only: each command runs on a different CI machine
npx playwright test --shard=1/3
npx playwright test --shard=2/3
npx playwright test --shard=3/3

In GitHub Actions a matrix starts the jobs for you:

YAML
# read-only
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: lts/*, cache: npm }
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test --shard=${{ matrix.shard }}/3 --reporter=blob
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: blob-report-${{ matrix.shard }}
          path: blob-report

Each shard writes a blob report. A final job downloads all of them and combines them into one HTML report with npx playwright merge-reports, so you read a single report instead of three. fail-fast: false lets the other shards finish when one fails, so you see every failing test in one run. The pipeline basics are in Running Tests in CI.

How does Playwright divide the tests between shards? It splits the list of tests (in file order, with fullyParallel) or of files into roughly equal-sized chunks. That balances the number of tests, not their duration. If all the slow tests land in shard 2, shard 2 finishes last and the pipeline takes as long as that one shard. Keep files and tests of similar size, or split big files, to keep the shards even.

Cypress: parallelization

Cypress parallelizes at the spec file level, and its built-in way depends on Cypress Cloud, the paid-tier hosted service (it has a free tier with limits):

Terminal
# read-only: the same command runs on several CI machines
npx cypress run --record --parallel --ci-build-id $GITHUB_RUN_ID

The machines contact the Cloud service, which hands each one the next spec to run. Because it remembers how long each spec took in earlier runs, it balances by historical duration: the slowest specs start first and every machine stays busy. Machines that start late or finish early are handled automatically. You need a project key (stored as a secret) and a matching --ci-build-id on all machines so they know they belong to one run.

Without Cypress Cloud, the usual alternative is to split the spec list yourself: a community plugin or a small script divides the specs into groups, one per CI job. Those splits are static, so they do not adapt to changing durations unless you feed the script timing data. Whichever option you choose, check the current documentation, because pricing and plugins change.

Balancing by duration

Why does the order matter? Imagine four machines and these test durations (in seconds): 90, 80, 30, 20, 20, 10, 10, 10. Handing tests out in order and splitting the list in equal chunks gives one machine the two longest tests (170 s) and another the tail (30 s). Handing out the longest remaining test to the least busy machine gives a much tighter result. This simple method is called the longest processing time first rule, and it is the same idea Cypress Cloud's duration-based balancing relies on.

The stand-in below simulates workers picking tests from a shared queue, as Playwright workers do. It prints a timeline per worker so you can see idle time.

Node.js Workers pulling tests from a queue

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Notice that going from 2 to 4 workers does not halve the time again. The suite can never finish faster than its slowest single test (checkout, 90 s here). That is the hard limit of parallelism, and why splitting very long tests into smaller ones helps.

Shard splitting: naive versus duration-aware

Now the multi-machine view. This stand-in splits the same tests into shards two ways. The naive split deals tests in file order into equal-sized chunks, as a count-based splitter does. The smart split sorts by duration and always assigns the next test to the lightest shard.

Node.js Naive shards versus duration-aware shards

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The smart split cannot beat the longest single test either, but it removes the waste: no shard sits idle while another is overloaded. Cypress Cloud does this with real timing history; with Playwright you can approximate it by keeping files similar in size, or by feeding a timing report into a custom splitting script.

Making the suite faster without more machines

Adding machines costs money and only hides slowness. Often the biggest gains come from the tests themselves:

  • Skip the UI for setup. Log in with a saved session or an API call instead of typing into the login form in every test.
  • Create data through the API, not by clicking through forms.
  • Replace fixed sleeps with waits for real conditions (see Auto-Waiting in Playwright). A waitForTimeout(3000) costs three seconds every time.
  • Stub slow third-party calls (maps, analytics, chat widgets) with network mocking.
  • Test at the right level. A form validation rule belongs in a unit or component test, not in a full e2e journey.
  • Reuse the app build and avoid rebuilding between shards.
  • Retry sparingly. Retries hide flakiness and add time. Fix the cause instead.

Measure before you optimize. Both tools can print the slowest tests (--reporter=list shows durations in Playwright; Cypress Cloud and most reporters list them). Start with the top five.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is the difference between a worker and a shard?

    Workers run tests in parallel inside one machine. Shards divide the suite between machines or CI jobs, and each shard can use several workers.
  2. 2. Why is fullyParallel: true only safe when tests are independent?

    With fullyParallel, tests from one file are spread over workers. A test that relies on state created by an earlier test would fail or interfere.
  3. 3. How does Cypress Cloud decide how to balance specs between machines?

    It records spec durations and hands the next spec to a free machine, starting the slowest first.
  4. 4. Four workers run a suite whose longest test takes 90 seconds. What is the shortest the suite can possibly take?

    A test runs on a single worker from start to end, so the suite can never finish faster than its slowest test. Splitting long tests helps.
  5. 5. Which change usually speeds up a suite more than adding machines?

    Removing repeated slow setup shortens every test. Sleeps and retries make the suite slower.

Summary

  • Workers parallelize inside a machine; shards split the suite across machines. Both require independent tests.
  • Playwright: fullyParallel, workers, --shard=n/total, blob reporter plus merge-reports.
  • Cypress: --record --parallel --ci-build-id with Cypress Cloud balances specs by past duration; without it, split specs yourself.
  • Balance shards by duration, not by test count. The slowest single test is a hard lower bound.
  • Faster tests beat more machines: fast logins, API setup, no fixed sleeps, stubbed slow calls, the right test level.

Next: Project: Test a Login Flow