Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 36 ยท Monorepos

Filtering and Task Orchestration

Run scripts in the right workspaces and the right order. Learn npm -w, pnpm --filter, Yarn foreach, affected builds, task pipelines and caching with Turborepo and Nx.

A monorepo with thirty packages raises two practical questions every day. Which packages should this command run in? (Just the one I changed? Everything that depends on it?) And in what order? (A library must be built before the apps that import it.) Then a third, once the repo grows: can we skip work we already did?

Package managers answer the first two with selectors and a topological order. Dedicated task runners (Turborepo, Nx) add caching. This lesson covers all three. Commands are read-only snippets; the runnable examples implement selectors, affected detection and a task cache in plain Node.js.

Selecting workspaces

npm

Terminal
npm run test --workspaces              # all workspaces (alias -ws)
npm run test --workspace=packages/ui   # one workspace (alias -w), by path or name
npm run test -w @acme/ui -w @acme/api  # several
npm run test --workspaces --if-present # skip workspaces without a "test" script
npm run build --include-workspace-root # also run in the root

npm runs workspaces in the order they are listed in workspaces, not in dependency order, so it is the weakest choice for builds.

pnpm

pnpm has the richest selector syntax:

Terminal
pnpm -r run build                       # recursive, in topological order
pnpm --filter @acme/web run dev         # one package
pnpm --filter "./packages/*" run test   # by folder glob
pnpm --filter @acme/ui... run build     # @acme/ui AND everything it depends on
pnpm --filter ...@acme/ui run test      # @acme/ui AND everything that depends on it
pnpm --filter "...[origin/main]" run test  # packages changed since main, plus their dependents
pnpm --filter "!@acme/docs" -r run build   # everything except docs

The trailing ... means "include dependencies" and the leading ... means "include dependents". The bracket form selects by git change.

Yarn Berry

Terminal
yarn workspaces foreach --all run build               # every workspace
yarn workspaces foreach --all --topological run build  # dependencies first
yarn workspaces foreach --since --recursive run test   # changed workspaces (git)
yarn workspace @acme/web run dev

Add --parallel to run independent workspaces at the same time and --jobs 4 to cap concurrency.

Implementing selectors yourself

Understanding "dependencies" versus "dependents" makes the pnpm syntax obvious. This stand-in builds both directions from one graph and selects with the same notation.

Node.js Stand-in: dependency and dependent selectors

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Affected builds: only what changed

On CI you rarely need to test everything. If a pull request touches only packages/ui, you need to test ui and everything that depends on it (web and docs), but not api. This is "affected" detection: take the changed files, map them to workspaces, then add dependents.

Affected only

Changed files git diff vs main
Owning packages
Their dependents
Run just those

Fewer jobs, faster pull requests.

1 / 2
Node.js Stand-in: compute affected packages from changed files

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Task runners: pipelines and caching

Selectors tell the manager where to run. A task runner understands what depends on what between tasks and remembers results. Turborepo and Nx are the two popular ones; both sit on top of workspaces and work with npm, Yarn or pnpm. A Turborepo pipeline looks like this (read-only):

JSON
{
  "tasks": {
    "build": {
      "dependsOn": ["^build"],
      "inputs": ["src/**", "package.json", "tsconfig.json"],
      "outputs": ["dist/**"]
    },
    "test": { "dependsOn": ["build"] },
    "lint": {}
  }
}
  • "^build" means "the build of my dependencies first". Without the caret it means another task in the same package.
  • inputs define what the task reads; outputs define what it produces.
  • The runner hashes the inputs (plus the dependencies' hashes and some environment). If the hash was seen before, it restores the outputs from the cache and replays the logs instead of running the task. A remote cache shares this across teammates and CI.

Tasks with no dependency between them run in parallel, so the whole graph runs as wide as your CPU allows.

This stand-in shows the heart of it: a hash of inputs deciding whether to run.

Node.js Stand-in: a tiny content-hash task cache

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Notice that changing utils invalidated ui and web even though their own sources were untouched: their hashes include their dependencies' hashes.

Which tool when?

Need Reach for
A handful of packages, simple scripts Your package manager's own -w / --filter / foreach
Topological order and filters pnpm -r and --filter, or Yarn foreach --topological
Caching and CI speed-ups on a growing repo Turborepo (simple) or Nx (full-featured, generators, graph UI)
Version bumps and changelogs Changesets (see the monorepo release project)

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does pnpm --filter ...@acme/ui run test select?

    A leading ... includes dependents; a trailing ... includes dependencies.
  2. 2. Why is a lockfile change usually treated as affecting every package?

    The lockfile covers the whole tree, so a change can alter any package's resolved dependencies.
  3. 3. In a task runner pipeline, what does "dependsOn": ["^build"] mean?

    The caret refers to upstream workspace dependencies, ensuring libraries build before their consumers.
  4. 4. Why did changing utils also invalidate the cache for ui and web?

    Downstream hashes combine upstream hashes, so any upstream change cascades.

Summary

  • npm uses -w/--workspaces; pnpm has -r and rich --filter selectors; Yarn uses workspaces foreach.
  • pkg... selects dependencies, ...pkg selects dependents; only pnpm and Yarn run in topological order by default or by flag.
  • Affected detection maps changed files to packages and adds dependents; treat lockfile and root config changes as global.
  • Task runners add task pipelines (^build), parallelism and content-hash caching.
  • Cache keys include dependency hashes so changes cascade downstream.

Next: node_modules Layouts, Hoisting and the pnpm Store