Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 36 ยท Rendering Modes and Functions

Build Caching and Incremental Builds

Understand the .cache and public folders, why builds slow down as sites grow, how to reuse caches in CI, and ways to cut Gatsby build times.

Static generation has one famous weakness: the build. A site with 50 pages builds in seconds; one with 50,000 pages and thousands of images may take an hour. Because the whole site is rebuilt after every content change, build time becomes the main cost of Gatsby at scale. This lesson explains what Gatsby caches, what it does not, and how to keep builds fast.

What happens during gatsby build

flowchart LR
  A["Bootstrap: plugins, config"] --> B["Source nodes"]
  B --> C["Infer schema and run transformers"]
  C --> D["createPages"]
  D --> E["Run page queries"]
  E --> F["Bundle JS with webpack"]
  F --> G["Render HTML for each page"]
  G --> H["Process images, write public/"]

Each stage can be skipped or sped up if its inputs did not change.

The two important folders are:

  • .cache: Gatsby's working directory. It holds the data store (all nodes), compiled query results, webpack caches and processed image variants. Safe to delete; the next build recreates it, slowly.
  • public: the final output uploaded to the host. Also safe to delete, and regenerated.

Neither belongs in Git (the starters add both to .gitignore).

Why reuse matters

On your laptop, the second gatsby build is faster because .cache still holds the work from the first. On a CI server that starts clean every time, nothing is reused unless you save the folders between runs. Most hosts (Netlify, Vercel, GitHub Actions) offer a cache for .cache and public. With them, only changed Markdown files re-run transformers and only affected images are reprocessed.

YAML
# GitHub Actions snippet (read-only)
- uses: actions/cache@v4
  with:
    path: |
      .cache
      public
    key: gatsby-${{ runner.os }}-${{ hashFiles('package-lock.json') }}-${{ github.sha }}
    restore-keys: |
      gatsby-${{ runner.os }}-${{ hashFiles('package-lock.json') }}-

The restore-keys entry means "use the newest cache for this lockfile, even if the commit differs", which is exactly what you want.

Gatsby invalidates the cache itself when gatsby-config.js, gatsby-node.js or plugin versions change, so a stale cache cannot silently produce wrong output. If something looks wrong anyway, run gatsby clean and rebuild.

Incremental builds: the idea

An incremental build rebuilds only the pages whose data changed. Gatsby tracks which nodes each page query used. If only one post changed, only that post's page and the pages that list it (the blog index, its tag pages) need new HTML. Gatsby's incremental-build support has historically depended on the hosting platform's build infrastructure (for instance the service that used to be Gatsby Cloud, whose capabilities moved to Netlify), so check what your host supports today. On a plain CI runner, the practical equivalent is: restore .cache and public, then run a normal gatsby build; unchanged work is skipped.

The core mechanism is change detection by content hash. This stand-in shows it.

JS Stand-in: rebuild only what changed

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The index page depends on every post, so it is always rebuilt. That is why pagination and "latest posts" lists limit how much incremental builds can save.

Reducing build time

Work through these in order, measuring after each change with gatsby build --verbose or the timings printed at the end of the build:

  1. Reuse caches in CI (above): usually the biggest win.
  2. Shrink images: resize originals, avoid AVIF for thousands of images, and use placeholder: DOMINANT_COLOR instead of TRACED_SVG.
  3. Defer old pages with DSG (see rendering modes).
  4. Cut GraphQL cost: avoid querying html or excerpt in index queries; query only fields you use.
  5. Raise the Node memory limit when the build dies with an out-of-memory error: NODE_OPTIONS=--max-old-space-size=8192 gatsby build.
  6. Tune parallelism: GATSBY_CPU_COUNT=logical_cores allows more workers for images and rendering.
  7. Remove unused plugins, because every plugin adds bootstrap time.
JS Stand-in: estimate image build time

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The numbers are invented for illustration, but the shape is real: images times variants times encoding time, divided by workers. AVIF encodes several times slower than WebP, which is why it can dominate a build.

Common mistakes

  • Committing .cache or public to Git.
  • Not saving caches in CI, then wondering why builds are slow.
  • Using gatsby clean on every build "to be safe", which throws away the cache.
  • Expecting an incremental build when a layout file or gatsby-node.js changed: that invalidates the cache.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is stored in the .cache folder?

    It is Gatsby's working directory; it speeds up later builds.
  2. 2. What should a CI pipeline do to make Gatsby builds faster?

    Restoring the caches lets Gatsby skip unchanged work.
  3. 3. Why does the blog index page rebuild whenever any post changes?

    Page queries track the nodes they read, and the index reads all posts.
  4. 4. Which is usually the slowest format to encode for a large image set?

    AVIF encoding is much slower, so it can dominate build time.

Summary

  • .cache holds Gatsby's working data; public is the output. Neither goes into Git.
  • Save both between CI runs so only changed content is reprocessed.
  • Incremental building relies on content hashes and tracked dependencies; support depends on your host.
  • Cut time with smaller images, fewer formats, DSG for the long tail, leaner queries and more memory or workers.

Next: Deploying Gatsby sites