Skip to content
Tutorials Center
My learning

Topics

Lesson 17 of 36 ยท Source and Transformer Plugins

Creating Nodes with sourceNodes

Write your own Gatsby source with the sourceNodes API: createNode, createNodeId, contentDigest, caching, touching nodes, relations between nodes and fetching a REST API.

No plugin exists for your company's internal API, or for a spreadsheet, or for a legacy database. The answer in Gatsby is simple: write the source yourself in gatsby-node.js. The sourceNodes API gives you the tools to fetch data and turn each record into a node. After that, your data behaves like data from any official plugin: queryable in GraphQL, usable by createPages, and filterable in GraphiQL.

All Gatsby code is read-only; a runnable node store and a fake API stand-in let you test the logic.

The shape of a node

A node is a plain object with a few required properties:

JavaScript
{
  id: createNodeId(`Product-${product.sku}`),  // unique, stable
  parent: null,                                // null for top-level nodes
  children: [],
  internal: {
    type: "Product",                           // becomes the GraphQL type
    contentDigest: createContentDigest(product), // hash of the data
  },
  // your own fields follow:
  name: "Espresso Beans",
  price: 12,
}
  • id must be unique across the whole site. createNodeId turns a string into a stable ID so it stays the same between builds.
  • internal.type is the type name. Product makes product and allProduct root fields.
  • internal.contentDigest is a hash of the node's content. Gatsby compares it to the previous build to know whether a node changed, which is how caching works. createContentDigest computes it.
  • Your fields must not clash with id, parent, children and internal. If an API returns its own id, put it under another name such as externalId.

A complete sourceNodes example

JavaScript
// gatsby-node.js (real Gatsby, read-only)
exports.sourceNodes = async ({ actions, createNodeId, createContentDigest, reporter }) => {
  const { createNode } = actions

  const response = await fetch("https://api.example.com/products")
  if (!response.ok) {
    reporter.panicOnBuild(`Products API failed: ${response.status}`)
    return
  }
  const products = await response.json()

  for (const product of products) {
    const { id, ...fields } = product
    createNode({
      ...fields,
      externalId: id,
      id: createNodeId(`Product-${id}`),
      parent: null,
      children: [],
      internal: {
        type: "Product",
        contentDigest: createContentDigest(product),
      },
    })
  }
}

fetch is built into Node 18 and newer. The helpers createNodeId, createContentDigest, reporter, cache and getNode are passed to every Node API. Use reporter.info, warn and panicOnBuild instead of console.log so messages show up properly in the build output and failed builds stop with a clear error.

After running gatsby develop, query it:

GRAPHQL
query { allProduct { nodes { name price externalId } } }

Relationships between nodes

GraphQL can follow links from one node to another. Store the other node's ID in a field ending in ___NODE (old style) or declare a @link in the schema (preferred, see schema customization):

JavaScript
createNode({
  id: createNodeId(`Review-${review.id}`),
  product: createNodeId(`Product-${review.productId}`), // link by ID
  text: review.text,
  internal: { type: "Review", contentDigest: createContentDigest(review) },
})
GRAPHQL
# with type defs: product: Product @link
query { allReview { nodes { text product { name } } } }

Caching and staleness

On rebuilds Gatsby keeps nodes from the previous build. Two techniques matter:

  • contentDigest lets Gatsby skip unchanged nodes without reprocessing them.
  • touchNode tells Gatsby "this node is still valid" when you did not recreate it, so it is not garbage-collected. Plugins that cache fetched data across builds call actions.touchNode({ nodeId }).

You can also store a timestamp in cache.set("lastFetch", Date.now()) and ask an API only for records changed since then (incremental sync). That is how large CMS plugins stay fast.

Local plugins

If your source grows, move it from gatsby-node.js into a local plugin: a folder plugins/my-source/ containing gatsby-node.js and a package.json. Add "my-source" to plugins in gatsby-config.js. This lets you reuse it, pass options and test it in isolation.

Stand-in: a node store with contentDigest

This stand-in is not Gatsby. It implements createNode with the required fields, a deterministic createNodeId, a digest to detect changes, and counts how many nodes are really created or updated between two "builds".

JS Stand-in: createNode and change detection

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Stand-in: a fake API and a relationship

JS Stand-in: fetch, create nodes, follow a link

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Never put API secrets in node data. Everything you put in a node can be queried and may land in page data. Read secrets from process.env inside sourceNodes and keep them out of the nodes you create.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which fields are required on a node you create?

    id and internal.type are required; contentDigest lets Gatsby detect changes and is expected by caching.
  2. 2. What does contentDigest enable?

    Gatsby compares digests to know whether a node changed.
  3. 3. What should you do if the API record has its own id field?

    Node id must be unique across all types. Generate it with createNodeId and keep the original separately.
  4. 4. Where should secrets for an API live?

    Environment variables stay on the build machine, unlike node data.

Summary

  • sourceNodes fetches data and calls createNode once per record.
  • A node needs a unique id, an internal.type and an internal.contentDigest.
  • createNodeId and createContentDigest give stable IDs and change detection.
  • Link nodes by storing the other node's ID and declaring @link in the schema.
  • Use touchNode, cache and local plugins as sources grow; keep secrets out of nodes.

Next: Schema Customization