Skip to content
Tutorials Center
My learning

Topics

Lesson 15 of 36 ยท Source and Transformer Plugins

Source Plugins and gatsby-source-filesystem

Bring data into Gatsby with source plugins: configure gatsby-source-filesystem, query File nodes, source several folders, and meet CMS and API source plugins.

Gatsby does not know anything about your data until a source plugin tells it. A source plugin reads some external place (a folder of files, a headless CMS, a REST API, a database) and creates nodes in Gatsby's data layer. After that, your queries can reach the data like any other. This lesson explains the idea, then focuses on the most common source plugin, gatsby-source-filesystem, which turns files on disk into File nodes.

Gatsby configuration is read-only here; stand-ins model what the plugin produces.

What a source plugin does

flowchart LR
  A["Folder of files"] --> S1["gatsby-source-filesystem"]
  B["Headless CMS"] --> S2["CMS source plugin"]
  C["REST or GraphQL API"] --> S3["custom or API plugin"]
  S1 --> N["Nodes in the data layer"]
  S2 --> N
  S3 --> N
  N --> Q["GraphQL queries"]

Different origins, one data layer.

Source plugins implement the sourceNodes API from gatsby-config and the Node APIs. Popular ones: gatsby-source-contentful, gatsby-source-sanity, gatsby-source-wordpress, gatsby-source-strapi, gatsby-source-mongodb and many community packages. When none fits, you can write your own with sourceNodes.

Installing and configuring gatsby-source-filesystem

Terminal
npm install gatsby-source-filesystem
JavaScript
// gatsby-config.js (real Gatsby, read-only)
module.exports = {
  plugins: [
    {
      resolve: "gatsby-source-filesystem",
      options: {
        name: "posts",
        path: `${__dirname}/content/posts`,
      },
    },
    {
      resolve: "gatsby-source-filesystem",
      options: {
        name: "images",
        path: `${__dirname}/src/images`,
      },
    },
  ],
}
  • name labels the source. It appears on each node as sourceInstanceName, so you can tell apart files from different folders.
  • path is the folder to read. Use an absolute path, built with __dirname, not a relative one.
  • You can add the plugin many times, once per folder.
  • Options ignore (glob patterns) and fastHash exist for large or special folders.

In development, the plugin watches the folder: add or edit a file and Gatsby updates nodes and the browser.

The File node

Each file becomes a node of type File. Query them in GraphiQL (lesson):

GRAPHQL
query {
  allFile(filter: { sourceInstanceName: { eq: "posts" } }) {
    nodes {
      name
      base
      ext
      relativePath
      absolutePath
      size
      modifiedTime(formatString: "YYYY-MM-DD")
      sourceInstanceName
    }
  }
}
Field Example for content/posts/espresso.md
name espresso
base espresso.md
ext .md
relativePath espresso.md (relative to the plugin path)
absolutePath the full path on disk
sourceInstanceName posts

File nodes alone carry no file content. Transformer plugins read them and create content nodes such as MarkdownRemark or ImagesSharp as children of the file node.

Common patterns

  • One folder per content type: content/posts, content/docs, src/images. Filter by sourceInstanceName to separate them.
  • Singular query: file(relativePath: { eq: "logo.png" }) finds a specific file.
  • Downloading remote files: the plugin also exports createRemoteFileNode, used in onCreateNode or sourceNodes to download an image URL into a local File node so image processing can run on it.

Stand-in: files become File nodes

This stand-in is not the plugin. It takes a list of virtual file paths and creates File-like nodes with the same derived fields, then filters by source name.

JS Stand-in: building File nodes

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Stand-in: a watcher that updates nodes

In development the filesystem plugin reacts to changes. This stand-in compares two snapshots of a folder and reports which nodes were created, updated and deleted, which is the work a watcher triggers.

JS Stand-in: detect created, changed, deleted files

Press Run (or Ctrl + Enter) to see the console output here.

No output.

If allFile returns nothing, check the path option first: it is resolved on your machine. A relative path that works from one directory fails from another. Also remember that Gatsby does not watch folders outside the path you configured.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a source plugin create?

    Source plugins create nodes, which later queries and plugins use.
  2. 2. Which option labels a gatsby-source-filesystem instance?

    name becomes the sourceInstanceName field on each File node.
  3. 3. How should the path option be written?

    An absolute path avoids problems with the working directory.
  4. 4. What does a File node contain by itself?

    A File node describes the file. Transformer plugins add content nodes.

Summary

  • Source plugins pull external data into Gatsby as nodes through the sourceNodes API.
  • gatsby-source-filesystem turns every file in a folder into a File node and watches it in development.
  • Use name to distinguish sources and an absolute path.
  • Query allFile with sourceInstanceName filters; check it first when data is missing.
  • Transformers turn File nodes into usable content nodes.

Next: Transformer Plugins