Updating Data in Gatsby Source Plugin

Question

I have created a simple gatsby source plugin that pulls in api data. So far, so good -- it pulls in the data exactly as it should.

However, there is one problem. The api data gets updated on a regular basis. That is to say, new articles get added to the api. I am wondering if it is possible to update the graphql data in Gatsby with the new articles without losing the data that was already fetched.

If so, how?

Thanks.

P.S. In case it is relevant, here is the code of the plugin:

const fetch = require("node-fetch")

exports.sourceNodes = (
  { actions, createNodeId, createContentDigest },
  config
) => {
  const { createNode } = actions

  delete config.plugins

  const processFeed = item => {
    const nodeId = createNodeId(`[NODE_NAME]-${item.id}`)
    const nodeContent = JSON.stringify(item)
    const nodeData = Object.assign({}, item, {
      id: nodeId,
      parent: null,
      children: [],
      internal: {
        type: `[TYPE_NAME]`,
        content: nodeContent,
        contentDigest: createContentDigest(item),
      },
    })
    return nodeData
  }

  ...

  return fetch([FETCH_URL], [FETCH_HEADERS])
    .then(response => response.json())
    .then(data => {
      data.items.forEach(item => {
        const nodeData = processFeed(item)
        createNode(nodeData)
      })
    })
}

Answer 1

Since the GraphQL server in Gatsby only runs during development (or while the build is being created), there's no way for a source plugin to push data on a regular basis out of the box. What you could do instead is trigger a new build on a recurring schedule (every N minutes/hours/etc) to fetch updated data.

If your builds take a while to complete and you want to speed up this process, you could also consider using Gatsby Cloud which has support for doing incremental builds (note: Gatsby Cloud is a paid SaaS product from the Gatsby core team).