Fetch Data from the Unsplash API and Render Using DSG

Hello again 👋.

In a previous post I discussed how to fetch and store data sourced from a remote API. In this post I’ll discuss how to turn this data into individual pages for your site.

If you’d prefer to jump ahead you can find the demo site and src code on the links below.

Demo: https://datachampiondsgcpnycphotos.gatsbyjs.io

Repo: https://github.com/PaulieScanlon/data-champion-dsg-create-pages-nyc-photos

Let’s talk about pages

In Gatsby, data that is stored in the data layer, and pages are two different things. The reason for this is because there’s multiple ways to query this data. You can either use it and re-use around your site using useStaticQuery or you can use it to create a page.

There’s also two methods available you can use to create pages with Gatsby. In this post I’ll be discussing how to use createPages.

And lastly, using Gatsby’s latest page rendering method DSG (Deferred Static Generation) I’ll be showing you how to defer the creation of your pages to keep your build times super fast! ⚡

In this post i’ll discuss the following methods:

• How to source data from the Unsplash API.
• How to add data to Gatsby’s data layer. 
• How to create pages using createPages.
• How to defer the creation of pages. 
• How to create a page template.

Unsplash API

To use the Unsplash API head on over to https://unsplash.com/developers and sign up.

Once you have an account you can create a new application.

You’ll then have access to your applications API keys.

Environment Variables

To ensure your API key is safe and secure you’ll need to store it as an environment variable. With Gatsby you can do this by creating a .env.development and .env.production at the root of your project. You can read more about Environment Variables in the Gatsby docs.

In my demo you’ll see the .env.example file which contains the following.

unsplash-js

In order to make HTTP request to the Unsplash API you can install and use unsplash-js, there’s more information on how to install and use this in the GitHub README.

With your environment variable setup and unsplash-js installed you can now make HTTP requests to the Unsplash API from gatsby-node.js.

Fetch Data from the Unsplash API and create nodes

There’s a few things going on in here so i’ll talk you through each step.

require dotenv

This allows Gatsby access to your environment variables. 

You could also add this to gatsby-config.js FYI ☝️.

node-fetch

As of Gatsby 4, Gatsby already uses node-fetch under the hood, you won’t need to install this npm package as it’s already part of Gatsby. To use it though you will need to require it in the gatsby-node.js file.

slugify

As above, Gatsby already uses @sindresorhus/slugify under the hood, you won’t need to install this npm package but again you will need to require it, to use it. 

createApi

This is part of the unsplash-js package and can be used to make authenticated HTTP requests to the Unsplash API using fetch (node-fetch).

sourceNodes

This is part of the Gatsby build step and is called during Gatsby’s bootstrap sequence. Within this function you can source data from absolutely anywhere using standard JavaScript HTTP methods.

unsplash.search

This is a specific method from the docs where you can define details of photographs you’d like to search for. In this case i’m searching for photos tagged with “nyc”, are “black and white” and are in “landscape” orientation.

If you pop in a console.log(data) and run gatsby build you should see something similar to the below.

You should be able to see in the response there’s an array of objects in response.results.

These objects contain information about each photograph returned by the Unsplash API and using a forEach you can iterate over each one and add it to Gatsby’s data layer using createNode.

Adding Data to Gatsby’s Data Layer

createNode is a special action that is part of Gatsby and can be used to add the data returned for each photograph to Gatsby’s data layer. There are a number of required arguments, they are as follows:

• ...item: Is all the photograph data returned from the Unsplash API.
• id: Is unique and is used by Gatsby to identify each node. I’ve used; item.id which is returned by the Unsplash API since it is already unique.  
• slug: Is a hyphenated value of blur_hash – any unique value will do here, and you’ll use this later to create the page path.
• internal.type: Is the name of the node(s) you’ll query shortly using GraphQL.
• internal.contentDigest: Is used as a kind of indicator to Gatsby that is used to determine if data is fresh and doesn’t exist in the data layer or has changed, in which case the data layer will be updated.

… and that’s it, you have now added all of the objects returned by the Unsplash API to Gatsby’s data layer and given them a name of NycPhoto. Now it’s time to turn this data into pages!

Creating Pages from Data

There’s a little more going on in this bit so i’ll walk you through it.

createPages

This is another one of Gatsby’s extensions points and is only called after the initial data sourcing, node and GraphQL schema steps have completed. 

graphql

As with all of Gatsby’s data querying methods GraphQL is used to return data about a certain type of node. In this case you’ll be querying allNycPhoto. 

nycPhoto is the name of the node(s) you created earlier and Gatsby prefixes “all” which means you’ll be able to query all nodes of this type.

The key parts i’d like to draw your attention to are as below:

• node: is the actual data object you created earlier and you’ll need access to id, slug and likes in the next step.
• next: contains the slug for the next “page” in the data set.
• prev: contains the slug for the previous “page” in the data set.

edges.forEach

Using a forEach you can iterate over all the nodes returned by the GraphQL query and create a page for each slug. 

createPage

Not to be confused with createPages, createPage is a Gatsby action that allows you to create a page from data returned by GraphQL.

• path: Is the slug you created earlier and will eventually be what you’ll see in the browser’s address bar.
• component: Is a page template / React component that will return the page query data.
• context: 
  • id: Will be used later by the page template to find the relevant node field in Gatsby’s data layer.
  • prev: will either pass previous or null depending on, if there’s a previous page or not.
  • next: will either pass next or null depending on, if there’s a next page or not.
  • defer: This is the important part and i’ll go into a little more detail about defer in the next section.

Deferring Page Creation

Let’s start with the problem. Gatsby can and will attempt to build all of your sites pages ahead of time. 

SSG (Static Site Generation) is far and away the fastest, most secure and most SEO optimized way to build a website, but it can come at a cost. 

Asking your local machine or the CDN to build all of your sites pages ahead of time can be quite a big job and can take some time – so don’t 😃.

With DSG (Deferred Static Generation) Gatsby is handing control over to you the developer. 

DSG works by observing what I refer to as defer strategies, if a page meets the requirements it will be built ahead of time, if it doesn’t, Gatsby will defer the static generation of the page until a user visits it. 

When this happens the page is server side rendered in much the same way as it would be if you were to use SSR but Gatsby Cloud for instance will then cache this page, meaning the next time a user visits that page it will behave exactly as it would have done had it been prepared ahead of time with SSG. 

In my example my defer strategy is based on “likes”, E.g

If likes is less than 100 defer statically generating the page, if likes is more than 100 go ahead and build it. 

But why likes? 

It’s my assumption that if a photo has received a lot of likes then it’s likely it’s a page that users will have visited regularly and if so, I’d like to give users the best possible experience and prepare the page ahead of time (SSG). If it’s an unpopular page I’ll defer the static generation of this page until someone visits it. (DSG).

There’s so many ways you could define a defer strategy, and like with defer itself Gatsby is handing control of this over to you.

Creating A Page Template

As mentioned above one of the arguments for createPage is component. This is a React component that will query Gatsby’s data layer by it’s id found in context and pass the returned data back to the page via the data prop.

At a minimum your page template could look something like this.

Gatsby’s page queries work by creating a singular query to the data layer and using the id from context, lookup the node in Gatsby’s data layer and return the resulting values. 

This can be see here:

And any data returned from this GraphQL query is then passed back to the page template via the data prop.

This can be see here:

The full template from my demo can be found here: src/templates/template.js. 

What you do with your data is entirely up to you. In my demo I’ve used the fantastic TailwindCSS to make it look pretty. There’s a guide in the Tailwind docs that will explain how to add TailwindCSS to your Gatsby project: Install Tailwind CSS with Gatsby.  

So there you have it, you can now source data, create nodes, decide how to defer the creation of pages, query page data and return data from Jsx. 

I’d love to see what you build so please do come find me on Twitter: @PaulieScanlon 

Ttfn 

Paul 🕺