This article is the first of a two part series, on the engineering behind my website. Originally published here
For the past couple of weeks, I’ve been rebuilding my personal website from scratch, live at https://jiahao.codes with the source code on GitHub. In this article, I'll tell the story of this long overdue rewrite and talk about the new static site framework I eventually settled on, Gatsby.
Background
For a bit of background, I originally built my website while I was an undergraduate back in 2016. It was a single page React application, with the design based around an interactive search box where visitors could type in things. The application would then change based on what was typed in, to show projects I’ve done, like a website resume or portfolio. I tagged it on Git for posterity here.
Some drawbacks were that the content was very coupled with the code to style the page, and it was difficult to change the theme without having to rewrite a lot of code. It uses a customized version of JSON Resume to display data, but ultimately I didn't do a good job and it was a bit of a mess.
Furthermore, being a single page application, it takes a little too long to load just to display some static content. I wanted to add routing and different pages to add blogging functionality, which it wasn’t really suitable for without major refactoring.
First Iteration — Jekyll
This led me to pick something very minimal and lightweight to rebuild it. In my time at Grab, I worked a little on maintaining our engineering blog which was built on top of Jekyll. I liked how minimal it was, you just had to pick a theme and drop in your posts in the markdown format. There is clear separation of concerns between the content and the site design.
It’s nice that pages, layouts, and categories in Jekyll are just small building blocks that are composed together to build a website, making it easy to reason about how everything fits together. I picked the Hyde theme to start off, and played around a little with the CSS to touch up a few areas.
When delivering content to users on mobile devices, it is important to optimize delivery of images to the users. Users on smaller mobile devices should not need to load large image resources, but instead should load the appropriate image for their viewport. HTML5 specifies the srcset attribute for <img /> tags which is ideal for this.
However, it would be a pain to have to manually convert images into different sized thumbnails for a post. I wanted an automated image processing pipeline to automatically resize the images extracted from the markdown documents, and then automatically populate the srcset attribute on the images in the output HTML document. I found the Jekyll Responsive Image Plugin great for this. It allows me to create templates which will be used by Jekyll when rendering the markdown and automatically does the image resizing.
Even so, when I tried to do more complicated workflows like adding CSS preprocessing with dependence on the JavaScript ecosystem with the Node Package Manager (npm), it became a lot more convoluted. Looking at a few recipes I’ve found, I would have to dive down the road of writing Gulp workflows and somehow connect them to Jekyll commands. I also chanced upon the Jekyll Asset Pipeline which seems what I could use. I didn’t dive too deep into it, but from brief glances it seems like I would have to come up with a lot of custom scripting to interface with JavaScript libraries on my own.
I guess having used webpack at work, I was pampered by this open source community where there are loaders and documented recipes for doing almost anything, granted that someone was willing to wade into the world of "JavaScript fatigue". Around the same time, @yangshun introduced me to Gatsby, a React static site generator which seemed really fascinating. It also seemed a good opportunity for me to get my hands dirty with frontend development again.
As I had some free time on my hands, why not rewrite everything again and keep myself updated with the ever-changing JavaScript ecosystem? Seems like a lot of fun!
Final Form — Gatsby
Gatsby is a static site generator that can render sites from markup documents using templates defined as React components. It functions similarly to Jekyll, where you can pick a starter project, drop in some markdown documents for articles, and be rewarded with a website with minimal effort.
It offers much much more, however. Gatsby lets me leverage all the modern tools for building web applications and to add interactive experiences for visitors like a fully fledged React application. Not only that, it is unlike traditional single page applications, and works without JavaScript! Things would certainly be more complicated if I were to add a JavaScript compilation pipeline to a Jekyll site, and a JavaScript framework would be a better fit.
How It Works
Let me briefly give a high level overview of how Gatsby works from my short experience using it.
Developing the application is the same as developing a React application, with useful things like hot module replacement to aid easy development already set up by Gatsby. What’s different is that there is a framework in place to use GraphQL queries to pull content into the client side browser environment. React components can define a query and the component will be hydrated with the result of the query as props when it is rendered.
For example, I defined a PostTemplate which will be used to render pages for articles.
When the <PostTemplate /> component needs to be rendered into a page, the accompanying exported pageQuery, a GraphQL query is made, and the results are passed in as props into the component.
The real magic happens when the website is compiled into a production bundle. Running npm run build will tell Gatsby to perform all the GraphQL queries defined and render all the React components into an HTML document, using a technique known as server-side rendering. This means that everything “React” is serialized and compiled to static HTML, ready to be viewed without JavaScript. Visitors to the site will then be able to quickly load and interact with the static version of the page.
Not only that, within the HTML document, there are instructions to load the JavaScript bundle of your application asynchronously. When it has been loaded, the content displayed in the browser will be dynamically replaced by the React application, gaining interactivity. This also happens with the other pages of your site — Gatsby will ensure that they are asynchronously loaded so that when you click on a link, the data is already cached on the browser for React to swap out the DOM elements that need to be changed. Everything is done to give the illusion of speed to the viewer while asynchronously loading everything in the background.
Plugins
Because of the APIs exposed by Gatsby for interfacing with its internals, powerful plugins can be built to extend the core functionality of Gatsby and reducing work by developers.
Node.js APIs
- Can be extended with a
gatsby-node.jsfile in the root of the project
The Node.js APIs let plugins extend or modify the heavy lifting performed by the Node.js process when compiling the application. Your gatsby-node.js file can export functions which modify the GraphQL data that is provided to React components when they are rendered. The APIs are also used by plugins to extend the internals of Gatsby e.g. the default webpack config can also be customized here.
Take the example of what happens during the processing of markdown files into pages. The gatsby-source-filesystem plugin scans directories and from files it finds, creates File nodes. These File nodes are then processed by gatsby-transformer-remark, parsing the markup into HTML with the Remark markdown processor.
Server-side Rendering APIs
- Can be extended with a
gatsby-ssr.jsfile in the root of the project
The server side rendering APIs allow hooks to be defined to modify the rendering process of the application. For example, the Typography.js Plugin uses this to inline the styles required into the DOM head when rendering.
Browser APIs
- Can be extended with a
gatsby-browser.jsfile in the root of the project
Finally, the browser APIs allows plugins to run code on lifecycle events while Gatsby is running in the browser. The Google Analytics Plugin uses these APIs to track the location of the user on route changes.
The best part about using Gatsby is that there plenty of plugins available leveraging these APIs that give us new features almost for free. Want to add offline mode to your application? RSS feeds? Styled components? Just lookup the available plugins and drop them into the gatsby-config.js!
Next Steps
Jump over to my website to read the follow up post where I showcase the amazing integrations and features I have set up, to be on the cutting edge of web development!
Check out the source code for my site on GitHub!