Achieving 20x Build Speedup: Introducing esbuild-scripts

Since the release of esbuild that gives a theoretical 100x speedup, I always imagined adopting it to significantly improve the build time of my own websites. After some investigation and proof-of-concept work, I figured out how to write some minimal amount of code to use esbuild, an extremely fast but limited build tool, to bundle and generate minified JavaScript for static sites.

The end result is a 20x speedup on my personal website:

20x performance improvement

Introduction to JS Bundlers#

A JavaScript bundler takes in one or several entry points, starts from there to include in the files they imported and all other transitive imports. When the files are imported, usually they are not in the form that the browser can execute directly. For example, some files might be written in TypeScript, and some might be using the latest syntax that does not have good browser support yet. Therefore, the imported files are usually first transpiled before being included in the final bundle. Finally, the bundle will be minimized by removing whitespaces, renaming identifiers, etc. The entire flow can be described by the chart below:

bundler chart

Usually, the transpilation part is done by Babel and the resolving dependency part is done by webpack. Minimization is usually done by a webpack plugin. Babel and Webpack are both written in JS. They allow excellent customization to support almost every possible need on the planet. However, such power of customization comes with performance cost: when every part of the system can be dynamically modified on a single-threaded JavaScript VM, much less can be done for performance.

esbuild changes the game by rewriting the entire JavaScript bundler in Golang, with as few AST passes as possible and maximizes the opportunity for parallelization. For more detail, you can take a look at esbuild's official explanation.

esbuild also provides some API for generating bundles and running dev servers:

import { serve, build } from 'esbuild';
// Start devserver
const server = await serve(
{ servedir: 'public', port: 19815 },
{ outfile: 'public/app.js' /* additional build config */ }
);
console.log(server.host, server.port);
// Build bundle
await build({ target: 'es2019', outdir: 'build' });

Therefore, if we only want to build what is similar to create-react-app, this is almost everything we need to do. We only need to create an HTML file that loads in the bundled JS and CSS.

Filename Hashing#

Although linking the file generated by esbuild directly to HTML is easy, it still has its shortcomings. One of the biggest issues is that it does not work well with browsers' caching strategy. The browser will usually cache the downloaded assets by filename, so that loading a second page of the same website doesn't need to reload everything again. However, if we always give the same name to the file, the browser may fetch the stale JS/CSS when the content has already been updated.

Therefore, most bundlers will generate a filename with a hash in it, so that the cache can be correctly invalidated by the browser. Below is an example of Next.js doing it:

Example of filename hashing on nextjs.org

esbuild also supports generating filenames with hashes with the following additional build configuration:

import { build } from 'esbuild';
build({
// Other build options ...
assetNames: 'assets/[name]-[hash]',
chunkNames: 'chunks/[name]-[hash]',
entryNames: '[dir]/[name]-[hash]',
});

However, generating the filenames based on content means that the original easy approach of referencing the generated JS/CSS directly to the static HTML file will no longer scale. Therefore, we will need a way to dynamically patch the original HTML file to include the JS/CSS with correct names.

Luckily, esbuild supports this use case. We only need to add the flag write: false, then we can access the output files with names:

import { build } from 'esbuild';
const { outputFiles } = await build({
// Other build options ...
assetNames: 'assets/[name]-[hash]',
chunkNames: 'chunks/[name]-[hash]',
entryNames: '[dir]/[name]-[hash]',
write: false,
});

Each output file contains the file content and the full path to the file, and we only need to parse the HTML and add in the CSS and JS as follows:

<!DOCTYPE html>
<html lang="en">
<head>
<title>test</title>
<!-- add css here -->
<link rel="stylesheet" href="/client-DJ6V5JGM.css" />
</head>
<body>
<div id="root"></div>
<!-- add js here -->
<script src="/client-X3L5TPIO.js"></script>
</body>
</html>

Server-side Rendering#

Generating an app with an empty <div id="root" /> and use React to saturate it at runtime is good enough for dynamic applications, but not very ideal for static sites. Static websites like personal websites and documentation websites care strongly about search engine indexing. Thus, we must saturate the children of the initial <div id="root" /> with pre-rendered HTML from the React code.

Fortunately, React already provides an official server-side rendering solution for Node:

import { renderToString } from 'react-dom/server';
import App from './App';
const stringOfServerSideRenderedHTML = renderToString(<App />);

To effectively use the result of renderToString, we need to create another entry point only for the purpose of server-side rendering:

// server-entry-point.tsx
import { renderToString } from 'react-dom/server';
import App from './App';
module.exports = renderToString(<App />);

The final line of module.exports makes this a CommonJS module importable by Node. With this entry point, we can instruct esbuild to generate a bundle that gives the full pre-rendered HTML once it's imported:

import { resolve } from 'path';
import { build } from 'esbuild';
await build({
// Other build config...
entryPoints: ['server-entry-point.tsx'],
platform: 'node',
format: 'cjs',
logLevel: 'error',
outfile: 'ssr.js',
});
const preRenderedHTML = require(resolve('ssr.js'));
attachPreRenderedHTML('public/index.html', preRenderedHTML);

There is one additional practical issue we need to resolve: there is some code that is designed to be only executable on the client-side (e.g. Google Analytics, etc). Therefore, the bundler and static site generator need to provide a way to gate the code.

Luckily, esbuild provides the define API to declare some global constants that can be inlined with valued defined at build time. A classic example is the process.env.NODE_ENV variable. By convention, it's set to either "production" or "development", and bundlers use this variable to generate production or development builds. In our use case, I declared a __SERVER__ global variable, and set it to true for SSR build and false for normal client build. To support TypeScript, we just need to declare it in a d.ts file along with other CSS type definitions:

// types.d.ts
declare module '*.css' {
const src: string;
export default src;
}
declare module '*.scss' {
const src: string;
export default src;
}
declare const __SERVER__: boolean;

Then users can reference the type in their own type definition file by:

/// <reference types="esbuild-scripts" />

Code Splitting#

After all the steps above, we might generate a big bundled JS file. This might be OK for small applications, but not desired for large web applications that might want to lazy-load some of the code. Bundlers like webpack and esbuild recognize the need, and will automatically generate split chunks when it sees a dynamic import like import('./dynamically-imported-component').

Enabling code splitting is easy in esbuild:

import { build } from 'esbuild';
build({
bundle: true,
format: 'esm',
splitting: true /* other configs */,
});

In React, we can use the lazily-imported components by lazy and Suspense:

import React, { Suspense, lazy } from 'react';
const LazyComponnet = lazy(() => import('./other-component'));
const element = (
<Suspense fallback={<div>Loading...</div>}>
<LazyComponnet />
</Suspense>
);

There is only one issue: React.Suspense does not support server-side rendering. Therefore, we need to provide a shim around it for applications that want SSR. We can easily do this with the __SERVER__ flag:

import { Suspense } from 'react';
const SSRSuspense = __SERVER__
? ({ fallback }) => fallback // use fallback in SSR mode
: Suspense;
export default SSRSuspense;

Final Comments#

It's important to note that the article is written based on my 0.1.1 release of esbuild-scripts. The information and strategies presented in this article might no longer be accurate when additional features are added.

Despite advanced features support, esbuild-scripts is still lacking some core features. In the near term, my primary goal would be adding support for multiple entry pages and filesystem-based routing. CSS modules probably will not be supported unless they can gain some native support from esbuild directly.

Want to try?#

yarn add esbuild-scripts
yarn esbuild-scripts help

You can also check the source and documentation on GitHub. Any contribution is welcomed.

It's the time to get a massive build-time reduction on your CI that you deserve!

Build time reduction