← Back to all work

Technical article Tooling esbuild

I built a CRO build tool. Here's what I changed.

Every CRO developer eventually builds their own tooling. You start with the boilerplate someone shared in a Slack group, hack on it for a few projects, and eventually you've got a Frankenstein Gulp file that nobody else can understand, including you, six months later.

I've been through this twice. The first tool was Gulp + Rollup. It worked, but builds took 8 to 15 seconds and every time I switched experiments I had to open the loader script and paste in the new ID. The second was a webpack-based setup from a previous employer. Faster, but same problem. Different syntax, same friction.

So I built abtestrig, a CRO-specific build tool that fixes the things I was working around every day.

The problem that started all of this: the loader script

Here's how most CRO dev setups work. You run a local dev server, then you install a browser extension (User JS & CSS, Tampermonkey, similar) that injects your local JavaScript onto the client's live site. The extension needs to know two things: what site to run on, and where to load the code from.

The second part is always hardcoded. Something like:

fetch('http://localhost:3000/SCR001.js')
  .then(r => r.text())
  .then(code => eval(code));

Every time you move to a new experiment (SCR002, SCR003, whatever) you open the extension, find that string, change it, save. It takes fifteen seconds. But you do it dozens of times a week, and it breaks your flow every single time. You forget to update it and spend ten minutes wondering why your changes aren't showing up, only to realise you're still loading the wrong file.

With abtestrig, the loader script has no experiment ID in it at all. Instead, when you run abtestrig watch --cn Screwfix --fn SCR001, the dev server writes a config.json with the active experiment details. The loader reads that on connect. Paste the loader script into your extension once, and you never touch it again.

// There is no experiment ID hardcoded here.
// The active experiment is announced over WebSocket by `abtestrig watch`.

This sounds like a small thing. It isn't.

Port discovery: the loader that survives server restarts

There's a second version of the same problem. Port 3000 is busy (another project, a local dev server, whatever) so your new server lands on 3001. Now you update the loader script again. Or you forget, spend five more minutes confused.

The scriptLoader probes ports 3000 to 3009 in parallel on startup:

const ports = [3000, 3001, ..., 3009];
Promise.any(ports.map(probe)).then(port => connect(port));

Each probe fetches /config.json and validates the response is actually an abtestrig server (has id + client, or a non-empty experiments array). Whichever port responds first wins. The actual port is written into config.json, so the loader knows where to connect the WebSocket.

The extension survives a server restart on a different port without any manual intervention. You run abtestrig watch, open the browser, the extension finds the server. That's it.

esbuild: builds in under 200ms

The old Gulp + Rollup pipeline took 8 to 15 seconds per build. With esbuild, a full build, including Sass compilation and PostCSS, finishes in under 200ms. Watch mode rebuilds in under 50ms.

This matters more than it sounds. At 8 seconds, a slow build is an interruption. You lose the mental context of what you were changing. At 200ms, it's imperceptible. You save the file, the browser updates, and you're still thinking about the experiment.

esbuild also handles the constants cleanly. The old tool used string replacement ({{ID}}, {{VARIATION}}) which meant shared.js was always dirty in git after starting the dev server. esbuild's define API replaces them at bundle time, so the file on disk never changes:

// shared.js: never modified, never dirty
export default {
  ID: __ID__,
  VARIATION: __VARIATION__,
  CLIENT: __CLIENT__,
  LIVECODE: __LIVECODE__,
};

esbuild inlines the real values during the build. Running git status mid-development shows nothing it shouldn't.

CSS hot-swap without a page reload

The old setup reloaded the page when CSS changed. On most client sites, a page reload means re-running the activation conditions, waiting for the dataLayer, waiting for React hydration, re-triggering the pollers. On a Next.js site with a 1-second DOM render delay, that's at minimum a second of staring at a blank state every time you tweak a margin.

abtestrig injects CSS via a <style> tag with a stable ID:

const style = document.createElement('style');
style.id = `cro-dev-css-${id}`;
style.textContent = css;
document.head.appendChild(style);

When the CSS rebuilds, the WebSocket sends a css-updated message, the loader fetches the new stylesheet, removes the old <style> tag, and appends the new one. No page reload. The experiment stays active, the DOM stays as-is, only the styles change.

For JS changes, a reload is unavoidable. The experiment code runs once and modifies the DOM, so re-running it means starting fresh. But CSS iteration is something different. A lot of CRO work is visual. Being able to tweak padding, colour, layout without losing the activated state makes that iteration genuinely fast.

Stack mode: interaction testing before things go wrong

There's a category of CRO bugs that only show up in production, when two experiments are live at the same time. One experiment adds a .product-badge element. Another experiment has a CSS rule targeting .product-badge. Neither test showed the conflict in isolation.

Stack mode runs multiple experiments on the same page, in a deterministic order, so you can catch these before they ship:

{
  "name": "homepage-conflict-test",
  "experiments": [
    { "client": "Screwfix", "fn": "SCR002", "variation": "1" },
    { "client": "Screwfix", "fn": "SCR003", "variation": "control" },
    { "client": "Screwfix", "fn": "SCR001", "variation": "1", "watch": true }
  ]
}

abtestrig stack stacks/homepage-conflict-test.json

The frozen experiments ("watch": false) are built once. The one you're actively developing ("watch": true) gets the full live-reload setup. JS injection is synchronous and order-deterministic: the first entry's code finishes executing before the next script tag is even parsed.

The scriptLoader gets the full list from config.json and injects them in sequence. Each experiment has its own isolated DOM slots (cro-dev-css-{id}, cro-dev-js-{id}), so live-reloading one doesn't disturb the others.

The global helper library

One thing that accumulated across both previous tools was a utils.js that got duplicated into every experiment folder. pollerLite, waitForElement, onUrlChange, copy-pasted every time, gradually diverging between projects.

abtestrig has a global lib/ directory at the repo root, importable from anywhere via @lib:

import { pollerLite, waitForElement } from '@lib/dom';
import { onUrlChange } from '@lib/url';
import { ensureTracking, fireEvent } from '@lib/events';

The @lib alias is resolved by an esbuild plugin at build time. No runtime overhead: only the functions you import end up in the bundle. The library itself is vanilla JS with no external dependencies, so bundle sizes stay predictable.

@lib/events handles the tracking setup that's genuinely annoying in CRO: you want to fire an event as soon as possible, but Tealium or gtag might not be loaded yet. ensureTracking queues events until the analytics pipeline is ready, prefers utag.link() if Tealium is present, falls back to injecting gtag.js if it isn't.

Client templates

Different clients have different activation patterns. Screwfix is a Next.js SPA: you wait for #__next, window.utag, window.__NEXT_DATA__ before doing anything. A static e-commerce site just needs body. A Tealium-heavy site needs utag.data to be populated before you can check page conditions.

abtestrig create

The create command detects the client name and picks the right template:

Screwfix           -> templates_custom/screwfix
Travis Perkins     -> templates_custom/travisperkins
Avon               -> templates_custom/avon
Everything else    -> templates/core

Each client template has the activation conditions, tracking setup, and boilerplate already wired up. Starting a new Screwfix experiment gives you a triggers.js that waits for the right things, an experiment.js with the URL-change handler pre-configured, and trackOptimizelyEvent imported and ready.

The --template flag overrides the auto-detection if you need to reuse a pattern across clients that don't match by name.

One command

The thing I wanted most from any build tool was for the whole local dev workflow to be a single command. No separate terminal for the dev server, no separate command for the watcher, no manual restart when something goes wrong.

abtestrig watch --cn Screwfix --fn SCR001

That starts the esbuild context, the Chokidar SCSS watcher, the Express server, and the WebSocket server. It handles port conflicts automatically. It cleans stale files from server-dist/. It tells you exactly what's running and where.

When you're done, Ctrl+C shuts everything down cleanly.

Open source

abtestrig is MIT licensed and available on GitHub. The scriptLoader, the stack format, the @lib API, the template system: all of it is documented in the README.

If you've been maintaining your own CRO build tooling and some of these problems sound familiar, it might be worth a look.

← Back to all work