The dev site is the thing I test

On 30 September I took my own website back to how it looked on 21 May. A redesign had gone live, including new content pages and a page for a product I was still shaping, and I wanted it off. At about quarter to one I asked for the previous layout back. By 13:14 the live site was rolled back to the May build, and every piece of new work was kept on a separate branch to carry on there.

Nothing had broken, as such. The problem was simpler and more common. Work I hadn't properly tested, on a phone, in the way a visitor would see it, was in front of the public. And I'd only really looked at it once it was there.

This is how I fixed that, the near miss a few days later that proved the fix mattered, and the rules I work to now. It applies to anyone letting AI build a website for them, whether that's one agent or a set of them.

Why it happened

By late September most changes to ai-ops.org were being built by my factory, the set of AI agents that picks up written jobs, builds them, checks its own work and hands it back. (I've written about how that started in Building the metafactory.) The factory is good at building. It's also good at proving its own work passes its own tests.

What it can't do is look at a page on my phone and tell me whether it feels right. Tests check that the code does what it should. They don't check whether the headline lands, whether a section reads as too salesy, or whether a page should exist at all yet. Only I can do that, and I'd been doing it too late.

The site at the time had one target. Building it published it. There was no in-between place where a finished change could sit and wait for me.

A separate place to build

The same afternoon, a few minutes after the rollback, the branch carrying the redesign was pointed at a separate copy of the site. The commit message says exactly what it does: "deploy this branch to the separate ai-ops-dev Worker, never the live one". A Worker is the small program on Cloudflare (the service that hosts my site) that serves the pages. Two Workers means two sites: the live one, and a dev site that's an exact build of the work in progress.

That was the mechanism. The habits took a few more days, and it took a near miss to make them firm.

4 October: the preview that wasn't

On 4 October a ticket for a separate offer page had a review step written into its instructions. The plain-English line said to deploy the work "to dev", as "a Cloudflare preview of the branch, not main", so a reader could see the page before it was signed off.

The command actually listed underneath was the general deploy command. And on that branch, the settings file named the live site as the target. A Cloudflare deploy takes its target from that settings file, not from the branch name and certainly not from the words in a set of instructions. Run as written, the "preview" step would have published the live site, including a page that wasn't signed off, with lines on it that had already been superseded and a note to myself about confidentiality that was never meant to be seen.

It was caught before it ran. But it shows the trap clearly. Instructions described the intent. The mechanism did something else. When an AI agent follows a written process, it follows the commands, and the commands win.

5 October: the rule

The next day I put the rule in my own words, and it's now at the very top of the site's instructions, above anything older that disagrees with it:

"if we want to test UI and UX the build deploys to a dev site and that dev site is the thing that is iterated on"

In practice that means five things.

1. Build into the real site on the dev branch. Mock-ups are allowed, but only early and cheap, and only when the direction is genuinely open. Once I've picked a direction, the work goes into the real pages, not more mock-ups.
2. Every change redeploys the dev site automatically. Each commit (a saved change) on the dev branch rebuilds the dev site within about five minutes. Nobody has to remember to do it.
3. Only I can get in. The dev site sits behind Cloudflare Access, a login gate that only lets through my own email addresses. It's private by default, so half-finished work is never one shared link away from a stranger.
4. Every job hands me test links. A ticket that changes the site isn't done when the code passes. It's done when it tells me the exact page to open and what to look for on it.
5. Live only when I say "ship it". The dev branch never goes to the live site on its own. Production happens when I've tested the dev site and said so, in those words. Payments on the dev site run in Stripe's test mode only, so nobody can be charged from a page that isn't live.

There's a sixth rule that follows from the others: a dev site that fails to build, or shows an error page, counts as a fault in whichever ticket caused it. If the thing I test isn't working, I can't test.

Two routes in, the same day

My first ask that day was blunt. I needed to be able to review dev sites, or I'd never be able to test the look and feel properly. I was travelling with a laptop that wasn't always open, so whatever we built had to work from my phone.

The first version was served from the box, my always-on server, over Tailscale (a private network that links my own devices). Each site's dev branch was checked every five minutes and rebuilt when it changed, and a failed build kept the last good version running. It worked, but only on my own devices.

The same day the Cloudflare route went in alongside it: the dev Workers behind Access, rebuilt on every dev-branch commit. The box version stayed as a fallback. Since then, every review page I get links straight to the dev pages that a ticket changed.

What the dev site has caught

The point of all this is that I find problems before visitors do. A few examples from the first ten days.

Pages that shouldn't have existed. On 6 October a final pass over the redesign found two case study pages that were placeholder copy ("This case study is being developed", with the industry listed as "TBC"). The listing hid them and the site map left them out, so the site had already decided they weren't public. But the page route built itself from the full list, so anyone who guessed the address could read them. Now only published case studies get a page at all, and an unpublished one returns "not found".

A form you couldn't read. The sign-up form at the foot of every article was using its light version on a dark background. The success message measured at a contrast of 2.35 to 1, well under the accessibility standard. There's now a check that reads what each form sits on and fails if the version doesn't match.

A home page that crashed. On 30 September the preview home page returned an error, because the hosting preview couldn't read the folder the articles live in, and the code assumed it always could. It now shows no articles rather than crashing, and a test proves it.

None of these were dramatic. All of them would have been mildly embarrassing in front of a prospective client. That's most of what a dev site saves you from.

Names need permission, so the build checks

The case studies raised a separate point. A website about how I work with clients wants real clients on it. A real client has to agree before they're named. And an AI agent writing helpful copy will happily name anyone it finds in my notes.

So the site has a consent check. It's a test that reads every published page, every component and every piece of content, and fails the build if any name on a blocked list appears. The list holds every client and contact who hasn't agreed to be named. A name only comes off the list when there's a recorded yes.

Matthew Webber, the founder of Five Education, agreed on 13 October to be named and quoted, so his name came off the list that day. Everyone else stays on it until they say yes. I don't have to remember who's agreed to what. The build remembers for me, and it fails loudly if a draft forgets.

Articles get checked too

The same idea runs through the articles section, including this article. Every article has to pass a set of house checks before the site will build:

• It uses one of four known categories.
• There are no long dashes anywhere in the title, summary or body (a small house style point, but also a reliable sign of copy written on autopilot).
• Articles that answer a reader's question carry that question, a two or three sentence answer, and a short set of questions and answers, so search engines and AI answer tools can quote them accurately.

None of that replaces me reading the piece. It means that when I read it, I'm reading for whether it's true and whether it sounds like me, not checking punctuation.

The numbers

From 30 September to 13 October there were 262 commits on the dev branch. The busiest days were 5 October with 77 and 12 October with 94. Each one rebuilt the dev site. None of them changed the live site.

That's the whole idea in one line of numbers. A lot of change, at whatever pace the factory can manage, and none of it public until I've looked.

If you're doing this yourself

You don't need a factory or a custom setup. Most website builders and hosts have some version of a staging site (their name for a dev site). Here's what I'd set up, in order.

1. Make the dev site a separate target. Not a setting you remember to change. A different site, with a different address, that the live site can't be published from by accident.
2. Lock it. Put it behind a login that only you, and anyone you choose, can pass. A private link is not the same as a locked page.
3. Rebuild it automatically on every change. If someone has to remember to update it, it'll be out of date the day you need it.
4. Test it on the device your customers use. For most small businesses that's a phone. Look at it the way a stranger would.
5. Make "go live" a separate, deliberate step. Choose a phrase. Mine is "ship it". Nothing goes live without it.
6. Turn your rules into checks. If something must never appear on your site, like an unconsented client name, write a check that fails when it does, and run it on every change.

What I've learned

The rollback on 30 September felt like a step back at the time. It was the most useful thing that happened to the site all autumn. It forced a clear line between building and publishing, and once that line existed the factory could build as fast as it liked. Speed stopped being a risk because nothing it built reached anyone until I'd looked.

The near miss on 4 October taught me something I'd carry well beyond websites. When you hand a process to an AI agent, the commands are the process. If the words say "preview" and the command says "publish", it will publish. So make the safe path the only path the commands can take.

If you'd like to set up something similar for your own site, let's have a chat about it.