Durable execution: agents that survive the server going down

Building a website for a client is not a thirty second operation. You research the business, write copy, generate images, wait for approvals, buy a domain, configure DNS, publish. In between, the client takes a weekend to reply. The whole thing can run for days.

Our first version of that was written the way everything gets written the first time: an in-memory process with a pile of states. It worked right up until we deployed. Then the process died, and the state of every project in flight died with it.

The missing pattern

What we needed has a proper name: durable execution. It is a class of engine that journals every step, so a workflow can resume exactly where it stopped no matter what crashed in between.

The core idea is that workflow state stops living in process memory and becomes a persisted fact. A workflow can sit waiting for a human answer for days or months without consuming anything, and when the answer arrives, it continues. No hand-written state machines, no "which step is each project on" tables, no scheduled jobs polling for changes.

We migrated to this and an entire category of bugs disappeared. We did not fix them: they stopped existing, because the thing causing them was gone.

How we use it

Each project type has its playbook: a sequence of phases with their tasks. Landing page, corporate site, online store, Google campaign, Meta campaign. The engine is the same; what changes is the definition.

Every task in a playbook can be resolved three ways, and that flexibility turned out to be the most valuable part of the design:

• An agent executes it, when the task is narrow and verifiable.
• A person executes it, when it needs judgement or client contact.
• It waits, when it depends on something external: client approval, DNS propagation, a payment clearing.

The same project can move from autopilot to human hands and back without the workflow noticing. That is precisely what lets you sell a subscription service without the operation depending on somebody watching.

What it does not solve

The limits deserve honesty, because enthusiasm around these tools runs high. Durable execution guarantees your workflow does not get lost. It does not guarantee your workflow is any good.

If your playbook has a badly designed phase, you now have that badly designed phase, executing perfectly reliably, forever. The engine amplifies whatever you feed it: it makes the good parts scale and the bad parts too.

That is why the real work was not in the infrastructure, which is solid and has been solved for years. It was in arguing, task by task, about what has to be true for each one to count as done. That argument is not one the engine can have, or the agent. That one is yours.