Writing

zot continuous: Agents That Survive

Oct 13, 20265 min read

A persistent, recoverable execution layer for zot. Every model request and tool call is committed before and after it runs, so a crash is just a pause. Here is what it is, how it works, and what you can build with it.

Agents are fragile in a very boring way.

Close the laptop lid, lose the SSH connection, hit Ctrl+C at the wrong moment, or let the process get killed, and the run is gone. At best you have a session file with half a conversation in it. At worst the agent was in the middle of a bash call and you have no idea whether that migration ran or not.

Event sourcing taught me that this is a solved problem, as long as you take history seriously. So I built zot continuous: the persistent, recoverable execution layer for zot.

What it is

zot continuous sits between you and the regular zot engine. The engine still does the actual work: talking to the model, running tools. zot continuous decides what runs, records what happened, and refuses to repeat an effect it cannot prove did not happen.

Every model request and every tool call is committed to a local store before and after it runs. If the process dies, the next step picks up from the last committed boundary. A crash is not a failure anymore, it is a pause.

It is pure Go, ships inside the same single binary, and adds no new dependencies. It is also explicitly experimental: the journal format, Go APIs, and host protocol may still change. Ordinary zot workflows are untouched.

The quickest way in

No daemon, just a store directory:

zot continuous run "summarize the README" --store ./store --tools read,glob
zot continuous run "now list every TODO" --store ./store --tools read,glob

The second call continues the same conversation with its full history. If something was interrupted, you resume it without adding new input:

zot continuous run --resume --store ./store --tools read,glob

There is no --continue or --session here. The store is the authority, not a session file.

Recovery that does not lie

The interesting part is what happens to a tool call that was running when the process died. zot continuous does not guess. Each tool declares a replay contract:

  • never (the default, including bash, write, edit): the model gets an interrupted error result and decides what to do next.
  • safe (read, glob): re-executed after a fresh authorization check.
  • idempotent: re-executed with the same stable operation key, which the receiver must enforce.
  • reconcile: the tool is asked whether the operation completed, and it only runs again if the answer is "not started".

Unknown outcomes are reported, never retried. A policy is never widened between the stored intent and the live tool. You can even ask beforehand what recovery would do:

zot continuous recover --dry-run --store ./store

That is the whole point: an agent that might have charged a credit card twice is worse than an agent that stops and tells you it does not know.

A host that clients come and go from

One-shot runs are nice. The real fun starts with a host:

zot continuous serve --store ./store --tools read,glob
zot continuous attach "summarize the README" --socket ./store/host.sock --workspace main
zot continuous attach --follow --socket ./store/host.sock --workspace main

serve owns the store, recovers interrupted runs, works through queued prompts for every conversation, and listens on a 0600 Unix socket (or an authenticated WebSocket). Clients submit, watch, and leave. Closing a client never cancels host work.

The regular TUI can attach too:

zot --continuous ./store/host.sock --continuous-workspace main

Open it in two terminals and both views stay in sync: prompts, streamed text, tool progress, and answers show up everywhere. Start a long refactor on your desktop, close the window, check on it from your laptop later. It just kept going.

What else is in the box

A lot, honestly. The highlights:

  • Persisted approvals. A tool call can require a human decision. The pending approval is committed, survives restarts, binds to the exact arguments, and is never implicitly approved. Decide it whenever: zot continuous decide <id> --allow --scope call --reason "reviewed".
  • Steering. Send new input while a run is active, and it joins at the next request boundary instead of fighting the current tool round.
  • Budgets. Set a USD limit per conversation or for the whole runtime. Unknown costs count as reservations, never as zero, and an exceeded budget fails the run before the request is sent.
  • Usage ledger. One immutable row per attempt and compaction.
  • Forks and resets. Branch a conversation at any committed entry, or restart the model context with a handoff note while keeping the full history searchable.
  • Compaction. Automatic, optionally in the background, and it never splits a tool round. Old entries stay in history, they just leave the model context.
  • Subagents and Swarm. Subagents are owned conversations on the host, scheduled and recovered like everything else. /swarm on an attached TUI runs every agent there.
  • Tasks and documents. Versioned tasks with timers, dependencies, retries with backoff, and cleanup as compensation. Typed, versioned documents for structured state like todo lists.
  • Remote workers. Run tools on another machine with zot continuous worker --root name=directory. A lost connection is an unknown outcome, reconciled later, never blindly retried.
  • Provider routing. Each conversation keeps its own provider and model, switchable without restarting the host.
  • Operations. verify, backup, restore, migrate, full-history search, and a SQLite backend if you prefer that over the local journal.

What you can build with it

This is where it stops being a feature and starts being a platform.

Long-running coding sessions. Kick off a big task on a dev box, detach, reattach from anywhere. Nothing dies with your terminal.

Agents with a human in the loop. Let the agent run freely, but require an approval for bash or deploys. The request waits, durably, until someone says yes. Hours later is fine.

Bots and integrations. Talk to the host over its JSON protocol with stable request IDs. A Slack or ticket bot can retry a submission as often as it likes and still get exactly one answer. Pending approvals and blocked runs land in a notification outbox with at-least-once delivery.

Scheduled and background work. Timers survive downtime and fire once afterwards. Reminders, nightly reviews, periodic checks, all as tasks with retries and cleanup.

Multi-agent setups. A read-only reviewer subagent with its own tool set, parallel threads, forks that explore alternatives from the same history.

Embedding it in your own Go program. The runtime is a Go package. The repository has runnable examples using a synthetic provider, so you can try crash recovery, approvals, forks, tasks, idempotent tools, and remote workers without credentials:

go run ./examples/continuous/07-approval
go run ./examples/continuous/15-remote-worker

Why I care about this

I come from the world of event sourcing and built GenesisDB, an event-sourcing database engine. zot continuous is basically an event store for agent execution: an append-only history of intents and results, with recovery that follows the facts instead of hoping for the best.

Agents are going to run longer, touch more systems, and do things that matter. That only works if the runtime knows exactly what happened, and is honest when it does not.

Give it a try, break it, and tell me what you build: zot.sh and the continuous docs.