Ayush Basak

The Agent Was Interrupted. Should Its Work Disappear?

Sep 16

Two agents walk into the same incident report. One is correcting the timeline. The other is reorganizing the service documentation around what the incident taught the team.

Both edits are useful. Both depend on material the other might change. Neither agent knew its complete read set before it started searching.

Now interrupt one of them halfway through. Should everything it wrote disappear?

For a transfer between bank accounts, the answer can be yes: a partial operation is unacceptable. For an independently useful, source-backed wiki page, deleting completed work may be the expensive part of recovery. The storage system cannot choose between those answers without knowing what the product promises.

Agent swarm concurrency control is the design of safe interruption boundaries: what a worker may observe, what it may publish, and which intermediate states remain valid when that worker stops. More agents do not resolve this question. They make it urgent.

Robert Escriva’s Agent Swarms are a Distributed Systems Problem describes Chroma’s Fission protocol for agents editing a shared wiki. Its central trade-off is retaining page-level progress rather than demanding one atomic publication of an entire reasoning batch. This article builds an original example around that idea and examines the guarantees an application must still enforce.

Try the interruption before reading the protocol

The lab follows a younger agent, B, preparing an incident page and an index link. An older agent, A, subsequently needs a lock held by B. B is interrupted, releases its locks, and later resumes.

Choose Safe early commits, advance to Wound, and inspect the public wiki. Then reset with Batch rollback. Finally try Unsafe order: the same interruption preserves a broken reference rather than useful progress.

Concurrency lab / a scripted example

Interrupt the agent. Inspect what survives.

Same interruption, different commit boundaries. No model calls, real locks, or performance measurements.

Choose a publication policy
  1. 1Ready
  2. 2Read
  3. 3First edit
  4. 4Wound
  5. 5Resume
  6. 6Finish
Agent B / younger batchReadyContested lock: None
Public wiki state
Incident pageAbsent
Index → incidentAbsent
Reference invariant holdsEvery published index link has an existing target.

Two batches share a wiki. Agent A arrived first; Agent B is younger. Advance to follow B’s work.

Earlier edits retained after wound: 0Prepared edits discarded: 0
Only B’s two-edit task is counted. A’s work is omitted. A wound means interruption, not proof of a detected deadlock. Switch to Unsafe order and advance to First edit to see why per-page atomicity is insufficient.

The example is intentionally small. It is a deterministic teaching sequence, not a Chroma implementation, distributed lock simulator, or performance benchmark. It counts only B’s two-edit task. The article remains readable without running it.

The thing you are protecting may not be the whole batch

A database transaction protects a defined set of invariants across its reads and writes. If a serializable transaction fails, blindly retrying its last statement can violate the assumptions that produced that statement. PostgreSQL explicitly requires retrying the complete transaction, including the decision logic that selected the statements and values. See serialization failure handling.

For an agent, that decision logic may include searches, document inspection, and expensive model calls. A retry can repeat discovery, not merely replay a short deterministic function.

That does not make optimistic concurrency control obsolete. Low-contention workloads, short commit phases, cached proposals, and targeted revalidation can still make it a sensible choice. Traditional transactions are not universally cheap either. The narrower observation is that long, contested reasoning batches make discarding an attempt especially painful.

Before choosing a protocol, split the work into three categories:

  • Evidence: retrieved sources, versions, and provenance. Often reusable, but potentially stale.
  • Proposal: a candidate edit derived from that evidence. Reuse requires checking its assumptions.
  • Published state: material readers can now rely on. It must satisfy the promised invariant immediately.

If all three live only inside a disposable agent context, every interruption looks like a full restart. Persisting them separately creates more recovery choices, even if the final publication still uses an ordinary transaction.

Fission keeps the page atomic, not the wiki update

In Chroma’s description, agents acquire exclusive locks when they first read pages, not when search returns possible matches. Individual page writes use the underlying storage transactions. A batch can publish several pages before its overall work is finished.

When a batch is wounded, already committed page edits stay. Locks are released, and a retry works from the state it finds, retaining the original arrival timestamp. That is not rollback with a clever name. It deliberately gives up all-or-nothing publication across the batch.

This distinction also explains why a transactional primitive remains useful. A logical wiki page may occupy several stored chunks. Readers should not receive half of its replacement merely because the higher-level batch permits partial progress.

The guarantee has moved to a smaller boundary. It has not vanished.

Nor has it become a guarantee that the prose is correct. Atomic publication can publish a false claim perfectly. Provenance checks, permission checks, and content validation remain application responsibilities.

Wound-wait is an ordering rule, not hindsight

Consider a total order over batches, with a stable tie-breaker. Older A requests a lock held by younger B: A wounds B. Younger B requests a lock held by older A: B waits. The Linux kernel’s wound-wait design documentation is a useful primary reference for the ordering principle and keeping age across retries.

The rule acts on conflicting lock requests. It does not need to wait until a complete deadlock cycle has been detected. After the required preemption and release, remaining waits point in one age direction, preventing a circular wait under the protocol’s assumptions.

Those assumptions matter in a distributed deployment. “Please stop” is not the same as “this process can no longer write.” A worker can pause, lose contact with coordination, and wake later with stale authority.

My implementation requirement would be a storage-enforced ownership check or fencing mechanism that makes writes from revoked owners fail. The mechanism must make checking authority and accepting the mutation an indivisible decision at the relevant boundary. A worker-side boolean called cancelled is not enough.

This is an application design requirement, not a claim about undocumented Chroma internals. The source describes fate sharing between its prototype’s job and lock manager; it does not justify inventing a general recovery guarantee for every possible deployment. For a deeper look at stale owners, see fencing writers at the storage boundary.

Prefix safety: interrupt after any edit

Let S0 be the initial wiki and e1 … en a sequence of published edits. Let I(S) mean that state S satisfies the invariant promised to readers.

text3 lines
Sk = apply(ek, Sk-1)

Prefix safety requires I(Sk) for every k from 0 to n.

This is stronger than checking only the final state. It is also relative to the chosen invariant. A wiki with a missing index link may be valid but less discoverable. A policy wiki containing incompatible access instructions may be dangerous even if every link resolves.

For the lab, the invariant is deliberately narrow: every published index link has an existing target.

The safe order publishes the incident page first and the index link second. If interrupted between them, the page exists without the new navigation entry. The unsafe order publishes the link first. An interruption preserves a dangling reference.

Neither example requires malformed writes. Each individual write can be atomic. Atomic edits do not automatically compose into a safe sequence.

Now replace the incident example with two pages that describe who can access customer exports. If updating one before the other can mislead readers into granting access incorrectly, do not rely on “the model usually chooses a sensible order.” Publish a versioned bundle behind one activation pointer, keep a multi-record transaction, or route the change through review. The right boundary follows the invariant, not the convenience of page-sized storage.

Keep progress without keeping mistakes forever

Preserving committed work is useful only if the work remains valid. “Already applied” should not be inferred from similar-looking prose alone.

A practical edit record might include:

json8 lines
{
  "operation_id": "batch-42:incident-17:timeline",
  "page_id": "incident-17",
  "source_ids": ["trace-301", "review-88"],
  "expected_revision": 12,
  "result_revision": 13,
  "validation": "accepted"
}

This is an illustrative application record, not a Chroma API payload. A production design must define which component can mark an edit accepted and which fields are immutable.

On retry, use the operation identity to look for a durable receipt, inspect current content and permissions, then decide whether to skip, repair, or generate a new proposal. Store the receipt atomically with the accepted mutation where possible; otherwise explicitly handle the gap between the mutation and its receipt. An LLM remembering that it wrote something is not a commit record.

If a previously accepted fact is later disproved, a corrective edit is legitimate. “No rollback on interruption” should not mean “no correction after new evidence.” Retractions need provenance and evaluation just as additions do. This is one reason a free-form wiki is not automatically a conflict-free replicated data type: merging arbitrary prose has no general algebra that preserves its meaning.

Goodput is useful, but count accepted work

Chroma frames the problem in terms of retaining paid reasoning. That is a valuable operational lens, provided retained output is not confused with useful output.

For a controlled experiment, I would define an application metric such as:

text1 line
accepted-edit efficiency = validated, retained edits / model cost

Track it beside task completion and regression rate. Otherwise, an agent can appear efficient by publishing many tiny, low-value updates or retaining incorrect conclusions.

Measure the tail as well as the average: longest lock hold, oldest unfinished batch, repeated wounds, time waiting on read-only dependencies, and resources spent repairing earlier edits. An exclusive read lock protects reasoning context but can serialize work on popular pages. A heavily referenced overview page can become a bottleneck even when most agents never intend to modify it.

Possible responses include smaller independent pages, an append-only evidence area, shorter critical sections with revalidation, or lower concurrency around hot resources. Each changes either coordination cost or the guarantees the application must check. “Add more workers” is not a contention strategy.

Failure policy: what happens next?

EventRequired application decision
Worker is woundedRevoke write authority, release ownership safely, preserve only already committed progress.
Lock ownership becomes uncertainReject writes until ownership is established; do not let a stale worker guess.
Commit response is lostLook up the operation receipt before repeating the mutation.
A search result is staleRead the authoritative page under the chosen concurrency policy before relying on it.
A partial update breaks a cross-page invariantStop early publication; use a larger atomic boundary or staged activation.
Source access is revokedRevalidate permissions and derived content; a past successful read is not permanent authority.
Retry budget is exhaustedLeave an explicit incomplete task with an owner; do not report success because some pages changed.

The policies above are proposed safeguards, not measured behavior of Foundation. They complement durable agent execution, which addresses a different question: how the orchestration itself survives interruption.

The decision I would make before building a swarm

Ask the product owner to name an acceptable halfway state.

For a knowledge ingestion system, “the new evidence page exists, but some navigation links are missing” might be acceptable. “The summary claims a policy is approved while its source says pending” probably is not. For money movement, inventory allocation, or access revocation, independently committing each step can be outright wrong unless the surrounding process handles the intermediate states explicitly.

Fission is interesting because it makes that trade-off explicit. It does not remove coordination. It chooses to preserve certain kinds of progress while reducing the scope of atomicity.

The next swarm I would evaluate would have an interruption test before a scale test: pause after every publication, kill the worker, retry with fresh context, and inspect both the visible state and the durable receipts. If the application cannot explain those states, a faster model will only produce them more quickly.

For related but distinct designs, read Kiro Flock’s shared-state coordination and database branches as agent sandboxes. Shared coordination, disposable environments, and safe incremental publication solve different parts of the same operational problem.

Sources and scope

The lab is a tested presentation model. It does not validate a production locking protocol, distributed failure detector, or semantic correctness of model-generated edits.

>