- Owner: Codex implementation agent
- Accountable product: V6
- Priority: P0 product acceptance slice
- Linked task: V6-001, replace in-process cognition state and preserve its execution and authority limits across restart and concurrent workers
- Source commits:
- durable kill/quarantine control plane:
d33a98f2cf35549c1eedcfbdf38906ddaabdd7da - durable run, budget, failure, and child state:
98ebbf320a11dd40cee3fad479a62e5811dd2461 - durable post-loss operator recovery:
50a3a76031f41f8485434694f7a887ae47254659
- durable kill/quarantine control plane:
- Evidence captured: 2026-07-24
Implemented boundary#
The canonical createDurableAgentRunRuntime composes AgentRunManager,
DurableAgentRunLedger, DurableAgentControlPlane, and
DurableKillSwitchRegistry over one RUN-002 DurableRunStore. Every cognition
run is projected into a RUN-001 common aggregate and checkpointed before the
first provider call, after every iteration and tool boundary, and at terminal
success or failure. Provider failures persist a failed terminal state with a
hashed error receipt rather than a raw provider body.
The common aggregate retains reconstructible bindings for agent family, provider, each tool grant, token/cost/call budgets, maximum iterations, per-call token ceiling, and anomaly thresholds. Fresh managers can therefore reserve and start a child from a durable parent that is absent from local memory. The ledger validates tenant ownership, refuses authority widening, requires bounded children under cost-capped parents, and uses optimistic RUN-002 transitions so concurrent workers cannot both oversubscribe the same remaining parent budget. Reservation and settlement are idempotent, and stale parent checkpoints merge authoritative remote child state instead of erasing it.
Kill switches remain scoped to agent, family, tenant, or provider. Kill activation/release and anomaly quarantine/release are durable control-thread interventions, so a newly constructed worker refuses the affected run before calling the provider. The run loop also polls the durable switch at each iteration boundary.
The production Ori Presence BFF no longer constructs an in-memory-only
AgentRunManager. It mounts the canonical durable runtime with
FileDurableRunStore and fails closed when OSHUN_V10_ORI_RUN_STORE_PATH is
absent. The Hetzner deployment supplies a dedicated BFF run-state volume and
includes it in the off-box backup inputs.
The runtime also composes DurableAgentRunOperator over that exact store. Its
pause, claim, takeover, plan-step edit, approval, and resume commands commit the
RUN-001 aggregate, optimistic run head, RUN-005 lifecycle, HITL-003 intervention
journal, claim lease, approval invalidation, and receipt in the file store's
single lock/fsync/rename transaction. A new process derives the same stable
thread, reconstructs the current plan and lifecycle, and continues from the
committed revisions instead of any local object.
The production BFF exposes the recovery authority at
/v1/admin/rail/ori/cognition-runs/:runId and its pause, claim, takeover,
edit-step, approve, and resume mutations. The BFF never accepts tenant or
actor identity from the payload: tenant comes from the validated request context
and actor comes from the authenticated principal. Only admin:*,
admin:studio, or admin:workspace:model can enter the surface. Every mutation
requires a caller-stable command ID, stable timestamp, and rationale; future
timestamps are bounded, payloads are strict, and reads return control metadata
rather than raw prompts or provider bodies. Missing durable authority, missing
tenant context, scope failure, and tenant-header mismatch all fail closed.
Adversarial and restart coverage#
The owning tests prove:
- an executing checkpoint exists before the first provider invocation;
- terminal completion, immutable limits, authority, and usage can be resumed from a new store instance;
- a fresh manager can trace and execute a child against a durable parent that it has never held in process;
- concurrent workers permit only one of two oversubscribing child reservations;
- a stale parent write cannot erase a remote child reservation;
- widened child tools are rejected after restart;
- repeated settlement is idempotent;
- provider failure persists
failedwithout retaining the raw error body; - tenant kill and quarantine survive worker reconstruction and prevent provider execution until durable release; and
- missing production run-store configuration fails closed;
- a fresh BFF and fresh file-store instance can inspect, pause, take over, version-edit, reapprove, resume, and inspect the same run after the first process closes;
- editing creates a human-authored contiguous plan version, abandons the stale claim, and invalidates the dependency-bound approval;
- resume before renewed approval is rejected, then succeeds after approval;
- stable command retries return the original receipt without a second event;
- concurrent recovery workers serialize competing claims so exactly one wins;
- terminal durable runs cannot be bootstrapped back into executing state; and
- operator scope, tenant mismatch, and absent production authority are rejected at the HTTP boundary.
Verification#
| Command | Result |
|---|---|
pnpm exec vitest run --config libs/iris/agents/core/vite.config.ts |
PASS — 16 files, 223 tests |
pnpm exec tsc -p libs/iris/agents/core/tsconfig.lib.json --pretty false --noEmit |
PASS; repeated after both rebases |
pnpm exec vitest run --config libs/v6/moirai-kernel/vitest.config.ts |
PASS — 2 files, 15 tests |
pnpm exec vitest run --config testing/prompt-injection/vitest.config.ts |
PASS — 5 files, 232 tests |
| Focused BFF operator route and Ori runtime tests | PASS — 2 files, 7 tests; repeated after rebase |
| Focused transactional-intervention contract | PASS — 1 file, 13 tests |
pnpm exec vitest run --config libs/contracts/vitest.config.ts |
PASS — 258 files, 5,094 tests |
| Direct contracts, Iris core, and BFF TypeScript checks | PASS |
node scripts/build.mjs from apps/oshun/bff |
PASS — 34.6 MB production server bundle |
docker compose -f infra/hetzner/docker-compose.yml config --no-interpolate --quiet |
PASS |
| Docs render, freshness, integrity, Python, and Playwright gates | PASS — 3,138 files; 55 Python; 188 browser pass / 16 skip |
| Strict ESLint over all changed TypeScript files | PASS — zero errors/warnings after ignored-file notices were suppressed |
Prettier and git diff --check |
PASS |
The first full contracts run exposed an unrelated repository-baseline omission: the alignment ratchet did not classify eight newly landed governance contracts. The ratchet now explicitly records that pending ownership debt, its focused 5-test suite passes, and the repeated full contracts run passes all 5,094 tests. No V6 implementation imports those governance contracts.
The BFF ratcheted typecheck completed but reported four unrelated app-own errors
in admin-isis-cli-surface-route.test.ts,
autonomous-content-metrics-store.ts, autonomy-route.test.ts, and
tenant-content-graph-store.spec.ts; no changed file was reported. The Iris Nx
build reached its unchanged tracing dependency and stopped on five
implicit-any diagnostics in libs/shared/tracing/src/genai.ts; the owning
Iris TypeScript check is green. Nx expanded the BFF build into a 332-project
dependency matrix, so it was stopped under the repository's small-host safety
rule and replaced with the BFF's direct production build script.
Artifact hashes#
| Artifact | SHA-256 |
|---|---|
libs/iris/agents/core/src/agentic/agent-run.ts |
1c8d89645d9681a1eca5198b9702e6f55589a61df4332654e8e94cc0e7eae2ed |
libs/iris/agents/core/src/agentic/common-run-adapter.ts |
4529dac247a2f1a19ff05bfaadaac4e33eb4c3632715464510b70a80c97e1ff9 |
libs/iris/agents/core/src/agentic/durable-control-plane.ts |
571441997aea1bfba693a890c41bee80e465fc43a10e11ea7968c04f0c5f474f |
libs/iris/agents/core/src/agentic/durable-run-ledger.ts |
642f0c84272bfe93e4855478b887b0eb52d83595b4e0d920dc03225d4b254648 |
libs/iris/agents/core/src/agentic/durable-runtime.ts |
10980fab1c5fac1c9130a1142f1124edf3b3ef44466e599fda95380fa96b3ee6 |
libs/iris/agents/core/src/agentic/durable-operator.ts |
94de5b28391d6fbe8a4b8ce5874d8c0a3c2ea6658821edeb801cb369c66ed98a |
libs/iris/agents/core/src/agentic/durable-operator.spec.ts |
f250f8882b10c9dda392296e82d64be4b419c90d0849b86450998a3583c219f9 |
libs/contracts/src/run-protocol/transactional-intervention.ts |
b8c4cbef8c3b4ffaab0e76eea949272565c6152081518ae39d80a39776d166f5 |
libs/contracts/src/run-protocol/transactional-intervention.spec.ts |
bc6d9b28988474005eadde8a56b7b5dfc26093546d346f6f553c83569ff0de93 |
libs/iris/agents/core/src/agentic/durable-control-plane.spec.ts |
3b6c3318a03051782ac28a9d707c6777671ad165caffca14849dfd626f3d85dd3 |
libs/iris/agents/core/src/agentic/durable-run-ledger.spec.ts |
e9dfeef7e71d9359f5a83968a7e68cd9f6a7102540728f3994bbd95bb58a5bd3 |
apps/oshun/bff/src/routes/v10-ori-cognition-operator.ts |
af0051ddc0c2f8666178d1740cb7098ed1c4f6bcfb2bc8e4affc64b77d46fe89 |
apps/oshun/bff/src/routes/v10-ori-cognition-operator.spec.ts |
9a04eaf7d89d53810c7d6c392198781b0474fba279f8bdcb3f8eb103162fef8e |
apps/oshun/bff/src/v10/ori-presence-runtime.ts |
e6f5eb5952a5fb8089701779bc2d7ea880c10a9a93a2e03cadd6697003c880e9 |
apps/oshun/bff/src/v10/ori-presence-runtime.test.ts |
947154edaea5671cab78e6c23f7f0ece9ccc5dba0c03421d89ddc41e4bfb7b44 |
infra/hetzner/docker-compose.yml |
1c1e204d939583e933c28ac0644313490c48ac025e9e4085cc463a358c378de4 |
V6/V6_ARCHITECTURE.md |
780997a6d56b50b1d054642bc5e1e5006f71612240b89b3aad5eda09011ca290 |
Honest boundary#
This evidence closes durable cognition state, preservation of execution and authority limits, and the authenticated post-loss operator control plane across restart and concurrent workers. “Resume” here means an atomic transition back to execution-eligible lifecycle state after dependency-bound approval; it does not claim that this evidence ran a live provider dispatcher after the transition. It also does not claim multi-host load testing, a live provider incident, restore-from-off-box rehearsal, a browser consumer for the new recovery routes, V6-002 live in-run co-authoring/mixed-lineage publication, or the V6-003 flagship slice.