ADR-0088: Eve operation-class reliability policy

• Status: Accepted
• Date: 2026-09-14
• Decision owner: SRE Lead
• Independent verifier: Reliability QA

Context#

Eve already had sound local mechanisms: bounded retries and a circuit breaker in @oshun/resilience, abort-aware assistant turns, retained confirmation outcomes, fenced work-item leases, resumable ledgers, and idempotent watcher deliveries. They did not form one total policy. A caller could select transport defaults without first classifying whether the operation was a read, a confirmed write, leased work, delivery, or verification. In particular, cancelling an already dispatched write can make its outcome unobservable even when it prevents no side effect.

Task 13.3 requires one contract for deadline, attempt timeout, retry/backoff and jitter, circuit breaking, backpressure, concurrency, idempotency/fencing, cancellation, and resumability. It inherits Task 13.1's fixed SLO and recovery budgets.

Decision#

1. @oshun/resilience's eve.operation-reliability.v2 is the canonical policy. Every Eve operation is classified as interactive-turn, read-only-dependency, confirmed-mutation, durable-leased-work, watcher-delivery, or artifact-verification before dispatch. A new class requires a versioned policy edit and direct verification evidence.
2. Every class has an explicit value for all ten Task 13.3 dimensions. A mechanism that is meaningless at that boundary is not-applicable with a reason; omission and inherited transport defaults are invalid.
3. Deadlines bound the whole operation and attempt timeouts fit inside them. Retries count the first attempt, are capped at five, name retryable failure classes, and use capped exponential backoff with the class-selected jitter mode. Read dependencies use full jitter; the deterministic watcher outbox uses no jitter so its persisted retry instant is replay-stable. Mutation is single-attempt unless its domain owns an independently durable idempotency receipt. Durable leased work has no automatic retry loop: each zero-delay re-acquisition is an explicit queue decision that advances its fencing token and acquisition count.
4. Interactive, read-dependency, and confirmed-mutation calls use the canonical circuit breaker. Leased work uses lease expiry, a bounded acquisition budget, and quarantine; its remote calls are separately classified as dependency operations. Watcher delivery uses its durable outbox, capped retry budget, and dead letter as the failure-isolation boundary; local deterministic artifact verification has no remote circuit. Every class has finite concurrency and finite backpressure; cloud concurrency always requires a human spend approval. Local leased work and verification remain concurrency one.
5. Every effecting operation must possess a nonempty idempotency key and stable outcome reference before dispatch. Leased work also presents its current monotonic fence. A confirmation action's one-shot claim and artifact verification's immutable artifact-version plus verifier-version run key are their exclusive grants, so separate fencing tokens are not applicable.
6. A disconnect before dispatch stops work. A read already in flight may be discarded. An effect already dispatched is not represented as cancelled: it continues to an outcome receipt, or durable work returns to its checkpoint/outbox under the same identity. admitEveOperation fails closed if effecting work lacks the identity needed to find that outcome. This is the normative "no unobservable side effect" rule.
7. AssistantActionConfirmBridge applies the guard to confirmed mutations and retains the action outcome for same-identity replay. Its finite permit gate rejects a ninth concurrent local mutation; its circuit isolates repeated adapter failure; and an attempt timeout returns a pending outcome reference while cooperative cancellation and final settlement continue updating that same receipt. The assistant runner applies a bounded signal and capacity gate, and its disconnect authority prevents later model, tool, confirmation, and turn writes. The retrieval client applies admission, finite capacity, per-attempt timeout, retry/backoff/jitter, and circuit breaking. Queue semantics own bounded acquisitions, spend-gated concurrency, idempotency, fencing, quarantine, and ledger replay. Lease-bound triage is admitted only from the current holder with the current fence; an expired lease grants no authority. Exact accepted-key replay is resolved before fresh-request authority checks so the original observable result survives lease release, expiry, or successor acquisition. Triage of an unleased item remains an observable failure fact available to any actor. A tenant-scoped transactional admission lock refuses new ready work at the durable backlog cap while reserving requeue capacity for already-active leases. Watcher state owns capped retries, stable delivery keys, dead letters, and pause/resume/cancel. Artifact verification admits the artifact-version plus verifier-version identity under concurrency one, cooperatively observes persistence boundaries before honoring timeout, retries only classified transient I/O, and installs idempotent per-item checkpoints before a terminal run-complete marker.
8. The contract verifier owns matrix totality and source bindings. Its negative control weakens a mutation to cancel-after-dispatch and must fail before the task can close.

Consequences#

• Reliability behavior is selected by semantic operation class rather than by whichever client happens to be used.
• A timeout is not permission to retry a write. Retry requires both policy allowance and the operation's stable replay identity.
• The policy is a maximum envelope. Domain owners may use shorter deadlines, fewer attempts, or lower concurrency, but weakening identity, fencing, or outcome observability requires a new decision and evidence.
• Confirmation outcomes are retained in process today. A process loss after an external mutation can exceed that lookup window; durable cross-restart receipts remain a limitation for the affected domain adapters and are not misrepresented as proven here.
• Load/soak capacity, destructive failure exercises, restoration measurement, production attainment, alert delivery, and game days remain Tasks 13.4–13.7.