# RUN-003 evidence pack — idempotent and reconcilable side effects

## Work-package charter

- **Task ID:** `RUN-003`
- **Owner:** Oshun common run-protocol effect-delivery maintainer;
  implementation owner: Codex
- **Accountable product:** Shared V1–V9 autonomous-content platform
- **Priority:** P0
- **Target milestone:** 2026-07-15 SOTA gap-closure program
- **Risk tier:** Critical mutation-deduplication, provider-ambiguity,
  publication-integrity, concurrency, ownership, and recovery risk
- **Dependencies:** RUN-001 immutable aggregate and artifact lineage; RUN-002
  durable append, atomic attempt allocation, replay, and shared-filesystem
  reference adapter; provider-specific idempotency and lookup guarantees
- **Affected schemas and runtime:** Deterministic effect identity, provider
  delivery policies, hash-chained effect journal, effect reducer and storage
  interface, provider coordinator, file-backed reference adapter, root/
  namespace/subpath exports, and focused CI
- **Rollout strategy:** Treat the portable effect protocol and file adapter as
  the RUN-004 conformance target. A product adapter may declare only the
  delivery guarantees its provider actually offers, and an ambiguous effect
  fails closed unless provider reconciliation makes retry safe.
- **Rollback strategy:** Revert the effect protocol, tests, file-store
  extension, exports, CI expansion, audit qualification, and ledger closure
  together. No product pipeline or database migration consumes this protocol
  yet, so RUN-003 has no product-data rollback.
- **Evidence-pack location:** This file

## Deterministic identity and ownership protection

`deriveDurableEffectIdentity` binds the delivery key to tenant, immutable run
identity, plan and step, durably allocated attempt ID and ordinal, tool,
provider target and operation, and sorted artifact lineage with content and
parent hashes. Tests vary the run, step, attempt, tool, target, and artifact
content independently and prove that each changes the effect key.

Owner scope and mutation-class labels are deliberately excluded from the
delivery key but included in the complete immutable effect-identity digest. This
makes an owner-controlled relabel collide with the existing delivery identity
instead of minting a second provider key. The store rejects that collision as an
idempotency conflict. It also checks the effect against the registered run's
tenant and immutable identity, a real plan/step, the exact store-allocated
attempt tuple, the provider policy, and current artifact hashes and parent
lineage before it accepts an intent.

The reference adapter cannot register a different immutable run under an
existing stable run/thread identity. A changed run-identity digest is rejected
before a new effect journal is created. Legitimate new attempts, targets, or
artifact revisions receive new keys because they represent different mutations.

## Durable journal and provider ordering

Every effect record contains immutable identity and provider policy plus a
strict, hash-chained journal. Events carry a stable event ID, effect key,
contiguous revision, previous-event digest, monotonic timestamp, idempotency
key, canonical detail digest, and event digest. Replay rejects a broken chain,
duplicate event or idempotency identity, time regression, illegal transition,
noncontiguous delivery attempt, mismatched retry authority, or changed effect
identity.

The legal lifecycle is:

1. persist immutable intent and request digest;
2. durably claim a delivery attempt with worker and lease identity;
3. call the provider with the exact effect key and any reconciled completed/
   pending artifact set;
4. persist the authoritative provider receipt and response digest, or persist an
   explicit unknown outcome;
5. reconcile unknown, partial, or expired-dispatch state with provider truth;
6. authorize retry only from the immediately preceding authoritative
   `not-applied` or resumable `partial` reconciliation; otherwise succeed, fail,
   wait, or block.

The provider callback observes the journal in `dispatching` state, proving both
intent and claim were committed before invocation. Applied, partial, and
rejected execution results require an authoritative receipt. Determinate lookup
results require a receipt, partial results require pending artifacts, and no
artifact may be both completed and pending.

## Reconciliation and safe delivery modes

Provider policy declares `exactly-once-key`, `idempotent-retry`,
`reconcile-before-retry`, or `at-most-once`, plus lookup, partial-resume, and
idempotency-field support. Keyed guarantees require an explicit provider key
field; partial resume is valid only for a keyed idempotent policy.

Duplicate coordinator calls return terminal durable state without reinvoking the
provider. A lost acknowledgment or timeout is first recorded as unknown. The
next coordinator call must query the provider before it can proceed:

- `applied` becomes durable success without redelivery;
- `not-applied` creates retry authority and reuses the same effect key;
- `partial` can resume only under a keyed policy that declares partial-resume
  support, with the reconciled completed and pending artifact sets passed to the
  adapter;
- `in-progress` remains non-deliverable and will be reconciled again;
- `unknown`, failed lookup, absent lookup support, or unsafe partial semantics
  blocks delivery instead of guessing.

An active durable worker lease prevents a concurrent worker from invoking the
provider. After lease expiry, a failover worker reconciles provider truth before
claiming the next contiguous delivery attempt. Optimistic revision checks and
the shared-filesystem lock give one durable claim winner across independently
constructed adapter instances.

Exactly-once behavior therefore depends on an authoritative provider honoring
the supplied key. The conformance mock proves that the coordinator supplies one
stable key and does not repeat a completed delivery; it is not evidence that a
real publication provider offers exactly-once semantics. When authoritative
provider truth cannot establish a safe path, the coordinator fails closed.

## Reference-adapter persistence boundary

`FileDurableRunStore` now implements both the RUN-002 run store and the RUN-003
effect store. Effect intents and journal events use the same locked, validated,
temporary-write, file-fsync, atomic-rename, and directory-fsync transaction
machinery as run events. Loading the store validates every effect identity and
journal chain before returning data.

The effect journal is a separate record within the same file state, not an
additional RUN-002 aggregate state event and not a database transaction spanning
a real remote provider. The write-before-call barrier closes the local crash
window before provider invocation; provider lookup and idempotency keys address
the unavoidable ambiguity after invocation and before result persistence.

## Automated scenario matrix

| Scenario                      | Conformance result                                                                                |
| ----------------------------- | ------------------------------------------------------------------------------------------------- |
| Required key tuple            | Run, step, attempt ID/ordinal, tool, target, and artifact-lineage variants produce distinct keys. |
| Owner/run reuse               | Owner or mutation relabel collides and is rejected; changed immutable run identity is rejected.   |
| Intent/result ordering        | Provider observes durable dispatch; authoritative result follows in the journal.                  |
| Duplicate delivery            | Repeated coordinator calls invoke the exactly-once mock once with one key.                        |
| Lost acknowledgment + restart | A new adapter instance reconciles `applied` and never redelivers.                                 |
| Timeout                       | Retry occurs only after authoritative `not-applied`, using the original key.                      |
| Partial success               | Reconciled completed/pending artifacts reach the keyed resume attempt.                            |
| Unsafe ambiguity              | No lookup, failed lookup, or unsupported partial resume produces a durable block.                 |
| Worker crash/failover         | Expired claim is reconciled before failover delivery attempt two.                                 |
| Concurrent workers            | Two adapter instances produce exactly one provider invocation and one durable success.            |
| Retry-authority integrity     | A reason not matching the immediately preceding reconciliation is rejected.                       |
| Restart/idempotency integrity | Journal digests survive adapter restart; conflicting event-key reuse is rejected.                 |

## Artifact identity

| Artifact                                                          | SHA-256                                                            |
| ----------------------------------------------------------------- | ------------------------------------------------------------------ |
| `libs/contracts/src/run-protocol/durable-effect-protocol.ts`      | `5f5e90061109a07140123e77efaa8dbc05642172f8bae8d03b95f9e17447271b` |
| `libs/contracts/src/run-protocol/durable-effect-protocol.spec.ts` | `dfd180f873b1e415e97ffa6b08e058c9627d319e8ab2d3b1d9bf5c788ce38af2` |
| `libs/contracts/src/run-protocol/file-durable-run-store.ts`       | `670cab4b3df8c39cb2a4f35f0b815b7a4042f3f5b95333a6b641fc4f0b61f0c4` |
| `libs/contracts/src/run-protocol/index.ts`                        | `cca361da05470e63afc1529f3180f93fa113689175179da3a491dd84da4f7673` |
| `libs/contracts/src/index.ts`                                     | `ee9f0f22297c7a4b29243221bf15e31290d6df75e479bcfe1bd7b83e39150f1d` |
| `package.json`                                                    | `3cf507687fef0e18b9168c23f785e19dbfa453f88447eacd7a8762e8d7652917` |
| `.github/workflows/run-protocol-contract.yml`                     | `930c1394f7b84d9610f2681c54f8a58a26b91cfb6ab841a61204ff3339c0b3a8` |

## Automated and adversarial verification

- `pnpm verify:contracts:run-protocol` passes targeted ESLint with zero
  warnings, **45/45** RUN-001/RUN-002/RUN-003 Vitest cases with one worker, and
  contracts source typechecking. RUN-003 contributes 12 focused cases.
- The RUN-003 suite covers all requested duplicate-delivery, lost-
  acknowledgment, timeout, partial-success, restart, failover, and
  concurrent-worker cases, plus key derivation, owner relabeling, durable
  ordering, unsafe-provider blocking, retry-authority integrity, and event
  idempotency conflicts.
- Existing common durability compatibility remains green at **54/54**
  `durable-job-orchestration` and `job-durability-recovery` cases.
- `pnpm verify:contracts:content-slot` remains green at **17/17**, including
  lint and contracts typechecking, because effect ownership validates the
  RUN-001 content and artifact aggregate.
- `NX_DAEMON=false pnpm exec nx run @oshun/contracts:build --parallel=1` passes.
  Runtime probes verify the effect schema version, coordinator, identity
  derivation, and provider-result schema through the root flat export,
  `RunProtocolContracts` namespace, and `@oshun/contracts/run-protocol` subpath.
- The build ran alone after the required resource gate. Available RAM remained
  10–11 GiB, swap had about 30 GiB free, and disk had 38 GiB free.
- Source pre-commit ESLint, Prettier, stub-indicator, asset locking, and V5
  documentation checks passed. The prohibited broad workspace typecheck was
  bypassed with `SKIP_TYPECHECK=1` only after focused contracts typechecking,
  tests, and the resource-gated build passed.

## Negative proof and task boundary

- No V1–V9 product pipeline calls the coordinator, no product database
  implements the store, no migration persists effect records, and no deployed
  worker or publication provider has been exercised. RUN-004 owns product and
  database adapters; RUN-005 owns scheduler, approval, and runtime lifecycle
  integration.
- The exactly-once provider is a deterministic conformance mock. There is no
  live provider receipt, placement observation, publication, rollback, canary,
  regional failover, or duplicate-delivery drill against production.
- The file adapter is a shared-filesystem reference implementation. It does not
  prove multi-host consensus, database isolation, network partitions, regional
  high availability, backup restore, encryption, key rotation, or production
  throughput.
- The provider call and local result append cannot be one atomic transaction.
  RUN-003 makes that boundary explicit and reconciles ambiguity, but correctness
  still relies on a provider truth surface and its declared key semantics.
- The fixture is synthetic contract/reference-adapter proof, not a product run,
  real artifact release, tool mutation, legal record, learning event, or
  operational recovery incident.
- No user-facing UI, mobile flow, media/model pipeline, database migration, or
  Unreal asset changed. Playwright/mobile/browser accessibility,
  provider/media/database, and Unreal execution are therefore inapplicable to
  RUN-003.

## Publication identity

- **Implementation and first remote-integrated tested source commit:**
  `aee2879220a22a0bc4402c5828a13acb135059a3`
- **Publication metadata commit:** `585ee7acd82e529ff54073ddced526e73e133f76`
