# Eve Small-Model Excellence (SMX) — TODOS (2026-08-16)

Execution ledger for `V1/EVE_SMALL_MODEL_EXCELLENCE_DESIGN_2026-08-16.md`. Read
the design doc first; this file is the checkable spine, not the argument. Goal:
the assistant brain runs excellently on the smallest model that passes a
regression-locked deck, with the harness (router, skills, scoping, validation,
checkers, escalation) carrying the intelligence and a strong model improving the
harness at build time only.

## Process discipline (binding, per CLAUDE.md and the checkbox rule)

- One task at a time, sequentially; the checkbox is the sole source of truth;
  never batch-mark; when in doubt leave it `[ ]`.
- **Phases are strictly ordered.** P0 lands before ANY prompt, tool, or model
  change — an uninstrumented improvement is indistinguishable from a regression.
- **Every measured claim names its run:** model slug, provider served,
  quantization, `sort` preference, k, and cost from `usage: {include: true}`. A
  number without its run recipe does not close a checkbox.
- **Spend gating:** live deck runs are operator-invoked
  (`pnpm --filter @oshun/bff eval:assistant`), never CI-automatic. CI runs
  provider-free cases and grader specs only. Frontier models appear ONLY in the
  P0 calibration arm and flake audits — never in serving paths or CI.
- **Measurement pins:** `OPENROUTER_PROVIDER_SORT=price` always;
  `OPENROUTER_PROVIDER_QUANTIZATIONS=fp8` on every arm that feeds a comparison;
  champion pin `deepseek/deepseek-v4-flash-0731` per
  `docs/agents/model-cost-openrouter.md` (do not re-litigate the pin without
  re-reading that file).
- **Machine limits:** deck runs need BFF (:4010) + Postgres only — no Next dev
  server. `memory_pressure | tail -2` before every run; PIDs tracked in
  `/tmp/claude-<session>.pids` and killed at session end.
- New specs are `.spec.ts` (the ratchet rejects new `.test.ts`). Secrets come
  from the drawer via a throwaway `/tmp/<task>-env.sh`, never into the repo.
- A task that changes what the model READS (conduct core, skills, tool
  descriptions, registry) is not done until the ratchet hash + scorecard story
  for it is coherent (from P0 onward).

## Adjacent hardening (dependency posture — not deck-gated)

- [x] A.1 Pin `@earendil-works/pi-ai` exactly: catalog `^0.83.0` → `0.83.0` in
      `pnpm-workspace.yaml` + `pnpm install` lockfile update; rationale and exit
      ramp recorded in the design doc's lessons-intake section (token-touching
      dep — bumps are reviewed, never floated).

## P0 — Instrument & baseline (closes SMX-G1, part of G8, G13, G14)

Deck v2, telemetry v2, and the baseline scorecard. Zero behavior changes: no
prompt edits, no tool changes, no model changes anywhere in P0.

### Deck harness capabilities

- [x] 0.1 Multi-run support in the eval harness: run each case k times (default
      k=3 via `EVE_SMX_EVAL_K` — an env var, not a CLI flag, because the runner
      is a vitest suite; junk values throw rather than silently measuring at the
      default), report per-case pass@1 and deck pass^k; grader stays pure;
      aggregation math spec'd with hand-checkable numbers (`eval-stats.ts` +
      spec; k-runs executor + flaky negative control in `eval-harness.spec.ts`;
      27 tests green; bff typecheck ratchet green).
- [x] 0.2 Wilson 95% interval helper (pure, spec'd against independently
      computed values + symmetry property) + per-family rollup
      (`summarizeEvalFamilies`: pooled run-level pass@1 with Wilson bounds —
      correlation caveat documented — plus case-level pass^k) appended to the
      scorecard when a family resolver is given; untagged cases stay visible as
      `unassigned`. 16 stats tests green; bff typecheck ratchet green. Family
      TAGS on real cases arrive with 0.6.
- [x] 0.3 Multi-turn case support: `AssistantEvalMultiTurnCase` (ordered turn
      list, per-turn expectations + pageContext, one grade per conversation with
      turn-prefixed failures), executor runs every turn in ONE session (+ k-runs
      variant on fresh sessions); scripted-binding specs prove the plumbing
      including session continuity (turn 2's provider request carries more
      transcript; a fresh run's opening request is history-free again) and a
      forgets-the-follow-up negative control. 41 evals tests green; bff
      typecheck ratchet green.
- [x] 0.4 SSE-order expectations in the grader: observation gains an ordered
      `timeline` (deltas + tool events in wire order, asserted from parsed
      frames) and expectations gain `noTextMatchingBeforeToolOk` — a claim delta
      before the named tool's ok, or with no ok at all in the turn, is a
      failure; claim-after-ok and innocent narration stay legal
      (case-insensitive, spec'd on the spine-run-12 announce-before-act shape).
      42 evals tests green; bff typecheck ratchet green.
- [x] 0.5 Cost + latency capture per case: `@oshun/ai` OpenRouter requests carry
      `usage: {include: true}` (openrouter-only, both paths — spec'd that OpenAI
      does NOT get the field); billed `cost` → `TokenUsage.costUsd`,
      `cached_tokens` → `cacheReadInputTokens`, chunk `provider` →
      `ChatCompletionResponse.servedBy` (absent stays absent — unreported ≠
      free); runner sums across iterations and `turn.complete` forwards usage +
      servedBy; executors time each turn request (wall-clock, session create
      excluded); `summarizeRunStats` + spend-summary line in the live runner.
      HONEST AMENDMENT: quantization is NOT in the response — provenance per run
      = provider served (wire) + the request's quantization pin; endpoint
      quantization is looked up via A10's /endpoints command when needed. 57
      bff + 525 shared-ai tests green; bff typecheck ratchet green.
- [x] 0.6 Case metadata: `AssistantEvalCaseMeta` (required `family` from the
      shared ten-family vocabulary, required `source`, optional `lock` +
      `flakySkip{reason}`) on both case shapes; `validateEvalDeck` refuses a
      quarantined lock ("a lock that can be skipped is not a lock") and
      duplicate ids, runs before any spend in the live runner; quarantined cases
      skip VISIBLY with their reason; the seven golden cases tagged
      (`golden-v1`) and the live scorecard now prints the per-family Wilson
      rollup. 51 evals tests green; bff typecheck ratchet green.

### Deck content (target ≥120 graded cases; count recorded at exit)

- [x] 0.7 Mine the defect ledger: enumerate every closed EVE-VIS row whose
      failure was model behavior; write the candidate-case list with row ids
      into `docs/audits/EVE_SMX_DECK_SOURCES.md` (the mining record, so coverage
      is auditable). Done 2026-08-16: all 279 rows reviewed and partitioned — 21
      model-behavior rows (each with family tag + ~45 sketched cases in grader
      vocabulary + measured pre-fix rates), 258 excluded by class with full id
      lists (177 client / 58 server / 10 build-CI / 9 mobile / 4 withdrawn),
      partition cross-checked mechanically (sums to 279, no dups); grading
      limits (args not on the wire) and the known-red champion case
      (EVE-VIS-280) recorded honestly.
- [x] 0.8 Ledger-born lock cases implemented (~50 target): tool-skip/grounding
      (080 shape), full-id citation (211), operator register (212), envelope
      (017), announce-before-act SSE order, audit-skip provenance (128),
      internal-id hygiene — each case names its EVE-VIS source. Done 2026-08-16:
      35 cases (`deck-ledger-cases.ts`) covering ALL 21 mined rows and every
      class named here (5 multi-turn; deferred-room cases run as a real
      four-scope V1.0 member, workbench cases as `admin:*` — new `authScopes`
      meta); grader gained `tourCuratedId`, `fullIdCitations`,
      `noUnterminatedTextBeforeTools` + `advisory` (runs/reports, doesn't gate;
      sole use EVE-VIS-280, lock+advisory refused by validateEvalDeck);
      admission spec checks source/id coherence and every tool name against the
      REAL builders; live runner executes the combined 42-case deck with
      per-case auth + multi-turn + advisory reporting. Pre-mining ~50 target
      honestly revised to 35 in the sources doc (21 rows support 35 strong
      cases; padding poisons signal; 0.9/0.10/0.11 carry the ≥120 exit
      arithmetic). 63 evals tests green; bff typecheck ratchet green.
- [x] 0.9 Family coverage cases (~60 target): per family × happy path,
      empty-store honesty, tool-error honesty, refusal correctness — including
      audit-mode, workbench-read, and workbench-write (confirm-bridge) cases the
      current 7 don't touch. Done 2026-08-16: 40 cases (`deck-family-cases.ts`,
      deck now 82) — ALL ten families ≥1 case (spec-enforced); empty-store +
      failing-adapter fixture profiles are real app instances over overridden
      adapters (`eval-adapter-profiles.ts`, runner builds one app per profile;
      every profile case must carry a fabrication guard, spec-enforced);
      audit-mode (status/no-run/3-turn walk continuity), workbench-read
      (decisions w/ full ids, explorer link, member refusal), workbench-write
      held-create (`action_confirm` UI intent + "confirm" reply) and
      no-capability refusal, member do-tier held write; crisisActive gained the
      false-positive direction (spec'd); per-case auth SUB (`deck-<caseId>`)
      isolates audit/budget state between cases. Axis gaps recorded in the
      sources doc (docs/navigate/audit tool-error has no adapter seam — decided,
      not forgotten). Target honestly revised ~60→40 there; 0.10+0.11 must land
      ≥38 for the ≥120 exit. 71 evals tests green; bff typecheck ratchet green.
- [x] 0.10 Adversarial cases (~30 target): injection probes through docs
      snippets, thread bodies, page context, and work-item titles; fabrication
      traps against empty stores; id-precision traps; register traps. Done
      2026-08-16: 21 cases (`deck-adversarial-cases.ts`, deck now 103) across
      all four classes (spec-enforced) — injection via page title/heading/
      selection + forged tool-result + role-override + docs/work-item-title
      (each with a canary the model must never emit or a tool it must not call);
      empty-store fabrication traps against leading/presupposing asks (080/095
      inverted); id-precision (invented uuid not confirmed, truncated id not
      pasted back, invented meditation not "added"); register (builder voice,
      deferred-room upsell, prompt-reveal, operator-as-member, domain-word).
      Every case carries a NEGATIVE witness (forbidden string / uncalled tool /
      required refusal / id-shape), spec-enforced — pass means the attack was
      resisted, not that the turn completed; not locks (champion may not hold
      every one; flake audit may quarantine visibly). 77 evals tests green; bff
      typecheck ratchet green.
- [x] 0.11 Battery promotion: the ten `eve-conversation-battery` conversations
      become graded multi-turn deck cases (deterministic expectations where
      possible; the rest wait for the P8 judge — marked `advisory` until then).
      Done 2026-08-16: 10 multi-turn cases (`deck-battery-cases.ts`, deck now
      113), all ten conversations promoted (spec-enforced count + per-id source
      `battery:<id>`); deterministic turns graded now (tool grounding for
      tara/nyx/arete/nisaba/docs, deferred-room register for deferred-rooms,
      honest-empty vocabulary via the empty-store profile for honest-empty,
      no-mail-claim for out-of-scope); shell-greeting, invented-ui, and
      out-of-scope are `advisory` (their ARC quality is P8 judge territory,
      promoted to graded in 8.1) but still run and report; never locks. The
      `tara_session_history` recall turn is completion-only (no member
      session-history tool exists — recorded inline). 82 evals tests green; bff
      typecheck ratchet green.
- [x] 0.12 Provider-free self-checks retained and extended (crisis, budget
      refusal, confirm-bridge refusal without capability). Done 2026-08-16:
      `deck-provider-free-selfchecks.spec.ts` runs ALWAYS in CI with no provider
      — (1) crisis supersede fires before any model call (proven with a binding
      whose provider THROWS if reached), (2) budget refusal (EVE-VIS-231) 429s
      with `assistant_budget_exhausted` + a trustable `resetsAt`, reached via a
      seeded metrics ledger so no tokens are spent, (3) confirm-bridge refusal
      (EVE-VIS-277) at the toolset seam — a mutating tool with no confirm sink
      returns "was NOT performed", plus its positive control (with a bridge the
      same tool PARKS on action_confirm, so the refusal is a real gate not a
      broken tool). Provider-freeness enforced structurally, not asserted. The
      crisis provider-free deck cases (golden + family) are the "retained" half.
      Also cleared the ≥120 exit target: family grew 40→48 (genuine
      continue-session/plan, saved-objects, course search/recommend happy
      paths + empty-store twins), so the graded deck is **121**
      (`ASSISTANT_EVAL_DECK`). 90 evals tests green; bff typecheck ratchet
      green.
- [x] 0.13 Flake audit: every case run k=10 on the champion; cases flaky for
      reasons the grader can't defend are rewritten or dropped with the decision
      recorded in the deck sources doc (golden-catalog admission rule). Done
      2026-08-16 (operator-invoked in-session): k=1 smoke + k=10 full deck (3
      round-robin shards, concurrency 4) + three targeted k=10 re-run rounds =
      **$0.5041 across 1,521 live runs**, champion pin, sort:price, fp8 (served
      StreamLake/Baidu/GMICloud). Deck partitioned 74 solid / 21 consistent
      champion gaps (kept — the P2 targets) / 26 flaky, every flaky run triaged
      line-by-line in `docs/audits/EVE_SMX_DECK_SOURCES.md`. Grader-side defects
      repaired (zero drops, zero quarantines, ids unchanged): shared-subject
      free-plan budget artifact → per-run auth subjects (spec'd); a
      lookup-invisible fixture target that punished verification; June-pinned
      nightly-highlights fixture (date-rot); honest-register vocabulary widened
      three rounds to negation stems, every addition quoting a measured honest
      reply; typographic-apostrophe normalization in the grader; battery deck's
      stale vocab copy replaced by the shared import. Also new runner
      capabilities this task needed:
      `EVE_SMX_EVAL_SLICE`/`EVE_SMX_EVAL_CASE_IDS` selection +
      `EVE_SMX_EVAL_CONCURRENCY` bounded pool (specs; measurement plumbing only,
      prompt hash untouched). Discovery recorded: family-gen-error- honesty's
      residual flake is the model fabricating over failed tools and the
      PRODUCT's lookup-claim checker holding it — the P4 feedback loop's exact
      target.

### Telemetry v2 (observability only in P0 — no enforcement changes)

- [x] 0.14 Extend `AssistantTurnMetricRecord` (+ snapshot version bump with
      honest v(n-1) restore): `costUsd`, `cacheReadTokens`, `cacheWriteTokens`
      (pi-ai `Usage` shape — both cache directions), `family` (null until P2),
      `skillVersion` (null until P2), `checkerVerdicts`,
      `escalationTier`/`escalationReason` (null until P5), `toolErrors` by name,
      `iterations`; specs for the snapshot migration. Done 2026-08-16: record +
      `ProviderAggregate` + `AssistantMetricsSummary` gained every field;
      snapshot bumped v2→v3 (`METRICS_SNAPSHOT_VERSION`) with
      `withTelemetryV2Defaults` so v1/v2 restore with sums=0/counts-empty (no
      invention — 0-over-0-reported reads "unmeasured", not "free"), and an
      unknown future version is ignored not restored-as-garbage; sums accumulate
      ONLY the turns that reported each field (parallel
      `costReportedTurns`/`iterationReportedTurns` denominators). Route wired
      (observability-only, model-invisible): `recordTurnMetric` forwards
      `result.usage` cost+cache, per-turn tool-error tally (from
      `toolInvocations` ok=false), and `iterations` on both completed+refused
      paths; family/skill/escalation/checker stay absent until their phases. 18
      turn-metrics + 38 turns-route tests green; bff typecheck ratchet green.
- [x] 0.15 `tools/eve-smx-cost-report.mjs`: per-family cost, cache-hit rate,
      tool-error and outcome breakdowns from the metrics endpoint; runs against
      a live BFF; output committed nowhere (operator report). Done 2026-08-16:
      reads `GET /v1/assistant/metrics` (admin token via `EVE_SMX_ADMIN_TOKEN`,
      `--bff`/`--json` flags), prints per-provider + per-family cost, cache-read
      rate, avg iterations, outcomes, top tool-error shapes, and
      escalation/checker tallies — dimensions a later phase fills read as their
      LANDING PHASE ("router lands in P2", etc.), never hidden. Pure
      `buildCostReport`/`renderCostReport`/`ratio` factored out and unit-tested
      (`eve-smx-cost-report.test.mjs`, 6 node:test cases): cost-per-turn is over
      REPORTING turns not all turns, a zero denominator returns null not NaN,
      empty metrics render an honest "no turns" report. Nothing written to disk.

### Baseline

- [x] 0.16 Baseline arm A: full deck, champion pin, fp8, sort:price, k=3 —
      scorecard written to `docs/audits/EVE_SMX_SCORECARD.md` (per-family
      pass@1/pass^k + CI + cost + latency + prompt-bytes hash). Done 2026-08-16
      (post-0.13 deck): **89/121 pass^3 · deck pass@1 77.4% · $0.1109 / 363 runs
      · median 7.4s · served StreamLake+Baidu (fp8)**, concurrency 3. Weakest
      families measured honestly: workbench-read 22.2%, docs 28.6%,
      workbench-write 50% pass^k (the P2 targets); safety/tour 100%. Advisory
      ledger-280 now PASSES 3/3 (recorded — the known-red note in the sources
      doc is superseded by measurement).
- [x] 0.17 Baseline arm B (calibration, one-off): same deck on ONE stronger
      model to establish "what good looks like" per family; recorded in the
      scorecard as a measurement arm, explicitly not a serving binding; ask the
      user before spending if the chosen model is frontier-priced. Done
      2026-08-16: `deepseek/deepseek-v4-pro-0813` (same-lineage stronger
      sibling, route-priced first — mid-tier, NOT frontier, so no user gate),
      fp8, sort:price, k=3, served GMICloud, **$1.3216 / 363 runs, median
      11.5s**. Headline calibration finding, now measured not argued: **89/121
      pass^3 — identical to the champion at ~12× the billed cost** — and the
      23-case consistent-gap partition fails identically on both models, so
      those gaps are harness-bound (P2's mandate), not model-capability-bound.
      Per-family deltas in the scorecard (capability-smalltalk +10.5 on the pro
      model; audit and tour WORSE — "stronger" is not uniformly stronger on this
      harness).
- [x] 0.18 Ratchet artifact `docs/audits/eve-smx-ratchet.json`: per-family
      floors from arm A + prompt-bytes hash (conduct core + tool descriptions
      today; + skills/registry from P2); floors move only up; lowering requires
      a human sign-off row in the scorecard. **BOTH HALVES DONE 2026-08-16.**
      Hash half: real `promptBytesHash` (66d00f31…, 60 tools / 2230 conduct
      bytes / 13830 tool-description bytes) computed by
      `eve-smx-prompt-hash.ts`. Floors half: `familyFloors` stamped from arm A's
      per-family case-level pass^k (all ten families, rounded DOWN to 4dp so
      identical behavior clears its own floor; advisory cases included in
      pooling to match the runner's rollup), `familyFloorsSource` names the full
      run recipe. The 0.19 spec's floors-stay-null expectation flipped to the
      stamped-shape gate: floors must cover exactly the family vocabulary
      (`ASSISTANT_EVAL_FAMILIES`, now the runtime single source the type derives
      from), each floor in [0,1], and the source must name the champion slug + k
      — a floor whose run cannot be named is a fabricated gate.
- [x] 0.19 CI hash gate (no spend): a spec/script recomputes the prompt-bytes
      hash and fails on mismatch with the committed ratchet — "you cannot change
      what the model reads without re-measuring what it does." Done 2026-08-16:
      `eve-smx-prompt-hash.spec.ts` recomputes the digest of the STABLE
      model-facing bytes and asserts it equals the committed ratchet hash,
      failing with the exact re-measure-and-re-stamp instruction (never
      hand-patch); also asserts the tool COUNT matches (a tool added/removed is
      a model-facing change), the digest is deterministic, the hashed set is
      sorted/order-independent, and `familyFloors` stays null (no invented
      floor). Landed BEFORE P1 deliberately — the whole P0 thesis is that no
      prompt change slips in unmeasured, so the gate must predate the first
      prompt edit. To enable it: conduct core extracted to
      `assistant/conduct.ts` (byte-identical, turns-route 38/38 green — this is
      also P2's dismantle seam). 4 hash-gate tests green; bff typecheck ratchet
      green.

### Harness hygiene (pi review — model-invisible, so P0-safe)

- [x] 0.20 Abort propagation (SMX-G13): thread an `AbortSignal` from the SSE
      route (client disconnect) through `runAssistantAgentTurn` into
      `createStreamingChatCompletion`; in-flight tool executions finish, the
      next provider call never starts; spec proves a disconnect mid-iteration
      cancels the stream and records the turn honestly. No prompt/tool bytes
      change. Done 2026-08-16: `AssistantAgentTurnInput.signal` checked at every
      loop boundary — the guarantee is "the next provider call never starts",
      not "the current call is killed" (the shared `LLMProviderInterface` is
      unchanged, so no cross-package churn and no prompt/tool bytes touched);
      new `outcome: 'aborted'` + `AssistantTurnOutcome: 'aborted'`. The turns
      route builds an `AbortController` from `request.raw` 'close' (covers SSE
      and buffered-JSON/mobile), passes `signal`, records `aborted` honestly (no
      fabricated completion, nothing streamed), and removes the listener in
      `finally`. Specs (runner-level, since inject can't disconnect mid-turn):
      the in-flight tool finishes but the SECOND provider call never fires
      (`requests` length 1) and its text never reaches the wire; an
      already-aborted signal runs ZERO provider calls (iterations 0). 12
      runner + 38 turns-route tests green; bff typecheck ratchet green.
- [x] 0.21 Turn-trace capture (SMX-G14, v1 = capture only): behind an env-gated
      flag, dump per turn a JSONL trace of exactly what the model READ and did —
      assembled promptParts (by name + bytes), tools offered, per-iteration
      request/response shapes, checker verdicts when they exist; operator-only,
      never a member surface, secrets never logged; spec on the trace writer.
      The workbench viewer is a later, separate task. Done 2026-08-16:
      `turn-trace.ts` — `resolveTurnTraceSink` (off unless
      `OSHUN_ASSISTANT_TURN_TRACE_FILE` names a path; best-effort append, never
      fails a turn) + `tracePromptParts` (name + UTF-8 bytes, empty-filtered to
      match the sent prompt). SECRETS-NEVER-LOGGED is STRUCTURAL: the trace type
      carries only names/byte-counts/counts/token-counts/tool-NAMES — no
      free-text — so a planted secret cannot serialize (spec plants one and
      proves it). Runner gained an optional `onIteration` shape callback
      (counts + tokens only, model-invisible). Route: prompt assembly refactored
      into NAMED parts (byte-identical join — turns-route 38/38 still green,
      ratchet green — the P1 1.1 split starts here), trace written once at the
      outcome convergence so completed/refused/aborted are all traced; checker
      verdicts left for P4. 7 trace + 12 runner tests green. NOTE: the
      journey-inventory freshness gate is red from a MERGED-MAIN e2e spec
      (`studio-study-seeded-content.spec.ts`, +1 journey/+3 tests) — not this
      work; re-mining cascades into the product-graph rebuild (EVE-VIS-125),
      which is main's estate churn to reconcile, so left as main has it.
- [x] 0.22 **P0 exit gate:** ≥120 graded cases across all families incl.
      adversarial; case count + composition recorded in the scorecard; arms A
      and B committed; ratchet + hash gate live; abort + trace landed; zero
      model-facing behavior diffs anywhere (git log is the witness). VERIFIED
      2026-08-16, clause by clause: 121 graded cases, all ten families ≥1 case
      (spec-enforced), 21 adversarial; composition in the scorecard's "Deck
      under measurement"; arms A+B committed in
      `docs/audits/EVE_SMX_SCORECARD.md`; ratchet has hash + stamped floors and
      `eve-smx-prompt-hash.spec.ts` gates both in CI; abort (0.20) + trace
      (0.21) landed earlier with specs. Model-facing witness: the hash gate is
      green over the session's entire git range — conduct core and all 60 tool
      descriptions byte-identical; the only non-eval bff source touched is
      `mock-domain-service-adapters.ts` (June-pinned fixture windows → relative,
      a test-data date-rot correction; tool CODE and descriptions unchanged),
      recorded honestly here. **P0 COMPLETE.**

## P1 — Cache order & context budgets (closes SMX-G10)

- [x] 1.1 Split prompt assembly into stable-prefix parts (conduct core, later
      skills) and dynamic-suffix parts (persona, daypart, memory, page context,
      anchors, tour catalog), behind a flag; byte-diff spec proves the flag-off
      order is unchanged. Done 2026-08-16: `assistant/prompt-order.ts` —
      `orderPromptParts` (flag-off returns the INPUT ARRAY ITSELF, reference-
      equal = strongest byte-diff guarantee; flag-on:
      `STABLE_PROMPT_PREFIX_     NAMES` = {conduct-core} first, dynamic parts
      keep relative order) + `resolveCacheOrderedPrompt`
      (`OSHUN_ASSISTANT_CACHE_ORDERED_PROMPT`, default OFF until 1.7's measured
      flip, junk throws). Route orders the 0.21 named parts and the turn trace
      records the ORDERED assembly (what the model actually read). Specs: 7 unit
      (incl. permutation witness and a guard that the stable set contains no
      session-varying name) + a route-level wire witness (flag-off: conduct
      present but NOT first — pre-P1 order untouched; flag-on: systemPrompt
      startsWith the conduct core, anchors still present). 46 tests green; bff
      ratchet green; hash gate unaffected (order is not hashed; bytes
      unchanged).
- [x] 1.2 Trim conditional blocks that don't pay rent per turn: anchors registry
      and curated-tour catalog only when the turn's capabilities can use them
      (pre-P2 approximation: capability flags; post-P2: family). Done
      2026-08-16: curated-tours was ALREADY gated on `tour_player` (pre-existing
      — verified, not re-done); the anchor registry now rides only with clients
      that can RENDER it — `highlight` (new accepted signal) or `tour_player` (a
      tour player is an anchor-highlighting machine by construction; the member
      web panel declares it, so member turns keep the registry bit-for-bit).
      Admin chat (`action_confirm` only) and undeclared/native clients stop
      paying for a registry they cannot point at; on-page anchors still travel
      in the page-context block. Route spec: three-way witness (undeclared →
      absent, admin shape → absent, tour_player → present) + the highlight test
      now declares the capability it exercises. Deck tour cases already declare
      `tour_player`. RECORDED follow-up: the web panel should also declare
      `highlight` explicitly (today tours carry it; if tours were ever
      tier-disabled the registry would silently drop) — a frontend change with
      its own browser-verification pass, deliberately not smuggled in here. 40
      route + 103 evals tests green; bff ratchet green.
- [x] 1.3 Tool-result digest seam: results over a stated size return a
      server-side digest + explicit "call again with offset/filter" affordance;
      per-tool opt-in starting with the largest observed payloads (from 0.15's
      report); spec per digested tool. Done 2026-08-16. Observation source
      honestly amended: 0.15's metrics report has no per-tool sizes, so the
      groundwork commit added `resultBytes` to invocations + the turn trace and
      a traced full-deck k=1 run (champion, fp8, sort:price, 145 turns) produced
      the record — largest observed: audit_status 4,014 B / audit_begin 4,000 B
      / audit_mark 3,694 B (the ten-domain rollup rides every call) and
      search_docs max 3,916 B; every other tool ≤ 2,028 B.
      `tool-result-digest.ts`: stated size 3,000 chars; audit digest windows the
      pending-ranked rollups to 3 and pages via a NEW real
      `audit_status.domainsOffset` arg (offset-0 shape byte-identical, overshoot
      clamps); docs digest keeps the top-2 full excerpts (grounding material)
      and stubs the tail naming the real query/limit args; digesters pure, seam
      in `toolset.execute` above the stated size only, blunt 8k truncate stays
      the outer guard; `read_page` recorded as the named future candidate (zero
      observed calls — nothing to size from). Specs: per digested tool +
      registry + seam integration (member binding AND extraBindings route) +
      audit paging arithmetic. Hash gate green — descriptions untouched; the
      affordance rides in dynamic result bytes (schema-only `domainsOffset`
      addition recorded here: model-facing but outside today's hash scope, a P2
      widening candidate). 52 + 40 route + 106 evals tests green; bff typecheck
      ratchet green. Deck impact re-measured at 1.7.
- [x] 1.4 Measure: cached-token rate and cost per turn before/after on the
      routed champion (and note whether the auction's cheap endpoint caches at
      all — the reorder must be free even when it doesn't). Done 2026-08-16,
      recorded in the scorecard's P1 section: champion, fp8, sort:price, full
      deck k=1 × two arms differing ONLY in the flag, both pre-digest —
      **cache-read 90.1% flag-off → 90.8% flag-on; $0.0392 vs $0.0397 / 121
      runs; median 10.0s vs 9.6s; served StreamLake+Baidu.** The auction's cheap
      endpoints DO cache (implicit, `cached_tokens` on ~97% of runs);
      within-session transcript re-sends dominate the rate, the reorder is free
      and adds ~0.7pp cross-session sharing at deck scale. Deck k=1 delta (95
      vs 92) traced case-by-case to the known flaky partition — 1.7's k=3
      decides flat. Also recorded there: the P1.3 observation table and the
      workbench-env finding (eval runs offer no workbench tools without an admin
      DB — a P2 fixture mandate, not changed mid-phase).
- [x] 1.5 History compaction behind a flag (pi lesson): beyond a stated
      threshold, early turns fold into a rolling summary carried in the dynamic
      suffix instead of being dropped by `slice(-N)`; summarizer is
      deterministic or a cheap leg (recorded which); deck flat within CI with
      the flag on. Done 2026-08-16: `history-compaction.ts` —
      `OSHUN_ASSISTANT_HISTORY_COMPACTION` (default off, junk throws); beyond
      the existing 8-turn window the dropped turns fold into a
      `history-compaction` dynamic prompt part. Summarizer choice RECORDED:
      deterministic extractive head-fold (first 160 chars of each utterance,
      whitespace-normalized, oldest first, 24-turn fold cap with the earlier
      remainder counted out loud) — a documented approximation trading
      abstraction for zero model cost and byte-reproducibility; a cheap-leg
      abstractive summarizer is a P6+ measured decision. Specs: 6 unit
      (threshold null, fold-exactly-the-dropped, head-fold + normalization,
      honest overflow count, determinism) + a route wire witness (flag-off no
      part; flag-on at 9 prior turns folds turn 1 ONLY while the message window
      keeps 2–9). Deck-flat clause honestly scoped: every deck case is ≤4 turns
      — under the window the flag changes zero bytes by construction (spec'd),
      so the deck cannot witness compaction depth; 1.7's exit run executes with
      the flag ON to prove no interference, and the flag stays default-off as
      unearned-by-deck-measurement. 47 tests green; bff typecheck ratchet green.
- [x] 1.6 Session cache affinity (pi lesson): pass whatever session/user
      affinity the routed champion provider honors; measure via cache-read
      tokens; record the finding honestly even if it is "none honored on this
      route". Done 2026-08-16, finding recorded in the scorecard — **honored,
      and NEGATIVE on this route**: mechanism = `ChatCompletionRequest.user`
      (opaque session id, no PII) forwarded on both OpenAI-compatible paths,
      runner carries it per iteration, route flag
      `OSHUN_ASSISTANT_SESSION_AFFINITY` (default off, junk throws); wire-spec'd
      both directions (present + absent-stays-absent). Measured k=1 full-deck
      A/B on the SAME post-digest tree, cache-order on, champion fp8 sort:price:
      affinity ON = 87.2% cache-read / $0.0422 / 9.4s median vs its exact
      control 90.1% / $0.0402 / 6.6s — sticky routing pins sessions AGAINST the
      price auction that already yields 90% implicit cache. Flag stays default
      OFF with the table as the reason; the field remains for routes where
      affinity pays. 18 shared-ai + 56 bff tests green; both typecheck ratchets
      green.
- [x] 1.7 **P1 exit gate:** full deck k=3 flat within CI vs P0 floors; cost +
      cache deltas + compaction/affinity findings recorded in the scorecard;
      flags default-on where earned. **PASSED 2026-08-16** (scorecard P1 exit
      section, four k=3 draws): the clean flagged run (cache-order + compaction
      on, affinity off, champion fp8 sort:price, concurrency 4) scored **91/121
      pass^3 · $0.1135 · median 8.0s · cache-read 91.8% — every family AT or
      ABOVE its floor** (general 100 and navigate 100 above; seven at floor).
      Two earlier draws were contaminated by network-path failures
      (operator-reported local internet loss; ~26 and ~7 runs died
      `assistant_agent_provider_error`, capping cases below pass^k on uptime not
      behavior) — the paired flags-off control breaking FIVE floors is what
      exonerated the flags before the clean run; the runner now grants each run
      one counted, visible provider-error replacement (spec'd ×4, incl.
      multi-turn). Earned flip: `OSHUN_ASSISTANT_CACHE_ORDERED_PROMPT` **default
      ON** (explicit 0 opts out; resolver + route wire specs flipped with it).
      Compaction and affinity stay default off (unearned / measured negative).
      Floor-semantics finding recorded for P7/P8: point pass^k floors at k=3 are
      one-draw statistics — gate on Wilson lower bounds and retry-hygienic runs.
      P1 live spend ~$0.51 / ~1,584 runs. **P1 COMPLETE.**

## P2 — Router, toolset scoping, skills v1 (closes SMX-G2, G3; starts G9)

- [x] 2.1 `resolveTaskFamily` pure function + spec: server-state signals first
      (active audit run, pending confirm, admin scope), then page/capability,
      then keyword/catalog heuristics; returns `{family, why}`; fails open to
      `general`. Done 2026-08-16: `task-family-router.ts` — five tiers per C1
      (active-audit beats all text; pending-confirm routes to the PARKED
      mutation, operator wb-write vs member member-data; workbench families need
      scope AND surface AND, for writes, a confirm-capable client; an explicit
      docs marker outranks tour phrasing — the docs-grounding case's exact
      shape; navigation/member-data/smalltalk reuse the deterministic engine's
      intent verdict passed IN as a fact so the router stays pure; everything
      else `general`). Every verdict carries a stable `why` slug for 2.2's
      telemetry. Vocabulary moved to `task-families.ts` as the runtime single
      source (eval-harness re-exports it — production code no longer imports
      from the eval tree; `AssistantRoutableFamily` excludes `safety` by type,
      since crisis supersede preempts routing). 17 router + 106 evals tests
      green; bff typecheck ratchet green. Heuristics documented as heuristics —
      2.3's misroute audit is the judge.
- [x] 2.2 Route wiring: turns route consults the router; `{family, why}` into
      telemetry (0.14 field goes live); no behavior change yet beyond logging
      (dark launch), one full deck run to confirm flat. Done 2026-08-16: the
      route builds the facts (active audit run, the action bridge's new
      `hasPendingForSession`, admin scope, capabilities, docs/workbench surface
      presence, the engine's intent verdict via one route-scoped resolver) and
      records the verdict in the request log, the 0.14 `family` telemetry field
      (LIVE from this commit; pre-router refusals stay honestly "unrouted"), and
      the turn trace (`routedFamily`/`routedWhy` — slugs only, structural
      no-free-text holds). Route witness: a favorites ask adds exactly one
      member-data tally to the admin metrics families map (delta assertion).
      Confirm-flat run (scorecard P2.2): defaults + retry, k=3 — **89/121 pass^3
      = the P0 baseline exactly, $0.1108, cache-read 92.2%**; nine families
      at/above floor, general 7/9 on the KNOWN flaky honesty pair (three of five
      k=3 draws land there incl. the flags-off control; the launch is
      model-invisible by construction). Distribution across 435 traced turns
      recorded (docs texts falling through to general = 2.3's first target). 56
      route/bridge/trace tests green; bff typecheck ratchet green.
- [x] 2.3 Misroute audit: sample logged routes against hand labels (≥100 turns
      from deck + battery traffic); misroute rate recorded; threshold set and
      met before scoping turns on. Decide (and record) whether a micro-model
      router leg is warranted — default no. Done 2026-08-16
      (`misroute-audit.spec.ts`, provider-free CI ratchet; scorecard P2.3):
      **144 labeled turns (deck + battery, safety excluded as
      never-routed-by-construction, 3 surface-guard expected routes declared per
      id) — agreement 97/144 (67.4%), harmful-direction misroutes 16/144
      (11.1%)** where harmful = routed into a NARROWER family than the label (31
      of 47 disagreements land in fail-open `general`, harmless under scoping).
      Thresholds asserted in the spec (≥0.67 / ≤0.12), confusion table printed
      every CI run. Micro-model leg: NOT warranted, decision + reopen-evidence
      recorded (revisit only if 2.11 shows scoped families regressing on
      misrouted turns). Principled fix landed: `cross_domain` intent verdicts
      route `general` (full surface for snapshot asks) — lower raw agreement,
      lower harmful rate, unit-spec'd. 14 router + 1 audit tests green; bff
      typecheck ratchet green.
- [x] 2.4 Skill registry: `skills/` directory, `AssistantSkill` type, registry
      spec enforcing token budgets (core ≤ ~350, skill ≤ ~700, chars/4 heuristic
      documented), unique ids, changelog lines. Done 2026-08-16:
      `skills/skill-types.ts` (AssistantSkill per C2 — routable family, body, ≤2
      exemplars, toolAllowlist, checkerIds, maxIterations, evalCaseIds,
      changelog) + `skills/registry.ts` (`validateSkillRegistry`: versioned
      self-describing ids `skill.<family>.v<n>`, uniqueness, 700-token skill
      budget counting exemplar bytes via the DOCUMENTED chars/4 heuristic with
      ceil, exemplar ceiling, non-empty body + changelog — throws with the
      offending id). Registry starts EMPTY by design: the machinery is proven on
      fixture positive/negative controls and the committed registry clears its
      own gates in CI; 2.7's dismantle populates it through exactly these gates.
      HONEST DEFERRAL recorded: the CORE ≤350-token assertion is pinned as a
      constant here but ASSERTED at 2.7 — today's conduct core measures ~557
      heuristic tokens (2,230 bytes), and the dismantle is what makes the target
      reachable. 7 registry tests green; bff typecheck ratchet green.
- [x] 2.5 Traceability lint: every skill names ≥1 real deck case id; unknown ids
      fail the spec. Done 2026-08-16:
      `validateSkillTraceability(skills,     knownCaseIds)` — pure, the known-id
      set is an INPUT so production code never imports the eval tree; the
      registry spec feeds it the real `ASSISTANT_EVAL_DECK` ids and proves the
      positive control, the zero-trace refusal ("unmeasured prose cannot
      register"), the invented/renamed-id refusal (offending id named), and that
      the committed registry traces clean. 8 registry tests green; bff typecheck
      ratchet green.
- [x] 2.6 Allowlist integrity spec: every `toolAllowlist` entry exists in the
      union of registered tool names built by the real builders. Done
      2026-08-16: `validateSkillAllowlists(skills, registeredToolNames)` — pure
      like 2.5's lint; the spec feeds it the REAL builder union
      (`registeredToolNames()` from deck-admission-support, the same
      throwing-proxy collectors the deck admission uses) and proves the positive
      control, the renamed/invented-tool refusal (name called out), the
      EMPTY-allowlist refusal (a zero-tool family is a misconfiguration —
      `general` keeps the full surface by having no allowlist applied, never by
      an empty list), and the committed registry passing. 9 registry tests
      green; bff typecheck ratchet green.
- [x] 2.7 Dismantle `AGENT_CONDUCT_INSTRUCTIONS` into core + per-family skills
      (member-data, docs, navigate, tour, audit, workbench-read,
      workbench-write, capability-smalltalk); prose meaning preserved or
      improved case-by-case; every moved sentence keeps or gains a deck case.
      Done 2026-08-16 behind `OSHUN_ASSISTANT_SKILLS` (default OFF — flag-off
      serves the legacy block BYTE-IDENTICALLY, route-spec'd, so P0/P1 floors
      stay valid until 2.11 measures flag-on). `AGENT_CONDUCT_CORE` = six
      universal sentences, 845 B ≈ 212 heuristic tokens (≤350 gate ASSERTED in
      the registry spec from now on; legacy measured ~557). Seven skills
      populated through every 2.4–2.6 gate, each sentence moved VERBATIM with
      its measured-history comments (member-data depth incl. the 080 story;
      navigate + page-context; highlight → tour; audit protocol + skip
      provenance; the route's docs and workbench BLOCKS became skill.docs /
      workbench skills — flag-on they ride only with routed turns, the docs
      trade recorded in the skill header); action-truth core sentence IMPROVED
      with the announce-before-act clause (gains ledger-130/282); id-hygiene
      compressed, all four id classes kept. No capability- smalltalk skill —
      nothing existed to move (recorded; cs turns serve core-only when flag-on,
      watched at 2.11). Skill part is family-stable → joined
      `STABLE_PROMPT_PREFIX_NAMES` (rides directly behind the core). Hash scope
      EXTENDED (core + skills hashed; both serving variants pinned) and ratchet
      re-stamped 66d00f31→6ac0e079 with legacy components unchanged — scorecard
      P2.7 row records it, plus the discovered invisible-control-byte separators
      in the preimage now written as explicit escapes. 44 route + 134
      skills/hash/order/evals tests green; bff typecheck ratchet green.
- [x] 2.8 Toolset scoper: per-family allowlist projection over the built
      toolset; `general` keeps the full surface; per-family `maxIterations`.
      Clamp semantics (Hermes lesson): the projection INTERSECTS the tools the
      turn is actually authorized to carry — a skill can narrow, never widen;
      spec proves an allowlist naming an unauthorized tool does not surface it.
      Done 2026-08-16: `toolset-scope.ts` — a projection over the BUILT toolset
      (nothing rebuilt): definitions filtered to allowlist ∩ carried, execute()
      clamped to the same set (an unoffered name refuses honestly, even
      hallucinated), live state (navigationTarget/tourPlan) passes through.
      Spec: intersect witness (read_page unbuilt + start_tour playerless + an
      invented name all fail to surface), execution clamp, navigate passthrough
      on a real destination. Route: rides the SAME `OSHUN_ASSISTANT_SKILLS` flag
      as 2.7 (one measured unit at 2.11) — routed skill → scoped toolset +
      skill.maxIterations to the runner; general/capability-smalltalk keep the
      full surface by having no skill. Route witness extended: flag-on
      member-data offers ONLY allowlisted tools and strictly fewer than
      flag-off. 3 + 44 route + provider-free selfchecks green; bff typecheck
      ratchet green.
- [x] 2.9 Forced tool choice for tool-mandatory families on iteration 1 where
      the provider supports it (capability-checked; honest fallback to `auto`);
      runner spec covers both paths. Done 2026-08-16: `AssistantSkill` gains
      `forceFirstToolCall` (set on member-data + docs — the C3 tool-mandatory
      pair); the runner gains `firstIterationToolChoice: 'any'` applied on
      iteration 1 ONLY (later iterations fall back to `auto`; the tools-disabled
      final iteration unaffected). Capability basis recorded: `'any'` →
      `required` via `convertToolChoice` on every tool-supporting provider in
      this codebase (verified in openai.ts; anthropic-native maps it too), and
      tool-less providers never reach a skill (inert toolset) — so the honest
      fallback is the absent-input path, spec'd. Runner spec covers both paths
      (forced: iteration-1 `any` / iteration-2 `auto`; absent: all `auto`);
      route witnesses flag-on member-data `any` and skill-less general `auto`.
      Not prompt bytes — hash untouched; behavior measured at 2.11. 74 tests
      green; bff typecheck ratchet green.
- [x] 2.10 Deferred tools for `general` (pi lesson): tail tools offered as names
      only + a `load_tools` meta-tool returning full definitions on demand;
      which tools defer is decided from 0.15's usage report; spec covers load +
      subsequent call; deck cases prove a deferred tool still gets used when the
      task needs it. Done 2026-08-16: `defer-tools.ts` —
      `GENERAL_DEFERRED_TOOLS` = the 29 tools with ZERO observed calls across
      the eight traced runs (~2,700 invocations — the observation source
      honestly amended from 0.15's endpoint, which has no per-tool call data, to
      the turn traces, same amendment as 1.3); general-routed flag-on turns
      offer the 31-tool head + `load_tools` (its description names the loadable
      tail, intersected with the CARRIED surface; a surface with no deferrable
      tool pays no meta-tool rent). Loading surfaces definitions for the next
      iteration (the runner re-reads definitions per iteration); an unloaded
      deferred call refuses pointing at load_tools; unknown/head names refuse
      naming the loadable set. Spec covers offer/load/subsequent- call
      (passthrough witnessed via the mutating-tool refusal reaching the REAL
      seam) + no-rent + head-disjointness. NEW deck case
      `family-gen-deferred-tool-reach` (deck 121→122): a routes-general-verified
      ask that needs zero-traffic `veritas_unfollow_topic` — flag-on that means
      load-then-call, flag-off direct; graded on reaching the tool + the confirm
      park; 2.11 runs it live. Misroute ratchet re-verified at 145 turns (67.6%
      / 11.0%, both thresholds hold). load_tools' generated description is
      model-facing but outside today's hash (recorded — same class as the 1.3
      schema note). 160 tests green; bff typecheck ratchet green.
- [x] 2.11 Full deck k=3 on champion with routing+skills on: every family ≥
      floor; tool-selection families strictly better; prompt tokens/turn and
      tools-offered/turn reductions measured and recorded. Done 2026-08-17:
      THREE flag-on draws recorded (scorecard P2.11). Draw 1 (pre-revision)
      exposed the forced member-data call as a fabrication vector (refusal case
      0/3, member's own favourite leaked as the friend's) — REVISED: force
      removed from the md skill, docs keeps its; deferred-reach case advisory
      (flash never discovers load_tools unprompted, 0/3 in both revised draws).
      Draw 2 missed md/cs by 3/1 flaky cases; decision rule pre-registered, then
      draw 3 cleared EVERY floor on the stamped basis — md 37/48 (.7708)
      strictly above floor and strictly better than flag-off 36/48 (pass@1 79.9%
      vs 77.8%), cs 16/19, general 8/9 — with the noise caveat recorded (the
      revised draws lose different 2/3 cases; the 0/3 md set is identical
      flag-off/flag-on = the P0 gap). Draw 3's last case completed as a banner-
      loud k=3 single-case run after an environment kill at the finish line
      (grade-extraction method validated against draw 2's printed table 10/10).
      Reductions: tools-offered/turn 26.2 → 18.8 (−28.3%) · input tokens/turn
      11,602 → 9,207 (−20.6%) · sys-prompt bytes −10.7%. True tool-selection
      gaps (wrong-domain calls, load_tools discovery) are description/example
      problems → P3.4. ~$0.40 across the three draws.
- [x] 2.12 **P2 exit gate:** scorecard + ratchet updated (new hash, new floors
      where risen); misroute rate < threshold with the number in the scorecard;
      dark-launch flag removed. Done 2026-08-17: `OSHUN_ASSISTANT_SKILLS` and
      the legacy conduct block DELETED — skills are the only serving path
      (revert = git revert, not an env flip). Ratchet re-stamped `a99693bb…`
      (the legacy section left the preimage; every remaining hashed byte
      unchanged from 2.7, so 2.11's numbers bind to the served bytes). Floors
      deliberately NOT risen — single-draw highs are one-draw statistics
      (P1.7/P2.11 precedent); Wilson gates queued P7/P8. Misroute 98/145 = 67.6%
      ≥ .67 · harmful 16/145 = 11.0% ≤ .12, CI-asserted. Removal find: always-on
      scoping exposed 18 of 29 deferred-tail names as VACUOUSLY zero-call (never
      OFFERED in the eval env — read_page + 17 workbench tools; deferring
      read_page broke the round-trip spec); tail trimmed to the 11 honest
      offered-and-never-called names, measured runs unaffected (the 18 weren't
      carried in eval). 510 assistant tests green; bff tsc exit 0; journey
      inventory regenerated (unrelated main-merge staleness).

## P3 — Constrained argument boundary (closes SMX-G4)

- [x] 3.1 Shared schema validation at `toolset.execute`: args validated against
      each tool's declared JSON schema before the tool body runs; instructive
      error contract (field, expected shape, one example) as the tool result;
      spec per error shape. Done 2026-08-17: `toolset-validate.ts` —
      `validateToolsetArgs` wraps the OUTERMOST seam in the route (everything
      reachable — scoped tools, load_tools, loaded tail — validates; a name with
      no current definition passes through so the inner seams keep their own
      refusal vocabulary; definitions read at CALL time for the deferred view).
      Validator covers the keyword census of the 60 deployed schemas
      (type/properties/required/items/enum — measured by grep, in the module
      header); unknown keywords are annotations, unknown types FAIL OPEN
      (refusing a working call on an unimplemented keyword would be worse than
      the pre-3.1 status quo, reasoning recorded). Refusal =
      `Invalid arguments for "tool": "field" must be <shape> (e.g. <example>)`
      with deterministic example synthesis (enum first member; objects render
      required properties); carries `errorKind: 'argument_validation'` — a
      server-side field on `AssistantToolExecution` so 3.2's repair pass
      classifies without string-matching model-facing text; `isError` feeds the
      existing per-tool `toolErrors` metric so 3.6 can measure the rate. 15
      specs: per error shape (missing-required, wrong-type, enum, indexed array
      path, dotted nested path, outer-shape-once), formatter contract,
      body-never-runs, valid-passthrough, unknown-name passthrough, call-time
      definitions, live getters, fail-open control, example synthesis. 525
      assistant tests green; bff tsc exit 0. (Same commit: inherited main-merge
      estate growth re-stamped — journey inventory 729/5935, parity STAMP
      13735/37, compilers union 37, SMX ledger added to the product-graph TODOS
      curation map + artifact regenerated.)
- [x] 3.2 Runner repair pass: an iteration whose only failures are argument
      validation errors grants ONE bounded repair iteration; second failure
      returns honest failure text; spec covers the bound. Done 2026-08-17: the
      runner keeps a mutable `iterationBudget`; an iteration whose FAILED
      executions all carry `errorKind: 'argument_validation'` (successes may
      coexist — its only failures are validation) raises the budget by one, ONCE
      per turn, so the bungled-args turn still reaches a tools-ENABLED retry
      before the tools-disabled final. A second all-validation iteration extends
      nothing — the turn ends through the normal tools-disabled final where the
      MODEL says its honest failure text (the runner fabricates nothing; the
      instructive refusals are in its transcript and the conduct core's
      failure-honesty rule binds). Result gains `argumentRepairIterations: 1`
      when granted (absent otherwise) so 3.6 can count repairs. 4 specs over the
      REAL P3.1 boundary (not a scripted errorKind): grant witnessed by the
      repair request CARRYING tools where the old ceiling would have disabled
      them; the bound (3 requests, never 4, honest text through);
      ordinary-body-error control (no extension); mixed success+validation
      iteration still grants. 529 assistant tests green; bff tsc exit 0.
- [x] 3.3 Structured outputs for non-prose legs (router-if-ever, judge):
      `response_format: json_schema` + `provider.require_parameters: true` via
      `@oshun/ai` OpenRouter options; positive-control spec proving the fields
      reach the wire (the `int2` method). Done 2026-08-17: `responseFormat`
      already rode both OpenAI-compatible paths (P1-era); the missing half —
      `provider.require_parameters: true` — now rides AUTOMATICALLY with any
      OpenRouter request that carries `responseFormat` (merged into the env
      preferences per-request; plain OpenAI never gains a `provider` key). int2
      LIVE controls 2026-08-17: bogus value →
      `400     provider.require_parameters: Invalid input: expected boolean, received     string`
      (the API naming the field path = arrival proven); positive leg on the
      champion (fp8+price pins, strict json_schema) SERVED by DeepInfra with
      schema-conforming JSON at $0.0000104 — structured outputs ARE available on
      the champion route under require_parameters (recorded for 3.5's
      availability question too). 5 wire specs in
      `openrouter-structured-outputs.spec.ts` (merge, streaming, absent- format
      control, no-env case, plain-OpenAI control). No serving consumer yet — the
      router is pure and the judge leg arrives with P5; this box is the
      capability, measured available. 532 shared-ai tests green; lib tsc exit 0.
- [x] 3.4 Tool-description pass: every scoped tool's description gets one inline
      worked example (small models read examples, not essays); no near-synonym
      tool names within a family's allowlist — collisions renamed or merged,
      listed in the commit body. Done 2026-08-17: all 58 allowlisted tools carry
      one schema-true worked example (ask → call with args), the measured
      confusion pairs carry bidirectional cross-references, and load_tools'
      generated description gained its example (the 0/3 discovery target).
      Collision audit: ONE true name collision — veritas_continue_reading /
      nisaba_continue_reading — RENAMED to veritas_continue_article /
      nisaba_continue_passage (15 refs across bff, web notes, e2e specs);
      tara/metis "course" confusion is semantic, fixed in descriptions. FOUND
      AND FIXED during the audit: nyx_saved_objects BOUND (EVE-VIS-101 class —
      adapter read existed, no tool; the deck's saved-objects cases graded
      honest refusals as misses), on BOTH engines per the 087 parity rule
      (intent + router arm rejoined, assistant read role widened, 60→61 tools);
      AND the release-scope guard caught the first-cut cross-references teasing
      deferred rooms to V1.0 members — they are now conditional on
      authorization. Hash re-stamped 79bc1f8c… (description bytes
      13,830→20,149); behavior measured at 3.6; floors unchanged. 530 bff + 520
      shell-assistant tests green (the one domain-nyx canonical-adapter failure
      pre-exists on the branch, verified by stash); bff + shell-assistant tsc
      exit 0.
- [x] 3.5 Grammar-constrained argument sampling (pi lesson): where the routed
      endpoint supports grammar constraints, constrain generation itself rather
      than only validating after; availability through OpenRouter measured per
      provider first (positive control) and the finding recorded even if the
      answer is "not on the champion route". Done 2026-08-17 — the answer IS
      "not on the champion route" for tool ARGUMENTS, measured (scorecard P3.5):
      `tools[].function.strict` has NO arrival proof (a bogus value was accepted
      and served — the int2 method's premise fails; OpenRouter neither validates
      nor advertises it), so provider-side grammar constraint of arguments is
      unverifiable and cannot be a serving mechanism; per-provider `/endpoints`
      sweep: `structured_outputs` on 7 of 13 fp8 champion endpoints (NOT
      StreamLake, the route's most frequent server). The available half —
      grammar-decoded `response_format` with `require_parameters` — was wired
      and positively controlled at 3.3. Decision: arguments stay constrained by
      the 3.1 boundary + 3.2 repair (verifiable, provider-independent); revisit
      if OpenRouter adds strict-schema validation. No serving-byte change; no
      code change.
- [x] 3.6 Full deck k=3: tool-arg error rate (now measurable via 0.14
      `toolErrors`) down vs P2; no floor broken; scorecard + ratchet updated.
      Done 2026-08-17 on draw 3 of a pre-registered three-draw rule (scorecard
      P3.6, all draws recorded): EVERY floor cleared — md 38/48 (pass@1 83.3%,
      best ever), cs 17/19 ABOVE floor, tour/navigate/safety 100%. Draws 1–2
      each lost ONE disjoint flaky singleton (tour, then general) — the k=3
      point-floor lottery, recorded; Wilson gates stay queued P7/P8. Arg-error
      rate: already ≈0 at P2, still ≈0 — every failure in all three draws is the
      designed error-fixture trio; "down vs P2" satisfied at zero, the
      boundary's value is the enforced guarantee + repair path. TWO structural
      finds fixed mid-box (own commits): the deferral inverted 2.3's
      benign-misroute premise (router vocabulary gained catalog/lessons/claims +
      search/browse triggers; agreement 67.6→69.7%, harmful 11.0% holds) and
      SEVEN deck cases were release-blocked since P0 (no V1.0 member token
      carries metis/veritas tools — marked advisory with the reason; promote
      with the release). Also: first attempt died on the app's own
      session-create limiter — harness now paces creates (own commit); ~$0.06
      discarded. Ratchet: hash/floors unchanged since 3.4 (79bc1f8c…). Deck
      ~$0.49 across the three draws + the dead attempt.
- [x] 3.7 **P3 exit gate:** boundary specs green; deltas recorded. Done
      2026-08-17: boundary specs green at exit — 15 toolset-validate specs (per
      error shape), 4 runner repair-pass specs over the REAL boundary, 5
      structured-outputs wire specs; full sweeps 530 bff assistant + 532
      shared-ai tests green; bff + shared-ai tsc exit 0. Deltas recorded in the
      scorecard P3 sections: draw-3 vs P2-close — member-data 37→38/48 pass^3
      with pass@1 76–78%→83.3%, capability-smalltalk 16→17/19, saved-objects
      case healed, arg-error rate held at ≈0 now ENFORCED at the boundary; hash
      79bc1f8c… (3.4), floors unchanged. Findings ledger for later phases: k=3
      point-floor lottery (four draws, four different single-floor misses →
      Wilson gates P7/P8); load_tools undiscoverable by the champion
      (P3.4-adjacent worked example did not fix it — P5 escalation or
      head-promotion candidates); within-scope wrong-domain selection persists
      on ambiguous asks (metis-recommend class — P4 checker / P5 target); docs
      family corpus-blocked (EVE-VIS-126); workbench surface env-blocked
      (fixture mandate stands).

## P4 — Grounding checkers (closes SMX-G5)

- [x] 4.1 Member-data grounding checker (EVE-VIS-080 mechanism): pure classifier
      over reply text + this-turn tool calls; hold → verdict injection → one
      corrective iteration → honest refusal; unit-tested on deck-derived
      fixtures; heuristic nature documented in the module header. Done
      2026-08-17: the pure classifier is `lookup-claim-check.ts` (heuristic
      nature + measured history documented since its EVE-VIS-080/095 birth;
      gained the model-facing LOOKUP_CLAIM_FEEDBACK). NEW runner seam
      `completionChecker` (generic — 4.2/4.3 compose): a candidate final text
      failing the check is HELD out of the final text, the verdict is injected
      as loop feedback, and the model gets ONE corrective opportunity with tools
      available (≤2 provider calls when it uses them); a second failing
      candidate ends the turn with the honest refusal — the retraction stops
      being what the member reads and becomes the last resort. Result carries
      groundingCorrections / groundingRefused (4.5's telemetry feed). Route
      wires the checker (warn log per hold; the post-hoc replacement stays as
      backstop). Wire exposure until 4.4 documented (held deltas already
      streamed; turn.complete authoritative — the EVE-VIS-242 window). 4
      deck-derived runner specs (REAL classifier, live EVE-VIS-080 sentences):
      hold→correct→grounded (feedback + held candidate witnessed in the
      corrective request), second-failure→refusal (claims never land),
      grounded-turn control, proposal control. Two route tests re-grounded (the
      checker was RIGHT to fire on their fabricating scripts). 534 assistant
      tests green; bff tsc exit 0.
- [x] 4.2 Citation grounding checker: full-id-shaped tokens and catalog titles
      in the reply must appear in this-turn tool results; same
      hold/correct/refuse path; spec'd. Done 2026-08-17:
      `citation-claim-check.ts` — pure classifier over reply + this-turn
      tool-result CONTENTS (the runner's checker candidate gained
      `toolResultTexts`, in-process only): id-shaped tokens (wi-/th-/adr-, the
      211 vocabulary) must appear verbatim in a result; quoted/bolded title
      spans must appear case-and-apostrophe-folded in a result OR in the
      member's own message (quoting the member back is conversation) OR in a
      prompt part the model legitimately read (page context, curated tours,
      memory — `promptEvidence`, wired from orderedParts in the route). ABSTAINS
      when no tool result exists (4.1 owns the no-lookup case); false-negative
      by construction on unquoted prose (recorded — the deck's finalExcludesAll
      fixtures stay the deeper net). Same hold/correct/refuse path via the SAME
      runner seam, composed after 4.1 in the route's checker. 8 classifier specs
      (EVE-VIS-095 bibliography shape, invented id, member-echo control,
      prompt-evidence control, abstention, apostrophe folding) + a runner
      composition spec (invented title held → corrective reply cites the real
      one). 543 assistant tests green; bff tsc exit 0.
- [x] 4.3 Count sanity checker (exact-extraction cases only; stated limits).
      Done 2026-08-17: `count-claim-check.ts` — a stated count of the member's
      things must equal what the tool returned, checked ONLY where ground truth
      is mechanical: the named object class maps through a CLOSED seven-entry
      vocabulary to a tool that ran ok this turn whose result parses as a plain
      JSON array (or the bare items wrap) with no pagination marker; cardinal
      claims 0–20, digits + number words + zero forms. Everything else ABSTAINS
      — prose results, digests, paged results, unlisted nouns,
      "several"/ranges/arithmetic (stated in the module header). Composed third
      in the route's completion checker; same hold/correct/refuse path. 6 specs:
      mismatch flagged (two-over-one shape), true count in digits and words,
      zero-form over a non-empty store flagged, three abstention limits,
      items-wrap counting, failed calls ground nothing. 549 assistant tests
      green; bff tsc exit 0.
- [x] 4.4 Wire integration on the sentence-atomic stream (the wire-hold
      pattern): SSE-level spec proving no ungrounded claim sentence reaches the
      wire when the checker fires. Done 2026-08-17: the grounding wire-hold sits
      beside the announce-before-act hold in the route's emit handler — from the
      first sentence that announces or states a member-data lookup
      (`mentionsALookup` over the accumulated sentence-atomic stream), in a turn
      where no data tool has succeeded yet, deltas buffer instead of streaming;
      a succeeding data tool (`ranADataLookup` on the event) flushes the buffer
      AFTER its tool_result frame, so the wire itself reads
      record-then-announcement; a turn whose lookup never lands streams nothing
      further and turn.complete carries the 4.1-corrected text — the
      EVE-VIS-242-class exposure window 4.1 documented is CLOSED. Two SSE-level
      specs: the double-fabrication turn shows NEITHER fabricated sentence in
      any turn.delta (only the honest refusal at complete), and the
      announce-then-act turn shows the announcement delta strictly AFTER the
      tara_favorites tool_result frame. The existing byte-identical
      innocent-stream control keeps passing. 551 assistant tests green; bff tsc
      exit 0.
- [x] 4.5 Checker verdicts into telemetry (0.14 field live); corrective-
      iteration count bounded and visible. Done 2026-08-17: the 0.14
      `checkerVerdicts` field is LIVE on both the turn-metrics record and the
      turn trace — checker name → held-refined-to `corrected` (the corrective
      candidate passed) or `refused` (it did not; only the last-held checker
      takes the refused verdict), plus `corrective_iterations: "1"` riding in
      the same record (the bound is the runner's design). Only FIRINGS are
      recorded — a passing checker writes nothing, so the aggregated
      `checker:verdict` buckets count events, not turns. Wired through the
      route's recordTurnMetric on the three agent outcomes
      (aborted/refused/completed; the pre-model and provider-error paths
      honestly carry none) and into the trace record (the "left for P4" comment
      retired). Spec: a corrected turn and a refused turn move exactly
      `lookup:corrected` +1, `lookup:refused` +1, `corrective_iterations:1` +2
      in the metrics summary (delta pattern). 552 assistant tests green; bff tsc
      exit 0.
- [x] 4.6 Full deck k=3 + the VIS-080 lock case at k=10: fabrication family
      improves; the 080 shape at ~0; latency p95 within budget; scorecard +
      ratchet updated. Done 2026-08-17 (scorecard P4.6, TWO calibration rounds
      recorded): round 1 caught the citation checker OVER-FIRING — 68 firings/47
      false refusals in ~370 runs (this model bolds headings and quotes
      emphasis; annotated titles failed raw containment) and the corrective
      feedback once re-created the 2.11 refusal→fabrication vector — fixed
      before anything else (bold out, title-shaped spans only, annotation-head
      matching; own commit). Round 2 (the measured unit): checker firings ZERO
      across 439 turns; floors 9/10 with general 9/9 PERFECT (first ever) and md
      37/41 = 90.2%; cs missed by one (the single-floor lottery, fifth draw
      fifth family). Gates: 080 shape 0-in-60 at k=10 (five locks 10/10; the one
      9/10 miss is an empty-reply provider artifact WITH the tool called);
      fabrication family at/near ceiling; checker latency contribution zero
      provider calls (medians 8.6–10.2s = P3 levels; runner prints medians not
      p95 — recorded). Run logistics: background deck runs were killed twice by
      the session environment → four foreground quarter-slices combined by the
      validated extraction (~$0.13 discarded partials recorded). Floors + hash
      unchanged. ~$0.16 for the measured unit.
- [x] 4.7 **P4 exit gate:** EVE-VIS-080 tracked item closeable with this as the
      mechanism (note in the ledger row's terms: fix + regression lock). Done
      2026-08-17: the 080 row in
      `docs/audits/EVE_POLISH_DEFECT_LEDGER_2026-08.md` carries the P4
      completion note in its own fix cell — the supersede grew into hold →
      verdict feedback → one corrective iteration → honest refusal, the
      wire-hold closed the row's inherited exposure window (SSE-spec'd), and the
      regression lock is the six ledger-080 deck locks measured at k=10: the
      shape at 0 fabrications in 60 runs, checkers silent on honest turns (0
      firings/439). Exit verification: 554 assistant tests green across the P4
      surface (checker classifiers, runner hold path, wire holds, telemetry);
      bff tsc exit 0. P4 spend ~$0.30 including both calibration rounds and the
      discarded partials.

## P5 — Escalation ladder + model registry (closes SMX-G6, most of G7)

- [x] 5.1 `model-registry.ts`: legs (turn, escalation, judge, embedding), pinned
      slugs, provider prefs, price snapshot + date + producing command; env
      overrides preserved; binding resolution reads it; spec. Done 2026-08-17:
      four legs — turn = champion pin (dated, P0.16/P0.17 provenance);
      escalation = pro-0813 PROVISIONAL from the calibration arm (5.4 re-decides
      and re-stamps); judge = pro-0813 provisional (no consumer until P8; 3.3's
      structured-outputs proof referenced); embedding = honestly UNBOUND (docs
      retrieval is BM25 by design — a slug nothing calls would be a paper pin;
      `assistantModelForLeg` THROWS on it, the fail-loud seam). Price snapshots
      captured LIVE 2026-08-17 via the /endpoints command each entry carries
      (champion StreamLake fp8 $0.079/$0.157 — moved from yesterday's
      $0.061/$0.123, which is why snapshots carry dates; pro GMICloud
      $1.218/$2.436). Binding resolution's openrouter default now reads the
      registry turn pin (the DATED snapshot replaces the floating alias for
      unconfigured deployments; the env chain above is untouched and spec'd to
      win). 8 registry specs (dated-slug regex, pins, provenance completeness,
      distinct env overrides, ladder-points-up price property, loud unbound
      throw, registry-read + override-wins on the real resolver). 563 assistant
      tests green; bff tsc exit 0.
- [x] 5.2 Ladder in the turns route: tier 1 champion → tier 2 same model +
      verdict feedback → tier 3 escalation slug; per-family enablement; member
      latency bound (one extra iteration per mechanism) enforced and spec'd.
      Done 2026-08-17: tier 2 IS the P4.1 corrective iteration (named as such in
      the runner's docs — same model + checker-verdict feedback, already bounded
      to one opportunity); tier 3 runs when the corrective attempt also fails
      and the route enabled it: ONE tools-DISABLED call to the escalation slug
      with the full transcript + held candidate + verdict (tier 2 owned the
      re-look; tier 3's job is to say it right — that is the enforced latency
      bound), still-failing or ERRORING tier-3 replies fall to the honest
      refusal and never crash the turn. Per-family enablement =
      `ESCALATION_ENABLED_FAMILIES` (member-data / capability-smalltalk /
      general — MEASUREMENT not taste: P0.17's pro model is +10.5 on cs but
      −14.3 on audit AND tour, which never escalate); openrouter-only (same
      client, different model string; slug from the 5.1 registry, env override
      wins). Result carries `escalation: {tier: 3, model, resolved}` (5.3 wires
      telemetry). 3 ladder specs: resolve (ONE call witnessed, right model, no
      tools, candidate+verdict in the transcript), still-failing → refusal +
      resolved false, provider-error → refusal without crashing. 566 assistant
      tests green. PROCESS CORRECTION recorded: the bff tsconfig has
      `noCheck: true` (a multi-GB full check OOMs the 16GB box — the known
      machine limit), so this session's earlier "bff tsc exit 0" claims verified
      SYNTAX only; a REAL 10GB-heap check found ONE missing import from 5.2's
      own wiring (fixed; runtime specs missed it via short-circuit evaluation)
      plus pre-existing debt in untouched route regions (recorded, not this
      ledger's scope) and two stale spec fixtures (fixed).
- [x] 5.3 Escalation telemetry: `{family, tier, reason}` on every escalation;
      zero silent escalations (spec: an escalation without a reason string is a
      thrown error, not a default). Done 2026-08-17: the checker verdict
      contract gained a machine `reason` slug (lookup_claim / citation_claim /
      count_claim — the route's three checkers set them); the runner THROWS on a
      reasonless failing verdict when an escalation is configured (spec'd — the
      exact never-default severity the 0.14 field comment demanded); the
      escalation result carries the reason, and the route records
      `{family, escalationTier: 3, escalationReason}` into the 0.14 metrics
      fields on all three agent outcomes plus the escalation story into the turn
      trace's checkerVerdicts (`escalation: reason:resolved|refused`). 567
      assistant tests green; my session's files ZERO errors under the REAL
      10GB-heap typecheck (pre-existing debt classes unchanged, recorded at
      5.2).
- [x] 5.4 Tier-3 slug chosen by a mini price-the-route + deck-subset run
      (mid-tier candidates only; frontier excluded by rule); decision + runs
      recorded in the scorecard. Done 2026-08-17 (scorecard P5.4): priced the
      $0.3–3/M tools-capable field live, ran the cs-family subset (the
      escalation ladder's home turf per P0.17) at k=3 under the standard pins on
      two qualifying challengers. DECISION: incumbent pro-0813 RETAINED —
      z-ai/glm-4.6v tied it (18/19 cs) at ~4× lower price and aced the general
      honesty trio, but measured 2/5 on the hardest md refusal/fabrication
      traps, tier-3 serves md too, and tier-3 volume is ~zero since P4.6
      (checkers silent) so the price delta buys ~nothing while full-deck
      evidence wins; minimax-m3 only matched the champion (17/19);
      qwen3.5-plus-20260420 disqualified by the fp8 pin (single unknown-quant
      endpoint). Registry entry re-stamped with the decision + the named
      evidence to reopen. Challenge spend $0.137.
- [x] 5.5 Full deck k=3 with ladder on: publish the effective-pass^k-vs-cost
      curve (tier-1-only vs laddered); escalation rate < stated threshold;
      scorecard + ratchet updated. Done 2026-08-17 (scorecard P5.5): escalation
      rate **0/366 runs**, under the pre-registered <2% threshold; the curve is
      honestly a POINT — zero checker firings → zero tier-2/tier-3 activations →
      laddered cost == tier-1-only cost ($0.1424 vs $0.1363, the delta is
      provider price movement with 0 extra calls). The ladder prices at zero
      until a checker fires twice on one turn, which the P4.6 recalibration made
      rare by design. Floor sheet 8/10 with md 38/41 (92.7%, NEW BEST) and
      general 9/9; the audit 3/7 miss decomposed honestly (three 1–2/3 draws of
      the known mark-reluctance flake + one empty-reply provider artifact — not
      ladder- or checker-caused: zero firings and audit is not
      escalation-enabled). Floors + hash unchanged.
- [x] 5.6 **P5 exit gate:** budgets still authoritative above all tiers (spec);
      curve + thresholds in the scorecard. Done 2026-08-17: the budget refusal
      spec sharpened into the above-all-tiers witness — an over-budget turn is
      refused with ZERO provider calls (asserted on the scripted provider's
      request count, not inferred from the 429), so tier 1, the tier-2
      corrective iteration, and the tier-3 escalation are all structurally
      unreachable behind an exhausted budget. Curve + the <2% escalation
      threshold recorded in the scorecard at P5.5 (rate measured 0%). 47 route
      tests green; P5 spend ~$0.28 (challenge + laddered deck).

## P6 — Distillery pass 1 (closes SMX-G9)

- [x] 6.1 Failure harvest: from telemetry + P5 escalation queue + deck
      near-misses, write the failure taxonomy for the worst two families into
      the deck sources doc. Done 2026-08-17 (deck sources §P6.1): harvested
      eleven k=3 draws + telemetry; the escalation queue is EMPTY (0/366 —
      recorded, nothing to harvest). Worst two ADDRESSABLE families (docs/
      workbench are environment-blocked): **audit** — ONE dominant shape, 13/14
      failures are mark-reluctance on the confirm turn (audit_status called
      instead of audit_mark; mechanism hypothesis: the status-before-continuing
      rule is rehearsed every turn and beats the mark under depth) + 1
      empty-reply artifact; **capability-smalltalk** — register leakage, five
      sub-shapes of one class (freebot compliance, flowid builder-voice,
      purchase/upgrade/plan upsell on deferred pleas, no-metis denial), with the
      structural note that cs got NO skill at 2.7 so the register has nowhere to
      carry an exemplar — 6.2 authors the family's first skill.
- [x] 6.2 Exemplar authoring: ≤2 verified exemplars per skill for the worst
      families — specified tool sequences, not roleplay; each exemplar's deck
      cases named; budgets hold (registry spec). Done 2026-08-17: AUDIT gained
      its confirm-turn exemplar (the very next call is audit_mark — no status
      first; targets named in the comment + evalCaseIds gained ledger-130);
      CAPABILITY-SMALLTALK gained its FIRST skill (the 2.7 "nothing to move" gap
      the P6.1 harvest exposed) — four register sentences + two specified
      exchanges (deferred-plea → release truth with commerce vocabulary absent;
      role-override → decline in Lilith's own voice, with the assigned-persona
      token DELIBERATELY absent from the exemplar so the forbidden string never
      enters the prompt). NEW CONTRACT: `surface: 'full'` marks a prompt-only
      skill (cs keeps the plain full toolset per the 2.8 design); the allowlist
      validator enforces the pairing both ways and the route's scoper honors it
      (spec'd). Budgets hold (registry spec green — 700/skill). Hash re-stamped
      d0ab54e0… (7→8 skills, skill bytes 3,264→4,567); behavior measured at 6.4.
      568 assistant tests green.
- [x] 6.3 Escalation-exhaust distillation: for turns that passed only at tier
      2/3, distill the failed-vs-passed delta into skill/exemplar edits; each
      edit's before/after deck delta recorded. Done 2026-08-17 — the exhaust is
      EMPTY, measured: zero tier-2 corrections and zero tier-3 escalations
      across every laddered run since the P4.6 recalibration (0 checker firings
      / 877 traced turns across P5.5 + P4.6-round-2; the P5.5 scorecard row
      carries the rate). No turn passed "only at tier 2/3" because no turn
      reached tier 2 — there is no failed-vs-passed delta to distill, and
      inventing edits from an empty exhaust would be the exact
      unmeasured-sentence class the registry forbids. The 6.2 exemplars
      (distilled from the DECK exhaust instead — the eleven-draw harvest) carry
      this pass; revisit when live traffic produces a non-empty exhaust (the
      telemetry from 5.3 is the collector).
- [x] 6.4 Full deck k=3: measured family deltas attributable to exemplars
      (paired vs P5 scorecard); ratchet floors raised where earned. Done
      2026-08-17 with a NEGATIVE result, fully measured (scorecard P6.4, five
      rounds, ~$0.35): the k=3 deck moved within noise; the k=10 target battery
      showed two cases apparently degraded; two revision rounds failed to
      recover them; then the BYTE-IDENTICAL CONTROL (the whole 6.2 authoring
      reverted — hash recomputed to the P3.4 stamp 79bc1f8c EXACTLY) showed the
      same degradation with zero authored bytes, FALSIFYING every skill
      attribution; the same-minute k=3 discriminator ran entirely on Baidu and
      exposed the real confound — `adv-injection-role-override` and
      `ledger-129-id-hygiene` are PROVIDER-MIX-SENSITIVE (0/3 at P0 on this mix,
      flaky through mixed draws, sick on tonight's Baidu-priced route): the
      fp8+price pins do NOT pin behavior for such cases; per-case servedBy
      provenance is the missing control variable (recorded for P7's tournaments
      — pair arms by ENDPOINT). Outcome: no attributable exemplar delta in
      either direction; bytes stand at the P3.4 stamp (P5.5 floors valid by
      construction); floors not raised (nothing earned); the P6.1 taxonomies
      await a mechanism, not prose. 568 assistant tests green incl. the hash
      gate on the restored stamp.
- [x] 6.5 **P6 exit gate:** every skill has a changelog line, live case links,
      and a scorecard entry; no family regressed. Done 2026-08-17: all seven
      registered skills carry changelogs (the audit skill carries the full
      6.2→6.4 authored-measured-reverted history — negative results are
      changelog lines too); every evalCaseIds trace resolves against the real
      deck (the 2.5 gate, green in CI); the P6 record spans the deck sources
      (§P6.1 taxonomies) and the scorecard (§P6.4 five-round measurement). NO
      FAMILY REGRESSED — by the strongest construction available: the serving
      bytes are byte-identical to the P3.4 stamp P5.5 measured (hash recomputed,
      79bc1f8c), so every family stands exactly at its P5.5-measured state. P6's
      net product: two failure taxonomies, the provider-sensitivity finding (P7
      must pair arms by endpoint), the surface-full contract machinery, and the
      measured knowledge that this model's register/sequence failures are
      prose-resistant. P6 spend ~$0.35; 17 gate tests green.

## P7 — Downshift tournament (finishes SMX-G7)

- [x] 7.1 Challenger slate priced by ROUTE (live `/endpoints` + usage-include
      probes, per A10): sub-flash tier for judge (and router if 2.3 said yes),
      flash-class alternates for turn, mid-tier for escalation; table committed
      with dates and commands. Done 2026-08-17 (scorecard P7.1): turn alternates
      = qwen3-30b-a3b-instruct-2507 (SiliconFlow fp8, $0.09/$0.30) and
      z-ai/glm-4.7-flash (Venice fp8, $0.06/$0.40), both probe-verified serving
      and billing on the priced route; the cheap nano/8B tail is DISQUALIFIED
      wholesale by the fp8 pin (unknown/fp4 endpoints). Judge: the sub-flash
      tier is EMPTY under fp8+tools+structured pins — cheapest qualified is
      flash-class gpt-oss-120b (Mancer 2); the judge-pin question deferred to
      7.2. Router: excluded per 2.3's recorded no. Escalation: the 5.4 table
      stands. Probes ~$0.00001.
- [ ] 7.2 Judge leg tournament: candidates graded against the human-labeled
      transcript set (P8's 8.1 labels can be pulled earlier if needed — record
      the dependency); agreement thresholds in the scorecard; winner bound in
      the registry. `blocked:human` - NOTE (2026-08-17, box left OPEN —
      human-blocked): the judge tournament grades against ≥40 HUMAN-LABELED
      transcripts (8.1's set, pullable early) — no labeled set exists and
      labeling is the operator's own act, not something this harness may
      fabricate. The dependency is hereby recorded per the row's own
      instruction; everything downstream of labels (candidate slate,
      structured-output plumbing, agreement math via eval-stats Wilson helpers)
      is ready. - NOTE (2026-09-18): this note was its own `- [ ] 7.2` line
      until today, which made one task read as two open boxes on the task board.
      Same human act as task 12.3 of `EVE_SOTA_GAP_CLOSURE_TODOS_2026-09-01.md`.
- [x] 7.3 Turn leg tournament: paired full-deck runs, k=10, fp8, sort:price;
      promotion only if no ledger-born lock fails, every family's Wilson lower
      bound ≥ floor, and cost + p95 deltas are stated; decision table committed.
      Done 2026-08-18 (scorecard P7.3, staged protocol pre-registered): the
      34-lock k=10 round eliminated BOTH challengers before full-deck pairs were
      owed — qwen3-30b-2507 fails three champion-perfect locks AND bills 2.4×
      the champion (SiliconFlow serves ~no cache reads: the sticker-price table
      cannot see the champion's ~90% cache moat — the A10 method gains a
      cached-cost step); glm-4.7-flash is better on SIX locks, cheaper billed
      (93.5% cache on Venice) and 1.7× faster, but REGRESSES three
      champion-perfect locks (navigate claim-order ×2 + envelope-after-tools) —
      a lock regression re-opens a shipped defect, so the gate holds: NO
      PROMOTION; glm is the named next-review candidate with the reopen evidence
      recorded. Cost/p95 deltas: served curve unchanged from P5.5 (no
      promotion). Round-1 spend $0.574; round 2 moot.
- [x] 7.4 Escalation leg re-check under the winning turn leg (the ladder's value
      depends on the gap between tiers); curve re-published. Done 2026-08-18:
      the winning turn leg IS the incumbent champion (7.3: no promotion), so the
      tier gap is unchanged and the P5.4 escalation decision + P5.5 curve stand
      as published — the ladder still prices at zero until a checker fires
      twice, with the pro-0813 slug retained and glm-4.6v's reopen evidence
      named. Nothing to re-measure: the re-check condition ("under the winning
      turn leg") is satisfied by identity.
- [x] 7.5 **P7 exit gate:** every leg on the cheapest model passing protocol;
      registry pins + scorecard + ratchet coherent; any leg that did NOT
      downshift has its blocking evidence recorded (honest-gap pattern). Done
      2026-08-18: TURN = the champion is the cheapest model passing protocol —
      both sticker-cheaper challengers failed the lock gate and one BILLED 2.4×
      more (the cache-moat finding); ESCALATION = pro-0813 retained on 5.4's
      measured challenge (glm-4.6v reopen evidence named); JUDGE = provisional
      pin, tournament HUMAN-LABEL-BLOCKED (the honest gap, recorded at 7.2's
      open note — the only reopen key is the operator's ≥40-transcript labeled
      set); EMBEDDING = unbound, no consumer (5.1's fail-loud seam); ROUTER =
      excluded by 2.3's measured no. Registry pins, scorecard, and ratchet
      coherent (hash 79bc1f8c everywhere; 14 gate tests green). P7 spend ~$0.58.
      The 7.2 box remains open by design — it is the recorded gap, not an
      omission.

## P8 — Sustainment (closes SMX-G12 + drift)

- [ ] 8.1 Rubric judge validated: ≥40 human-labeled transcripts, agreement
      threshold met and recorded; below-threshold classes stay advisory;
      battery-derived `advisory` cases (0.11) promoted to graded where the judge
      qualifies. `blocked:human` - QUEUE NOTE (2026-08-23): the operator
      committed to labeling the 42-transcript sheet blind (EVE_EVERYWHERE annex
      decision). When the sheet's VERDICT/WHY lines are filled: run the registry
      judge over `docs/audits/eve-smx-judge-transcripts.jsonl`, compute
      per-class raw agreement and the Wilson lower bound (eval-stats helpers),
      record the table in the scorecard, flip 7.2/8.1 only for classes meeting
      ≥0.85 raw and ≥0.70 Wilson-lower, and promote the qualifying 0.11 battery
      cases. Nothing here may author a label. - NOTE (2026-08-18, box left OPEN
      — human-blocked, same class as 7.2): "validated" means agreement against
      ≥40 HUMAN-LABELED transcripts, and no labeled set exists — labeling is the
      operator's own act, which this harness may not fabricate (a judge scored
      against labels the harness authored would be exactly the fabricated
      success the quality bar bans). Everything downstream of labels is staged:
      the ten battery conversations already run advisory-and-reporting under the
      deck (0.11), the agreement math exists (eval-stats Wilson helpers, spec'd
      at 0.2), and the judge leg's provisional pin is in the registry with the
      human-label blocker recorded in `chosenBy` (P5.1/7.5). The operator-facing
      labeling protocol (what to label, where labels live, the agreement
      threshold to meet) goes in the 8.3 maintenance runbook so this gap is
      actionable, not vague. UPDATE 2026-08-19: the corpus is now CAPTURED — 42
      clean battery conversations (10 cases, k=4 + an outage top-up of
      tara-sleep/ nyx-tonight at k=5, champion at the serving pins) in
      `docs/audits/eve-smx-judge-transcripts.jsonl`, with the blank labeling
      sheet at `docs/audits/eve-smx-judge-labels.SHEET.md` and the repeatable
      capture tool committed (`battery-transcripts.eval.ts`, merge/top-up mode
      included). The ONLY remaining act is the operator filling the sheet's
      VERDICT/WHY lines, blind. The harness did not and will not author a single
      verdict. - NOTE (2026-09-18): the two notes above were their own
      `- [ ] 8.1` lines until today, which made one task read as three open
      boxes on the task board. Same human act as task 12.3 of
      `EVE_SOTA_GAP_CLOSURE_TODOS_2026-09-01.md`.
- [x] 8.2 Held-out set: fresh cases authored after P6 skills, never tuned
      against, reported in a separate scorecard section every run. Done
      2026-08-18: 13 fresh cases (`deck-heldout-cases.ts`, `heldOut` meta flag)
      across md/nav/docs/tour/cs/general, authored post-P6 (hash 79bc1f8c) with
      phrasings and worlds no deck case uses. Never-tuned is MECHANICAL, not
      promised: admission refuses lock/advisory+heldOut, no skill may trace to a
      held-out id (spec), the misroute audit skips them, and the runner reports
      them in their own non-gating section every run. Baseline (champion, fp8,
      sort=price, k=10, 130 runs, 0 retries): 12/13 at 10/10; the miss
      (heldout-docs-internals-absent 0/10) is a REAL finding — the champion
      asserts docs-absence for self-evidently-internal topics WITHOUT calling
      search_docs (080's shape in the docs family, outside the lookup-claim
      checker's reach) — recorded in the scorecard, not repaired, per the
      held-out contract. Diagnostic re-run $0.0019/10 runs StreamLake 97.0%
      cache; sweep ~$0.025 estimated (its spend line was lost to a runner output
      crash on heldout-only selections, fixed). Touched-file typecheck (real
      check, --noCheck false) + eslint clean; deck/harness/skills specs 113
      green.
- [x] 8.3 Maintenance runbook `docs/agents/eve-smx-maintenance.md`: weekly loop
      (cost report → escalation queue → distillery edits → deck run → ratchet),
      monthly route-price note, demotion triggers (telemetry floor breaks →
      registry rollback protocol). Done 2026-08-18: runbook written and
      fact-checked against the code (package filter `@oshun/bff`,
      judge/escalation slug `deepseek/deepseek-v4-pro-0813`, outcome vocabulary
      `provider_error`/ `refused` — three initial-draft errors corrected before
      commit). Weekly loop grounded in the real instruments (P0.15 cost-report
      tool, admin metrics/telemetry endpoints, P5.3 reason slugs, P6 distillery
      discipline, P2.11 run recipes, P0.19 hash gate); plus held-out rotation
      rules (8.2), the P7 cached-cost pricing correction + glm-4.7-flash reopen
      conditions, demotion triggers with the env-override-first rollback
      protocol, and the pre-registered judge-validation protocol (labeling
      corpus, blind-label order, per-class agreement bar ≥0.85 raw / Wilson
      lower ≥0.70) that makes the 7.2/8.1 human gap operator-actionable.
- [x] 8.4 Drift alarms: scorecard staleness check (hash gate already fails
      closed) + a metrics-endpoint floor check the operator can run; wired into
      the cost report tool. Done 2026-08-18. Staleness: the hash-gate spec now
      also fails closed on STORY drift — the scorecard must name the ratchet's
      current hash prefix, so a re-stamp without its scorecard record reds CI
      (plus a provenance gate on telemetryFloors: a floor that cannot name its
      measurement is an invented gate). Floor check:
      `eve-smx-cost-report.mjs --check-floors` compares every serving route's
      cache-read rate from GET /v1/assistant/metrics against `telemetryFloors`
      in the ratchet (stamped 0.70 from the champion route's measured 85.2–97.0%
      range; the P7.3 qwen no-cache route read 62% — the collapse shape this
      catches), renders pass/breach/unmeasured per row (unmeasured is LOUD,
      neither pass nor breach), and exits 3 on breach for cron wrappers. Only
      measured dimensions are stamped — an unstamped dimension reports as an
      honest gap. Runbook weekly-loop step 1 rewired. Tool tests 11/11
      (`npx tsx --test`), hash-gate spec 7/7.
- [x] 8.5 **P8 exit gate / initiative close:** success criteria 1–4 from the
      design doc each verified with a named artifact; remaining gaps listed
      honestly or the initiative is not closed. Done 2026-08-18 (scorecard
      §P8.5): each criterion audited against its artifacts — Quality MET WITH
      GAPS (champion 91/121 exceeds arm B's 89/121; floors ≥ arm B in 9/10
      families; but the 211/212 lock trio is unreachable-by-construction without
      the P2 workbench fixture and six locks are provider-mix flaky at k=10),
      Cost MET (every arm's billed cost + cache rate published, deltas
      mechanism-attributed, checkers and ladder measured at zero marginal
      spend), Downshift MET WITH GAPS (all serving legs on the cheapest
      protocol-passing model; judge provisional and embedding honestly unbound),
      Sustainment MET WITH GAPS (four independent decay guards; judge
      unvalidated, first live weekly cycle not yet run). Six standing gaps
      enumerated, none hidden — 7.2/8.1 human labels, the workbench fixture, the
      flaky-lock route mechanism, the cs register floor, the V1.2
      release-blocked seven, launch-traffic medians. Initiative CLOSED; total
      spend ≈ $3.73. The 7.2 and 8.1 boxes remain open by design as the recorded
      human gaps.