# HITL-005 evidence pack — measurable human-authority SLOs

## Work-package charter

- **Task ID:** `HITL-005`
- **Owner:** Oshun BFF, creative-autonomy, Agentic Studio, Lilith, Yemaya, and
  Studio Authoring maintainers; implementation owner: Codex
- **Accountable product:** V1 Oshun autonomous-content platform
- **Priority:** P0
- **Target milestone:** 2026-07-15 SOTA gap-closure program
- **Risk tier:** Critical human-authority, rollback, and autonomy-escalation
  risk
- **Dependencies:** RUN-001 through RUN-005 and HITL-001 through HITL-004
- **Affected runtime:** V1 AgentRun lifecycle and V1 creative-autonomy BFF
- **Rollback strategy:** Revert the source commit. The new SLO snapshot uses its
  own versioned durable key, and category-autonomy snapshot restoration retains
  the prior configured value when the new field is absent.
- **Evidence-pack location:** This file

## Canonical durable measurement plane

`HumanAuthoritySloStore` is the single V1 measurement and autonomy-gate
authority. The BFF restores it from durable snapshot key `human-authority-slo`
before wiring AgentRun lifecycle behavior, and every mutation emits a new
snapshot. Stable observation IDs make latency, activity, edit, appeal, drill,
and autonomy-raise records idempotent across duplicate delivery and restart.

The operator-only `GET /v1/agentic/autonomy/authority/slo` endpoint produces a
point-in-time report from this durable measurement state plus canonical review
queue and run-control histories. The report exposes its targets, values,
denominators, readiness, and blockers rather than returning an opaque health
flag.

## Metric definitions and targets

| Metric                                    | Canonical definition                                                                                                                                                                                                                           | Target                                           |
| ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ |
| Takeover acknowledgment latency           | Milliseconds from entry into `yemayaTakeover` until the pause, checkpoint, override, and durable run mutation complete; both success and failure are observed.                                                                                 | 100% successful and p95 at or below 2,000 ms     |
| Checkpoint-resume latency                 | Milliseconds from entry into `yemayaHandback` until checkpoint validation, override revocation, Yemaya handback, Agentic resume, and durable mutation complete; both success and failure are observed.                                         | 100% successful and p95 at or below 30,000 ms    |
| Human active minutes                      | Cardinality of unique `(actor ID, UTC minute bucket)` pairs with a canonical authoring activity. Activity kinds are draft open, autosave, checkpoint, branch create/update, comment, checklist signoff, lifecycle transition, and replacement. | Measured, no synthetic session-duration estimate |
| Comments                                  | Canonical comment activities divided by distinct review items.                                                                                                                                                                                 | Reported count, denominator, and rate            |
| Edit distance                             | Learner token diff for each human replacement: mean and p95 changed fraction plus total added and removed tokens.                                                                                                                              | Reported from real replacement inputs            |
| Replacement, regeneration, veto, rollback | Matching canonical review audit events divided by distinct review items. Regeneration is the existing `consume-tweak` event.                                                                                                                   | Reported count, denominator, and rate            |
| Fork                                      | Canonical Agentic Studio `fork` control events divided by distinct runs.                                                                                                                                                                       | Reported count, denominator, and rate            |
| Appeal                                    | Durable, deduplicated appeal intake divided by distinct review items. The rationale is retained only as SHA-256 in this measurement plane.                                                                                                     | Reported count, denominator, and rate            |
| Correct edit incorporation                | Replacements for which every canonical assertion passes divided by all measured replacements.                                                                                                                                                  | 100%                                             |

Latency percentiles use the nearest-rank method over successful observations;
empty samples do not pass. The report also returns p50, p95, maximum, sample
count, and success rate so operators can distinguish missing data from a fast
healthy path.

## Correct-incorporation proof

A replacement is first recorded as not incorporated. Only after the review queue
commits the human variant does the HTTP transaction update the same observation.
It is correct only when all of these independent assertions are true:

1. The merged authoring document digest equals the output's document digest.
2. The variant run output equals the intended human output.
3. The canonical RUN-001 artifact SHA-256 equals the replacement SHA-256.
4. The transactional intervention contains the matching `merge-human-branch`
   operation.
5. Every required V1 compellingness gate verdict was recorded.
6. The replacement has signed mixed human/agent lineage.
7. The review queue's committed variant equals the canonical replacement and its
   newest audit action is `human-replacement`.

This makes the percentage a cross-store correctness measurement, not merely a
count of successful HTTP responses.

## Recurring-override policy review

Every allowed Lilith decision whose recurrence threshold requires review is
projected into a durable policy-review record with its route, surface, policy
check, tenant, threshold, recurrence count, and exact matched override IDs. The
identity includes the matched occurrence set, so a later recurring set cannot
silently reuse an earlier resolution.

An open recurring-override review blocks an autonomy increase. Resolution is an
operator-only action at
`POST /v1/agentic/autonomy/authority/policy-reviews/:reviewId/resolve`; it
requires non-empty rationale and records reviewer, timestamp, and rationale
SHA-256. A second resolution is rejected.

## Operational autonomy drill gate

`runHumanAuthorityDrillSuite` always executes the six required checks, captures
duration, catches failures, and requires at least one evidence reference for a
check to succeed. Only the latest suite is eligible. It must be no more than 30
days old, evidence-bearing, and exactly **6/6** successful. The focused store
test proves **5/6** blocks and a later **6/6** suite passes; it also proves an
otherwise-passing suite becomes ineligible when stale.

The route integration uses real local state machines for each drill:

| Drill              | Executed proof                                                                                                                                       |
| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| Takeover           | AgentRun Yemaya takeover creates a checkpoint-backed override; handback resumes the run.                                                             |
| Restart            | A paused takeover snapshot is JSON-cloned into a fresh lifecycle store, which restores the paused status and takeover owner.                         |
| Duplicate delivery | The same RUN-001 transactional approval command is delivered twice; the second result is marked duplicate and the effect journal remains length one. |
| Stale approval     | Dependencies mutate after approval; execution with the old approval fails with `stale-approval`.                                                     |
| Authorization      | A customer attempts a transactional approval and receives `unauthorized-intervention`.                                                               |
| Rollback           | A canonical review item is approved, published, and rolled back to the terminal `rolled-back` state.                                                 |

The operator-only `PUT /v1/agentic/autonomy/categories/:categoryId/autonomy`
runtime path permits only a one-level increase and requires rationale. It calls
the SLO gate before mutating the live category used by the director. A
missing/failed/stale drill suite or any open recurring-override review
returns 409. A passing suite plus resolved reviews records a durable,
rationale-hashed autonomy-raise audit and persists the new category level. The
route test proves blocked-before-drills, blocked-after-6/6-while-review-open,
successful-after-resolution, and restart restoration of the raised level.

## Automated verification

- Human-authority SLO store: **4/4** passing, including every metric formula,
  recurring-review closure, **5/6** fail-closed behavior, **6/6** readiness,
  freshness expiry, restart, and observation deduplication.
- Autonomy route: **8/8** passing, including real six-drill execution, appeals,
  recurring-review blocking/resolution, category raise auditing, and restart.
- AgentRun lifecycle store: **7/7** passing, including measured takeover and
  checkpoint-resume latencies.
- V1 AgentRun lifecycle HTTP integration: **8/8** passing, including global SLO
  observation of recurring Lilith review and Yemaya latency.
- BFF autonomy bindings: **28/28**; bound autonomy-plane restart/end-to-end:
  **6/6**.
- Additional BFF regression: AgentRun HTTP routes **14/14**, OpenAPI runtime
  drift **1/1**, and OpenAPI coverage **3/3**.
- Changed-source ESLint completed with zero errors and zero warnings; BFF
  production build completed successfully; `git diff --check` passed.
- Repository-standard BFF baseline-ratcheted typecheck reported zero new
  app-owned errors against a zero-error baseline. This official gate completed
  after the final source edits.

## Artifact identity

| Artifact                                                   | SHA-256                                                            |
| ---------------------------------------------------------- | ------------------------------------------------------------------ |
| `apps/oshun/bff/src/agentic/human-authority-slo-store.ts`  | `6be55ecf795481813f87c747cbc7ce482507366bb33218204505ab6eab8512a9` |
| `apps/oshun/bff/src/agentic/autonomy-route.ts`             | `c681c08d24b144ef84553b595ec6112c31cb26643a41c63dfb5cb1164c7e75dc` |
| `apps/oshun/bff/src/agentic/autonomous-authoring-store.ts` | `0b00616416a157522aba1690da95074f1bc1e082cc4dc8e5097453a4dac5c170` |
| `apps/oshun/bff/src/agentic/run-lifecycle-store.ts`        | `c8c96b008d41faddffedcd815a696d084ec3e341f7d892e94cb43a9265d1642c` |
| `apps/oshun/bff/src/server.ts`                             | `81fbd41be5f559b381ddc76c4b9f52845382d4371ed768c9bf7d60a99538a542` |

## Resource-safe verification

The final probe reported 15 GiB RAM, 31 GiB swap, and approximately 38 GiB free
disk. Focused Vitest gates used one worker and expensive checks ran
sequentially. The final BFF ratcheted typecheck used its documented 7 GiB heap;
its TypeScript process peaked near 5.1 GiB RSS while available memory remained
above 8 GiB. No broad Nx, Next, Expo, Tauri, or workspace-wide sweep was run.

## Scope and negative proof

- This package changes authenticated backend/control behavior, not rendered web,
  PWA, admin, mobile, or Unreal UI. The frontend skill, Playwright UI
  automation, mobile automation, and Unreal validation are therefore
  inapplicable to HITL-005.
- The SLO report measures durable local observations and canonical histories; it
  does not claim external APM ingestion or production traffic volume.
- Latency and correct-incorporation targets are visible SLO health indicators.
  Per the task wording, the enforced precondition for raising autonomy is fresh
  **100% drill success** plus closure of every recurring-override policy review.
  Empty traffic samples therefore do not silently fabricate latency or edit
  compliance, but they are not additional rollout blockers beyond that stated
  authority gate.

## Publication identity

- **Measurable authority source commit:**
  `1de8d8948b84bfd16d21ad9475851a2e4d7fa7aa`
- **Closure evidence commit:** `183f670ea269a43ecaa496936b9c223b21ba18b1`
