Work-package charter#
- Task ID:
HITL-005 - Owner: Oshun BFF, creative-autonomy, Agentic Studio, Lilith, Yemaya, and Studio Authoring maintainers; implementation owner: Codex
- Accountable product: V1 Oshun autonomous-content platform
- Priority: P0
- Target milestone: 2026-07-15 SOTA gap-closure program
- Risk tier: Critical human-authority, rollback, and autonomy-escalation risk
- Dependencies: RUN-001 through RUN-005 and HITL-001 through HITL-004
- Affected runtime: V1 AgentRun lifecycle and V1 creative-autonomy BFF
- Rollback strategy: Revert the source commit. The new SLO snapshot uses its own versioned durable key, and category-autonomy snapshot restoration retains the prior configured value when the new field is absent.
- Evidence-pack location: This file
Canonical durable measurement plane#
HumanAuthoritySloStore is the single V1 measurement and autonomy-gate
authority. The BFF restores it from durable snapshot key human-authority-slo
before wiring AgentRun lifecycle behavior, and every mutation emits a new
snapshot. Stable observation IDs make latency, activity, edit, appeal, drill,
and autonomy-raise records idempotent across duplicate delivery and restart.
The operator-only GET /v1/agentic/autonomy/authority/slo endpoint produces a
point-in-time report from this durable measurement state plus canonical review
queue and run-control histories. The report exposes its targets, values,
denominators, readiness, and blockers rather than returning an opaque health
flag.
Metric definitions and targets#
| Metric | Canonical definition | Target |
|---|---|---|
| Takeover acknowledgment latency | Milliseconds from entry into yemayaTakeover until the pause, checkpoint, override, and durable run mutation complete; both success and failure are observed. |
100% successful and p95 at or below 2,000 ms |
| Checkpoint-resume latency | Milliseconds from entry into yemayaHandback until checkpoint validation, override revocation, Yemaya handback, Agentic resume, and durable mutation complete; both success and failure are observed. |
100% successful and p95 at or below 30,000 ms |
| Human active minutes | Cardinality of unique (actor ID, UTC minute bucket) pairs with a canonical authoring activity. Activity kinds are draft open, autosave, checkpoint, branch create/update, comment, checklist signoff, lifecycle transition, and replacement. |
Measured, no synthetic session-duration estimate |
| Comments | Canonical comment activities divided by distinct review items. | Reported count, denominator, and rate |
| Edit distance | Learner token diff for each human replacement: mean and p95 changed fraction plus total added and removed tokens. | Reported from real replacement inputs |
| Replacement, regeneration, veto, rollback | Matching canonical review audit events divided by distinct review items. Regeneration is the existing consume-tweak event. |
Reported count, denominator, and rate |
| Fork | Canonical Agentic Studio fork control events divided by distinct runs. |
Reported count, denominator, and rate |
| Appeal | Durable, deduplicated appeal intake divided by distinct review items. The rationale is retained only as SHA-256 in this measurement plane. | Reported count, denominator, and rate |
| Correct edit incorporation | Replacements for which every canonical assertion passes divided by all measured replacements. | 100% |
Latency percentiles use the nearest-rank method over successful observations; empty samples do not pass. The report also returns p50, p95, maximum, sample count, and success rate so operators can distinguish missing data from a fast healthy path.
Correct-incorporation proof#
A replacement is first recorded as not incorporated. Only after the review queue commits the human variant does the HTTP transaction update the same observation. It is correct only when all of these independent assertions are true:
- The merged authoring document digest equals the output's document digest.
- The variant run output equals the intended human output.
- The canonical RUN-001 artifact SHA-256 equals the replacement SHA-256.
- The transactional intervention contains the matching
merge-human-branchoperation. - Every required V1 compellingness gate verdict was recorded.
- The replacement has signed mixed human/agent lineage.
- The review queue's committed variant equals the canonical replacement and its
newest audit action is
human-replacement.
This makes the percentage a cross-store correctness measurement, not merely a count of successful HTTP responses.
Recurring-override policy review#
Every allowed Lilith decision whose recurrence threshold requires review is projected into a durable policy-review record with its route, surface, policy check, tenant, threshold, recurrence count, and exact matched override IDs. The identity includes the matched occurrence set, so a later recurring set cannot silently reuse an earlier resolution.
An open recurring-override review blocks an autonomy increase. Resolution is an
operator-only action at
POST /v1/agentic/autonomy/authority/policy-reviews/:reviewId/resolve; it
requires non-empty rationale and records reviewer, timestamp, and rationale
SHA-256. A second resolution is rejected.
Operational autonomy drill gate#
runHumanAuthorityDrillSuite always executes the six required checks, captures
duration, catches failures, and requires at least one evidence reference for a
check to succeed. Only the latest suite is eligible. It must be no more than 30
days old, evidence-bearing, and exactly 6/6 successful. The focused store
test proves 5/6 blocks and a later 6/6 suite passes; it also proves an
otherwise-passing suite becomes ineligible when stale.
The route integration uses real local state machines for each drill:
| Drill | Executed proof |
|---|---|
| Takeover | AgentRun Yemaya takeover creates a checkpoint-backed override; handback resumes the run. |
| Restart | A paused takeover snapshot is JSON-cloned into a fresh lifecycle store, which restores the paused status and takeover owner. |
| Duplicate delivery | The same RUN-001 transactional approval command is delivered twice; the second result is marked duplicate and the effect journal remains length one. |
| Stale approval | Dependencies mutate after approval; execution with the old approval fails with stale-approval. |
| Authorization | A customer attempts a transactional approval and receives unauthorized-intervention. |
| Rollback | A canonical review item is approved, published, and rolled back to the terminal rolled-back state. |
The operator-only PUT /v1/agentic/autonomy/categories/:categoryId/autonomy
runtime path permits only a one-level increase and requires rationale. It calls
the SLO gate before mutating the live category used by the director. A
missing/failed/stale drill suite or any open recurring-override review
returns 409. A passing suite plus resolved reviews records a durable,
rationale-hashed autonomy-raise audit and persists the new category level. The
route test proves blocked-before-drills, blocked-after-6/6-while-review-open,
successful-after-resolution, and restart restoration of the raised level.
Automated verification#
- Human-authority SLO store: 4/4 passing, including every metric formula, recurring-review closure, 5/6 fail-closed behavior, 6/6 readiness, freshness expiry, restart, and observation deduplication.
- Autonomy route: 8/8 passing, including real six-drill execution, appeals, recurring-review blocking/resolution, category raise auditing, and restart.
- AgentRun lifecycle store: 7/7 passing, including measured takeover and checkpoint-resume latencies.
- V1 AgentRun lifecycle HTTP integration: 8/8 passing, including global SLO observation of recurring Lilith review and Yemaya latency.
- BFF autonomy bindings: 28/28; bound autonomy-plane restart/end-to-end: 6/6.
- Additional BFF regression: AgentRun HTTP routes 14/14, OpenAPI runtime drift 1/1, and OpenAPI coverage 3/3.
- Changed-source ESLint completed with zero errors and zero warnings; BFF
production build completed successfully;
git diff --checkpassed. - Repository-standard BFF baseline-ratcheted typecheck reported zero new app-owned errors against a zero-error baseline. This official gate completed after the final source edits.
Artifact identity#
| Artifact | SHA-256 |
|---|---|
apps/oshun/bff/src/agentic/human-authority-slo-store.ts |
6be55ecf795481813f87c747cbc7ce482507366bb33218204505ab6eab8512a9 |
apps/oshun/bff/src/agentic/autonomy-route.ts |
c681c08d24b144ef84553b595ec6112c31cb26643a41c63dfb5cb1164c7e75dc |
apps/oshun/bff/src/agentic/autonomous-authoring-store.ts |
0b00616416a157522aba1690da95074f1bc1e082cc4dc8e5097453a4dac5c170 |
apps/oshun/bff/src/agentic/run-lifecycle-store.ts |
c8c96b008d41faddffedcd815a696d084ec3e341f7d892e94cb43a9265d1642c |
apps/oshun/bff/src/server.ts |
81fbd41be5f559b381ddc76c4b9f52845382d4371ed768c9bf7d60a99538a542 |
Resource-safe verification#
The final probe reported 15 GiB RAM, 31 GiB swap, and approximately 38 GiB free disk. Focused Vitest gates used one worker and expensive checks ran sequentially. The final BFF ratcheted typecheck used its documented 7 GiB heap; its TypeScript process peaked near 5.1 GiB RSS while available memory remained above 8 GiB. No broad Nx, Next, Expo, Tauri, or workspace-wide sweep was run.
Scope and negative proof#
- This package changes authenticated backend/control behavior, not rendered web, PWA, admin, mobile, or Unreal UI. The frontend skill, Playwright UI automation, mobile automation, and Unreal validation are therefore inapplicable to HITL-005.
- The SLO report measures durable local observations and canonical histories; it does not claim external APM ingestion or production traffic volume.
- Latency and correct-incorporation targets are visible SLO health indicators. Per the task wording, the enforced precondition for raising autonomy is fresh 100% drill success plus closure of every recurring-override policy review. Empty traffic samples therefore do not silently fabricate latency or edit compliance, but they are not additional rollout blockers beyond that stated authority gate.
Publication identity#
- Measurable authority source commit:
1de8d8948b84bfd16d21ad9475851a2e4d7fa7aa - Closure evidence commit:
183f670ea269a43ecaa496936b9c223b21ba18b1