# LEARN-001 evidence pack — outcome-grounded improvement loop

> **Status: repository-complete / closed.** The shared contract and Agentic
> Studio runtime now implement the complete evidence-to-improvement lifecycle,
> including exact attribution, hard-case sampling, versioned diagnosis and
> proposals, non-compensatory evaluation, human approval, staged promotion,
> automatic rollback, and dependent-artifact recertification. This closure does
> not claim a live telemetry deployment or a promoted production proposal.

## Work-package charter

- **Task ID:** `LEARN-001`
- **Owner:** V1-V9 evaluation governance, Agentic Studio feedback/rollout, and
  creative-autonomy maintainers; implementation owner: Elikem Agudogo
- **Accountable product:** shared V1–V9 autonomous-content learning and
  evaluation plane, with Agentic Studio as the invoked lifecycle runtime
- **Priority:** P0-G/P1
- **Target milestone:** 2026-07-15 SOTA gap-closure program
- **Risk tier:** high human-authority, policy/autonomy promotion, regression,
  manipulation, and dependent-artifact release risk
- **Dependencies:** canonical evaluation and run/artifact attribution contracts;
  Agentic Studio feedback and paired champion/challenger evidence;
  creative-autonomy preference learning; real signal feeds, seven-dimension
  evaluators, human approval, rollout monitoring, and artifact recertification
  for production use
- **Affected schemas and content slots:** canonical learning-signal,
  diagnosis/proposal, evaluation, approval, rollout, promotion-event,
  dependent-artifact, and engagement-governance contracts plus Agentic Studio
  snapshot version 1 and feedback adapter. Any V1–V9 slot can be referenced by
  exact ID, but this work package changed no prompt, model output, rendered
  content, media byte, or released slot itself.
- **Linked task IDs:** `LEARN-001`, `LEARN-002`, `EVAL-001`, and `EVAL-004`
- **Audit finding:** creative autonomy already learned from ratings, edits,
  rejections, and vetoes; Agentic Studio already supplied held-out gold sets and
  paired champion/challenger statistics. No canonical layer joined those pieces
  into an exactly attributed, human-governed, regression-safe system-change
  lifecycle.
- **Repository objective:** make improvement proposals traceable to outcomes,
  test them on every required quality/operational axis, prevent unsafe direct
  engagement optimization, and make rollout history replay-verifiable.
- **Rollout strategy:** additive canonical contracts and an exported,
  snapshot-safe Agentic Studio store/adaptor. Production composition must supply
  real signals, evaluators, approvals, monitoring, and recertification evidence.
- **Rollback strategy:** revert source commit
  `b1210f90706db87b94c82b0f1b2f78c6a4d629b2`.
- **Evidence-pack location:** this file.

## Closure matrix

| Ledger requirement                                                                                                                                  | Closure proof                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| --------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Collect explicit preference, edits, replacements, vetoes, gate failures, complaints, corrections, task outcomes, and approved contextual engagement | `LearningSignalKindSchema` enumerates all nine kinds. `OutcomeLearningLoopStore.collectSignal` parses, deduplicates, snapshots, and restores them. The production feedback adapter converts existing Agentic Studio feedback without inventing diagnostic scores.                                                                                                                                                                               |
| Attribute every signal to slot, run, step, artifact, prompt, model, provider, rubric, policy, source, intervention, and cohort                      | Every signal requires every named reference plus exact artifact and prompt SHA-256. The feedback adapter rejects run or artifact substitution. Missing or blank attribution fails schema admission.                                                                                                                                                                                                                                             |
| Sample hard, novel, high-impact, high-disagreement, drifted, and failed trajectories                                                                | `selectOutcomeLearningSignals` derives all six typed reasons from a versioned policy, deduplicates the population, orders deterministically by reason count/impact/ID, applies a bounded cap, and fails loud when nothing qualifies.                                                                                                                                                                                                            |
| Produce versioned natural-language diagnoses and proposed prompt, rubric, tool, routing, policy, or workflow changes                                | Diagnoses require versions, narratives, evidence-bound findings, and exact selected-signal coverage. Proposals cover the six requested change types plus autonomy, require base/candidate/diff/rollback identities, and are versioned contiguously by the runtime.                                                                                                                                                                              |
| Evaluate held-out capability, regression, safety, reliability, cost, latency, and subgroup suites                                                   | Every offline and rollout evaluation carries threshold-derived probes for all seven dimensions. The executable runner requires all seven providers. Missing, duplicate, failed, or substituted candidate evidence blocks advancement; contamination-control and held-out suite evidence are mandatory.                                                                                                                                          |
| Require human approval for policy/rubric/autonomy changes                                                                                           | Those proposal kinds cannot even parse with approval disabled. Advancement requires a signed, exact-version human decision; missing, rejected, machine-authored, or substituted approval blocks.                                                                                                                                                                                                                                                |
| Promote through offline, shadow, canary, and champion/challenger; automatically roll back on regression                                             | The only forward path is `proposed → offline → shadow → canary → champion_challenger → production`. Champion/challenger evidence must bind the candidate, contain at least 100 samples, report p≤0.05, and carry a ready verdict. The adapter accepts only Agentic Studio's paired McNemar `challenger-ready` report. Any failed monitored probe derives `regressionDetected` and automatically transitions an active rollout to `rolled_back`. |
| Invalidate and recertify dependent artifacts                                                                                                        | Production promotion invalidates every exact affected-artifact reference. Release readiness remains false until each artifact is recertified against the exact proposal version with evidence. Rollback restores the prior-version artifact state.                                                                                                                                                                                              |
| Never optimize raw clicks/retention directly                                                                                                        | Primary objectives are restricted to explicit outcome, preference, safety-adjusted quality, or task success. Clicks/retention can appear only as diagnostic engagement metrics. Contextual engagement signals require approved use plus user-intent, safety, explicit-outcome, and preference evidence.                                                                                                                                         |

## Canonical evidence and attribution plane

`LearningSignal` is the unit of learning authority. It retains the observed
outcome, human-readable summary, evidence references, hard/failure flags, and
novelty/impact/disagreement/drift scores. Its attribution binds the signal to:

- slot, run, and step;
- artifact reference and SHA-256;
- prompt reference and SHA-256;
- model and provider;
- rubric and policy;
- source references;
- intervention; and
- cohort.

There is no optional “best effort” attribution path. An absent intervention is
represented explicitly (for example, `intervention:none`) rather than silently
losing the dimension. Existing Agentic Studio feedback becomes an
`explicit_preference` signal only when its run and output bundle match the
supplied canonical attribution. Novelty, impact, disagreement, and drift scores
remain caller-supplied evidence; the adapter does not fabricate them from a star
rating.

Contextual clicks, retention, completion, dwell time, and return use have a
separate constrained envelope. They are admitted only with an approved-use flag,
user-intent reference, and non-empty safety, explicit-outcome, and preference
evidence. The proposal objective schema deliberately has no raw click or
retention primary metric.

## Diagnosis, proposal, and non-compensatory evaluation

Sampling uses versioned thresholds and records the entire population plus every
selected reason. A diagnosis must account for every and only the selected hard
cases, provide a natural-language narrative, and cite its signal evidence from
each finding. The runtime requires diagnosis versions to be contiguous from 1.

Change proposals bind a diagnosis version, target, base and candidate versions,
diff, objective constraints, affected artifacts, and distinct rollback target.
Proposal versions are also contiguous. Policy, rubric, and autonomy proposals
are structurally human-approval-required.

`runLearningProposalEvaluation` requires a runner for all seven dimensions:

- held-out capability;
- regression;
- safety;
- reliability;
- cost;
- latency; and
- subgroup outcomes.

Each probe derives pass/fail from its score, threshold, and direction. A high
mean cannot compensate for one failing axis. The evaluation binds the exact
proposal/candidate, held-out suite, and contamination-control evidence.

## Staged promotion, rollback, and recertification

Every rollout stage retains seven real probes and monitoring evidence in the
snapshot. Champion/challenger and production stages additionally retain the
baseline/candidate comparison, sample size, p-value, verdict, and evidence. The
Agentic Studio adapter maps only its paired McNemar `challenger-ready` report;
an inconclusive, worse, undersized, or non-significant report fails loud.

Promotion events and their stage-evidence records form a one-to-one contiguous
history. Snapshot restore recomputes all invariants: exact proposal/candidate,
one-stage transitions, automatic-system rollback identity, stage/evidence
agreement, and final stage. Removing stage evidence or rewriting a proposal,
diagnosis, event, or lifecycle stage cannot manufacture a production state.

A failed monitored probe derives regression automatically. An active candidate
then rolls back without depending on a caller-supplied manual transition. Only
production promotion invalidates dependent artifacts, and release readiness
stays false until every exact artifact is recertified with evidence for the
promoted proposal version.

## Adversarial fixture matrix

| Fixture                                                                                    | Result                                                     |
| ------------------------------------------------------------------------------------------ | ---------------------------------------------------------- |
| Signal omits provider or another attribution field                                         | Admission fails.                                           |
| Feedback is rebound to another run or artifact                                             | Adapter rejects substitution.                              |
| Raw retention is supplied as the primary objective                                         | Proposal schema rejects it.                                |
| Contextual engagement lacks intent/safety/outcome/preference evidence                      | Signal schema rejects it.                                  |
| Diagnosis omits one sampled hard case                                                      | Runtime rejects non-exact sample coverage.                 |
| Proposal skips a diagnosis or version                                                      | Runtime rejects unknown/non-contiguous lineage.            |
| One evaluation runner/probe is missing                                                     | Evaluation fails loud or blocks.                           |
| Safety fails while other six dimensions pass                                               | Proposal remains ineligible.                               |
| Policy/rubric/autonomy proposal lacks signed human approval                                | Offline advancement blocks.                                |
| Caller jumps directly to canary                                                            | Transition rejects the skipped stages.                     |
| Champion evidence is inconclusive, undersized, non-significant, or names another candidate | Adapter/schema rejects promotion evidence.                 |
| A monitored stage probe regresses                                                          | Active rollout automatically transitions to `rolled_back`. |
| Snapshot loses stage evidence or substitutes the diagnosis                                 | Restore rejects the tampered snapshot.                     |
| Production artifact is recertified against another proposal version                        | Recertification rejects the substitution.                  |

## Automated verification

### Current admitted refresh

The current refresh ran against repository commit
`9ad2678676d3c0800b9ac90bb37bb0fa1bc262a6` with lockfile SHA-256
`08b055786e9b8a6104d5313601d7eef3205dedcf395066fd9a8264c9859d690f` on host
`app-dev-server`, Ubuntu 24.04.4 LTS, Linux `6.8.0-134-generic` x86_64. The
relevant toolchain was Node.js `v24.15.0`, pnpm `10.25.0`, TypeScript `6.0.3`,
and Vitest `4.1.7`.

No live model/provider, production telemetry, customer signal, human approval,
canary, production candidate, or release artifact was used. Fixture identities
and in-memory lifecycle transitions prove the repository authority boundary;
they are not promotion evidence.

| UTC interval                                      | Exit | Exact command and result                                                                                                                                                                                                 |
| ------------------------------------------------- | ---: | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| 2026-07-29T11:47:49.626Z–2026-07-29T11:47:50.693Z |    0 | `pnpm exec vitest run --config libs/contracts/vitest.config.ts libs/contracts/src/common/outcome-learning-loop.test.ts --maxWorkers=1` — canonical signal, evaluation, promotion, rollback, and restore tests passed 9/9 |
| 2026-07-29T11:47:56.714Z–2026-07-29T11:48:00.448Z |    0 | `pnpm --dir libs/oshun/agentic-studio exec vitest run src/feedback/outcome-learning-loop.test.ts --config vitest.config.ts --maxWorkers=1` — Studio runtime/adaptor tests passed 6/6                                     |
| 2026-07-29T11:48:06.293Z–2026-07-29T11:48:14.357Z |    0 | `pnpm --dir libs/oshun/creative-autonomy exec vitest run src/steering/steering.test.ts src/steering/edit-diff.test.ts --config vitest.config.ts --maxWorkers=1` — preference-learning/edit tests passed 34/34            |
| 2026-07-29T11:48:22.258Z–2026-07-29T11:48:38.148Z |    0 | `pnpm exec tsc --noEmit -p libs/contracts/tsconfig.lib.json`                                                                                                                                                             |
| 2026-07-29T11:48:38.156Z–2026-07-29T11:48:53.169Z |    0 | `pnpm exec tsc --noEmit -p libs/oshun/agentic-studio/tsconfig.lib.json`                                                                                                                                                  |
| 2026-07-29T11:48:53.170Z–2026-07-29T11:49:09.585Z |    0 | `pnpm exec tsc --noEmit -p libs/oshun/creative-autonomy/tsconfig.json`                                                                                                                                                   |

The admitted current suites passed **49/49** value assertions and all three
owning typechecks:

- **Unit/schema/contract:** all nine signal families, exact attribution,
  sampling, diagnosis/proposal lineage, seven non-compensatory evaluation
  dimensions, approval, stage ordering, rollback, recertification, snapshot, and
  contextual-engagement constraints are executable assertions.
- **Integration:** the Agentic Studio adapter consumes canonical feedback and
  paired champion/challenger results; the creative learner exercises real
  preference, edit, calibration, deduplication, snapshot, and restoration
  behavior.
- **Adversarial/negative:** missing/substituted attribution, raw-retention
  objectives, incomplete evaluation, missing or mismatched human approval, stage
  skipping, statistically ineligible champion evidence, monitored regression,
  tampered snapshots, and wrong-version recertification fail for their intended
  reasons.
- **Repository E2E:** the in-memory lifecycle reaches ordered promotion,
  automatic rollback, invalidation, and recertification through public runtime
  APIs. It is not represented as a deployed learning service or real production
  promotion.

UI/accessibility automation, screenshots/media probes, mobile/Unreal automation,
production performance/load testing, and a deployed E2E are inapplicable to this
evidence-only refresh and the underlying contract/runtime change. No rendered
surface, media artifact, mobile flow, Unreal asset, production store, or live
network boundary changed.

### Historical implementation verification

At source publication, the owning packages passed **4,706/4,706** non-duplicated
tests:

| Surface                          |      Result |
| -------------------------------- | ----------: |
| Canonical contracts full package | 4,095/4,095 |
| Agentic Studio full package      |     343/343 |
| Creative autonomy full package   |     268/268 |

After final stage-evidence retention and lifecycle-history hardening, the exact
source focused suites passed 9/9 canonical contract tests and 6/6 Agentic Studio
runtime/adaptor tests. After rebasing an upstream root TypeScript configuration
change, both focused suites passed again (15/15), and all three affected
typechecks passed: contracts, Agentic Studio, and creative autonomy.

The contracts and Agentic Studio emitted TypeScript builds passed before and
after the TypeScript-configuration rebase. Explicit ESLint over all changed
production and test TypeScript passed with zero warnings. Prettier and
`git diff --check` passed.

Representative commands:

```text
pnpm --dir libs/contracts exec vitest run --testTimeout=120000
pnpm --dir libs/oshun/agentic-studio exec vitest run --config vitest.config.ts
pnpm --dir libs/oshun/creative-autonomy exec vitest run --config vitest.config.ts
pnpm --dir libs/contracts exec vitest run src/common/outcome-learning-loop.test.ts
pnpm --dir libs/oshun/agentic-studio exec vitest run src/feedback/outcome-learning-loop.test.ts
pnpm exec tsc --noEmit -p libs/contracts/tsconfig.lib.json
pnpm exec tsc --noEmit -p libs/oshun/agentic-studio/tsconfig.lib.json
pnpm exec tsc --noEmit -p libs/oshun/creative-autonomy/tsconfig.json
pnpm exec tsc -p libs/contracts/tsconfig.lib.json
pnpm exec tsc -p libs/oshun/agentic-studio/tsconfig.lib.json
pnpm exec eslint --no-ignore --max-warnings=0 <all changed TypeScript>
pnpm exec prettier --check <all changed non-lock files>
git diff --check
```

## Bounded verification notes

- Before the 2026-07-29 focused refresh, the host had 15 GiB RAM, 31 GiB swap,
  11 GiB available memory, and 16 GiB free disk. An unrelated checkout's
  long-running Vitest worker and this checkout's Nx daemon were active, so no
  broad contracts/workspace matrix or production build was started; the three
  task-focused suites and three typechecks ran serially.
- The host had 15 GiB RAM, 31 GiB swap, and long-lived Codex, Claude, and Nx
  processes. Package checks ran serially. The full contracts run was supervised
  at intervals under 30 seconds; available memory stayed at least 4.5 GiB, above
  both stop thresholds.
- Full contracts verification emitted an existing missing `src/index.js.map`
  warning and completed successfully: 179 files and 4,095 tests.
- The conventional creative-autonomy `tsconfig.lib.json` path does not exist.
  The package's actual `tsconfig.json` was used and passed before and after the
  upstream root-configuration rebase.
- Two source rebases integrated unrelated V10 motion-channel work. The first
  changed root TypeScript configuration, so focused tests, all three typechecks,
  and both emitted builds were repeated. The second shared only the unrelated
  binary lockfile and required no redundant source verification.
- The pre-existing `pnpm-lock.yaml` modification was preserved byte-for-byte
  through hooks and both rebases (Git blob
  `a37431842e330b3a06b030947c25bebdc72bc2b1`). Generated Cargo locks and
  `libs/urania/` were not staged.
- The source commit used the hook's documented `SKIP_TYPECHECK=1` only after all
  typechecks and emitted builds passed. Hook lint, formatting, asset locking,
  and documentation validation passed. The initial commit scope produced a
  warning, so the commit was amended to the allowed `studio` scope before
  publication.
- No customer UI, PWA, admin, mobile, Unreal, or visual behavior changed.
  Frontend skill, Playwright, mobile automation, screenshots, and Unreal
  verification were inapplicable.

## Artifact identity

| Artifact                                  | SHA-256                                                            |
| ----------------------------------------- | ------------------------------------------------------------------ |
| Canonical outcome learning contract       | `14eb2226b5058284eeab3d0615028457dd7236a5511bce3b3a6bb6d750424d47` |
| Agentic Studio learning runtime/adaptor   | `328940ac2648aba510f71c3500cd0f4da5d2dbe56e5893bebd45f0d55ecf4959` |
| Paired champion/challenger implementation | `606f6b89ffc58d07d56f3a066ab012b806949357fe25dd5c832c1651ef6e2d06` |
| Existing creative preference learner      | `1a1154c947899bcd7546652e5ffa9499383623c12244e8c7ccdbc783f8b69b3f` |

All four recorded hashes still match the admitted current source. The repository
runtime snapshot is version 1; diagnoses and proposals carry exact positive,
contiguous versions. Production model, provider, source, prompt, rubric, policy,
intervention, and cohort identities are caller-supplied exact references and
hashes rather than one hard-coded test version. This evidence refresh changes
none of those versions.

## Release, human-authority, blocker, and generated-status disposition

- The repository verdict is `repository-complete / closed` for LEARN-001.
  Production promotion remains unavailable without real signals, seven
  held-out/monitored dimensions, contamination evidence, staged runtime results,
  and dependent-artifact recertification for the exact proposal.
- Policy, rubric, and autonomy changes structurally require a signed human
  decision bound to the exact proposal version. Publishing this repository
  contract does not itself require such a decision. No reviewer identity,
  signature, policy approval, canary approval, or production promotion is
  fabricated here.
- Remaining external dependencies are durable deployment storage, real signal
  producers, diagnosis/evaluation providers, authorized human reviewers, rollout
  traffic and monitors, and exact artifact recertification. Fixtures cannot
  discharge those dependencies.
- No generated capability/status artifact advertises a live learning loop or
  production promotion. The completed-work-package admission verifier derives
  only this checked task's evidence-pack presence and identity; no generated
  green deployment claim was hand-edited for this refresh.

## Scope and remaining seams

- The runtime exposes a JSON-safe snapshot/restore boundary. Deployment
  composition must bind it to the owning durable store and real signal feeds;
  this package does not claim a live telemetry deployment.
- Natural-language diagnoses, dimension evaluations, human approvals, rollout
  monitors, and recertification proofs remain provider/operator evidence. Their
  absence blocks progress; fixtures are never promotion authority.
- The feedback adapter covers the existing explicit-preference stream. Edit,
  replacement, veto, gate-failure, complaint, correction, task-outcome, and
  approved contextual-engagement producers can submit the same canonical signal
  contract without a lossy generic feedback projection.

## Publication identity

- **Source commit:** `b1210f90706db87b94c82b0f1b2f78c6a4d629b2`
- **Source publication tip:** `b1210f90706db87b94c82b0f1b2f78c6a4d629b2`
- **Current-lineage equivalent source commit:**
  `64abd65ca5899f8901a54866321da95200bcb96b` (same source tree after rebase)
- **Closure evidence commit referenced by the ledger:**
  `9f88c864fe82dfad635a40557ef3269e7f87a3b5`
- **Current-lineage equivalent closure commit:**
  `3f41dba5d6d091bf5e2e4b3c958ac429d4ec363f` (same evidence tree after rebase)
- **Current refresh source commit:** `9ad2678676d3c0800b9ac90bb37bb0fa1bc262a6`
- **Current refresh evidence commit:** this evidence-only publication commit;
  the immutable identity is the Git commit containing this file.
