# EVAL-002 progress evidence — licensed human-gold acquisition boundary

> **Status: not closed.** This pack records completed repository work and the
> remaining external acquisitions. It is not evidence that any human author,
> learner, teacher, rater, or evaluator has been recruited.

## Work-package charter

- **Task ID:** `EVAL-002`
- **Owner:** evaluation-data governance, V5 authored-case content, and V9
  learning-research programs; repository implementation owner: Codex
- **Audit finding:** durable operator accept/reject rows and deterministic split
  helpers existed, but they did not establish best-authored quality, licenses,
  consent, exact artifact provenance, real runtime stages, rater governance,
  contamination freedom, or actual V5/V9 human cohorts
- **Repository objective:** prevent legacy feedback, fixtures, or planning
  manifests from being admitted as licensed human gold; preserve missing human
  work as owned, machine-readable acquisition blockers
- **External objective:** commission and contract the human-authored corpora and
  recruit the consented cohorts represented by the registry
- **Rollback strategy:** revert source commit
  `238ebf35f23959be140663567563a5401550e67e`
- **Evidence-pack location:** this file

## Completed repository work

The canonical `LicensedHumanGoldCorpus` record now carries:

- schema, corpus, version, seal, revocation, and manifest identity;
- best-authored positive, known-bad adversarial, borderline/high-disagreement,
  and pairwise-preference roles;
- optimization, development, and release-holdout splits;
- exact artifact bytes, author identity, provenance manifest, license receipt,
  consent proof, retention/deletion state, access policy, and audit reference;
- real provider/model, render, cook, in-engine, and placed evidence with exact
  provider, model, engine, build, environment, and placement versions;
- locales and cultural contexts per content family;
- versioned instructions, rubric, conflict policy, adjudication policy, rater
  expertise/locale/credentials, ratings, disagreement, and adjudications;
- human cohort recruitment, aggregate roster digest, consent, privacy,
  retention, access, independence, locale, expertise, and evidence;
- every optimization exposure and exact/near/semantic contamination analysis.

`evaluateHumanGoldCorpusReadiness` compares a corpus with a versioned coverage
policy. It requires best-authored, borderline, and release-holdout items for
every content family; known-bad items for every hard gate; per-family locale and
cultural coverage; configured real-output stages; active rights, consent,
retention, access, and rater governance; clean holdout contamination coverage;
and configured item/cohort acquisitions. `assertHumanGoldCorpusReady` raises a
typed error containing every exact blocker.

The contract/Prisma alignment table explicitly records the corpus as
`code_only`. Existing `CrossCuttingGoldSetEntry` rows remain valid operator
feedback, but no persistence projection falsely upgrades them into licensed
corpus, artifact, rater, cohort, exposure, or contamination records.

## Optimization and release admission

The Agentic Studio admission boundary recomputes readiness on every request; a
caller cannot substitute a stale passing report. Purpose determines the only
eligible split:

| Purpose                                              | Admissible split  |
| ---------------------------------------------------- | ----------------- |
| Model, prompt, rubric, policy, or judge optimization | `optimization`    |
| Development evaluation                               | `development`     |
| Release evaluation                                   | `release_holdout` |

Explicit wrong-split, duplicate, unknown, or access-unauthorized requests fail
with typed errors. Every successful admission binds the exact corpus manifest,
policy, consumer version, item IDs, artifact digests, and selection digest, and
emits exposure records that the caller must append atomically. Release holdouts
cannot silently enter model training, prompt/rubric/policy optimization, judge
calibration, or development evaluation.

## Machine-readable external acquisition register

`evaluation/human-gold/acquisition-registry.v1.json` contains six acquisition
requests and deliberately records **zero acquired units**:

| Request                                               | Required | Acquired | Status  |
| ----------------------------------------------------- | -------: | -------: | ------- |
| Portfolio best-authored content-family corpus program | 1 corpus |        0 | Blocked |
| Portfolio hard-gate adversarial corpus program        | 1 corpus |        0 | Blocked |
| Structured V5 authored-case corpus                    | 25 items |        0 | Blocked |
| V9 real learner cohort                                | 1 cohort |        0 | Blocked |
| V9 practicing-teacher cohort                          | 1 cohort |        0 | Blocked |
| V9 independent evaluator cohort                       | 1 cohort |        0 | Blocked |

For cohort requests, the registry counts cohort acquisitions, not participants.
The eventual coverage policy must set participant minima from an approved,
powered study protocol. No sample size, contract, evidence receipt, or
participant identity is fabricated here.

The blockers require, among other things, commissioned human authors, executed
rights/consent/data-processing terms, structured/cooked/placed/playtested V5
cases, approved sampling and study protocols, participant privacy/withdrawal
controls, target-learner and practicing-teacher recruitment, and an evaluator
independent of the optimization team. These actions require external authority
and coordination beyond a repository mutation.

## Adversarial fixture matrix

| Fixture                                                                        | Result                                           |
| ------------------------------------------------------------------------------ | ------------------------------------------------ |
| Provider/runtime output labeled best-authored                                  | Schema rejects the substituted baseline          |
| Known-bad item names no expected hard-gate failure                             | Schema rejects the item                          |
| Borderline item lacks measured high disagreement/adjudication                  | Schema rejects the item                          |
| Provider, engine, build, or placement version is absent                        | Stage evidence is invalid                        |
| Rights or consent proof names another artifact                                 | Corpus schema rejects cross-subject substitution |
| Rating or adjudication names another item                                      | Corpus schema rejects cross-item substitution    |
| Rights expire, data is deleted, access is suspended, or a rater conflicts      | Readiness is blocked with exact subjects         |
| Release artifact digest appears in an optimization split                       | Readiness is blocked for holdout overlap         |
| Holdout is exposed to prompt optimization                                      | Readiness and admission block the exposure       |
| Holdout analysis is missing or contaminated                                    | Readiness is blocked                             |
| V5/V9 cohort or item requirement is absent                                     | Readiness is blocked; no fixture satisfies it    |
| Caller requests unknown, duplicate, or wrong-split items                       | Admission fails with a typed error               |
| Caller presents a draft corpus                                                 | Admission recomputes readiness and fails         |
| Registry request claims acquisition without full count, contract, and evidence | Registry schema rejects the claim                |

## Automated verification

The integrated repository work passed **4,390/4,390** non-duplicated owning
package tests:

| Surface                                                                  |      Result |
| ------------------------------------------------------------------------ | ----------: |
| Canonical contracts full package, including all migration/rollback cases | 4,053/4,053 |
| Agentic Studio full package, including purpose/split admission           |     337/337 |

The new coverage consists of 15 contracts/acquisition-registry tests and 7
Agentic Studio admission tests. Production library typechecks passed for both
packages. Targeted Nx builds passed for `@oshun/contracts` and
`@oshun/agentic-studio`, including the latter's six build dependencies. Explicit
ESLint over every changed TypeScript file passed with zero warnings; Prettier
and `git diff --check` passed.

Representative commands:

```text
pnpm --dir libs/contracts exec vitest run --testTimeout=120000
pnpm --dir libs/oshun/agentic-studio exec vitest run
pnpm exec tsc -p libs/contracts/tsconfig.lib.json --noEmit
pnpm exec tsc -p libs/oshun/agentic-studio/tsconfig.lib.json --noEmit
NX_TUI=false pnpm nx run @oshun/contracts:build --parallel=1 --skip-nx-cache
NX_TUI=false pnpm nx run @oshun/agentic-studio:build --parallel=1 --skip-nx-cache
pnpm exec eslint <all changed TypeScript files> --max-warnings=0 --no-warn-ignored
pnpm exec prettier --check <all changed non-lock files>
git diff --check
```

## Bounded verification notes

- The first full contracts run passed 4,049 tests but four Prisma migration
  cases exceeded their fixed 30-second timeout while another workspace agent's
  Nx graph workers saturated CPU. The exact migration suite then passed 9/9 with
  a 120-second ceiling, and the complete contracts suite passed all 175 files
  and 4,053 tests under the same supervised ceiling.
- Contracts still emit the repository's existing missing
  `libs/contracts/src/index.js.map` warning; it does not affect execution.
- The first Agentic Studio Nx build exposed that dependency-output resolution
  loses callback inference for the new Zod-derived types. Explicit type
  annotations preserve the same source types; the rerun compiled all seven build
  targets successfully.
- The source commit used the hook's documented `SKIP_TYPECHECK=1` after both
  production typechecks, full package suites, and targeted builds passed.
  Common-contract changes otherwise expand the hook into an unsafe broad
  workspace typecheck on this host. Hook lint, formatting, asset, stub, and
  documentation gates passed.
- Verification ran sequentially on a 15 GiB RAM/31 GiB swap host while another
  agent held active Nx workers. Available memory remained above 9 GiB before the
  full gates.
- The unrelated working `pnpm-lock.yaml` blob was not staged and was preserved
  across the source rebase.
- No customer UI, PWA, admin, mobile, Unreal, or visual surface changed.
  Frontend skill, Playwright, mobile automation, screenshots, and Unreal
  verification were inapplicable.

## Artifact identity

| Artifact                           | SHA-256                                                            |
| ---------------------------------- | ------------------------------------------------------------------ |
| Licensed corpus/readiness contract | `186d7026111c70472da2c267e12369446370ba67606e02a5fa747965a8a72e7e` |
| Purpose/split admission boundary   | `68b94b6f1f4e4b8efd53869c1a1bb10328ed488e50b430e78364347e9dd54605` |
| External acquisition registry      | `52e5782f5e6fe177a6977bdda21cbdef208bf74f50fd5c8f0212c7c6e9691ae7` |

## Remaining work and authority boundary

All eight EVAL-002 ledger boxes remain open. Closing them requires actual
licensed and consented human work—not more fixtures:

1. Approve the complete content-family and hard-gate sampling frames.
2. Contract and compensate qualified human authors and raters under usable
   rights, provenance, retention, access, deletion, and withdrawal terms.
3. Produce, adjudicate, and seal positive, adversarial, borderline,
   culturally/linguistically diverse, and preference items.
4. Reserve release holdouts before prompt/rubric/policy/model optimization and
   run contamination analysis against every optimization source bundle.
5. Structure, cook, place, and play all 25 V5 cases with exact author and rights
   evidence.
6. Approve a powered V9 human-participant protocol, recruit real learners,
   practicing teachers, and independent evaluators, and capture consented
   outcome evidence.

The assistant has no authority to contract, compensate, recruit, consent, or
message those people. Repository work now makes that missing authority visible
and prevents release or optimization from bypassing it.

## Publication identity

- **Repository source commit:** `238ebf35f23959be140663567563a5401550e67e`
- **Repository source publication tip:**
  `238ebf35f23959be140663567563a5401550e67e`
- **EVAL-002 closure evidence commit:** not applicable; EVAL-002 is open
