# AUD-007 evidence pack — canonical test coherence

## Work-package charter

- **Task ID:** `AUD-007`
- **Owner:** Oshun portfolio audit-integrity maintainer; implementation owner:
  Codex
- **Accountable product:** V1–V10 autonomous-content portfolio
- **Priority:** P0
- **Target milestone:** 2026-07-15 SOTA gap-closure program
- **Risk tier:** High audit-evidence and release-claim integrity
- **Dependencies:** AUD-003 canonical verification manifest and authenticated
  record, AUD-006 generated capability truth, and the 54 credited Vitest
  configurations
- **Affected schemas and content slots:** New `oshun.test-coherence-manifest.v1`
  audit schema and generated coherence view; no shipped content slot or customer
  flow changed
- **Rollout strategy:** Register one source-bound domain-value assertion per
  credited configuration, classify the proof actually exercised, require
  explicit external evidence before any high-evidence classification, and
  publish limitations beside the authenticated counts
- **Rollback strategy:** Revert the schema, manifest, generator, report, audit
  section, package scripts, and workflow changes together. Do not leave the
  canonical counts without their proof-class boundary.
- **Evidence-pack location:** This file and `evidence/v1-v9/AUD-007/`

## Implemented coherence contract

### 2026-07-22 scope amendment

The active contract now samples 54 credited configurations from
`scripts/audit/v1-v10-verification-manifest.json`. V10 contributes 19
configurations with 782/785 tests passed, 3 explicitly pending, and 0 failed.
The current canonical commands are
`pnpm generate:audit:v1-v10-verification-record` and
`pnpm verify:audit:v1-v10-verification-record`. Publication identities,
distribution counts, and command results below retain the initial 2026-07-15
evidence window as history.

1. `scripts/audit/test-coherence-manifest.json` maps exactly one sampled test
   file and domain-value assertion to each of the 54 Vitest configurations
   credited by `scripts/audit/v1-v10-verification-manifest.json`. The generator
   rejects missing, duplicate, out-of-configuration, or out-of-filter samples.
2. Every sample is classified as `unit`, `contract`, `integration`, `e2e`,
   `real-provider`, `engine`, or `live`. Non-unit classes require a
   source-resident classification marker. Real-provider, engine, and live
   classes additionally require an external evidence reference, so a local suite
   cannot impersonate them.
3. The initial publication evidence distribution was 17 unit, 5 contract, 9
   integration, and 4 local/in-process E2E configurations. The generated view
   carries the active distribution, including V10's 18 unit and 1 contract
   samples. Real-provider, engine, and live counts remain zero.
4. Twenty-seven configuration samples bind a negative or deliberately broken
   fixture. This includes the V8 case evaluation mutation that drops a required
   deduction edge and must fail G4. The adversarial contract suite also removes
   a registered domain assertion in memory and proves the coherence gate fails,
   and it proves that assigning real-provider status without external evidence
   fails.
5. `evidence/v1-v9/AUD-007/test-coherence.generated.md` publishes the sampled
   claim, assertion marker, proof class, negative-fixture marker, weakness, and
   uncovered claims beside each recorded pass count. Canonical audit section 2.2
   is generated from the same data.
6. `.github/workflows/test-coherence.yml` enforces source-marker, record, and
   generated-report drift on the full credited source boundary. The existing
   audit-record workflow also runs the coherence verifier.

## Publication identity

- **Coherence implementation source commit:**
  `d76869b2ab861c4958f89c7447c2236c1ea7920f`
- **Canonical verification-record publication commit:**
  `977b335bbcb04a409c466c45d8c804e2505dd509`
- **Verification-record run interval:** `2026-07-15T17:21:30.977Z` through
  `2026-07-15T17:26:12.076Z`
- **Host/runtime:** `dev-server`, Linux `6.8.0-110-generic` x64, Node.js
  `v24.14.1`, pnpm `10.25.0`, Vitest `4.1.7`, git `2.43.0`
- **Provider/model gates:** no OpenAI, Anthropic, RunPod, or database credential
  was present; no real-provider, engine, or live execution is claimed

| Artifact                                             | SHA-256                                                            |
| ---------------------------------------------------- | ------------------------------------------------------------------ |
| `scripts/audit/test-coherence-manifest.schema.json`  | `4240ee120dc660b937d6ba9670f6f6694474053581eb2615a5e4b68f5ecf1536` |
| `scripts/audit/test-coherence-manifest.json`         | `ffe203876c3dd1eafd5dd9f1d2c4d5d1ab526eafffbcdc1434ad3fc60ea95928` |
| `scripts/audit/generate-test-coherence.mjs`          | `8d324634d531a4dd392b60c0e33e3406e6ca7bd2fcbcc8af3bf090a00267606a` |
| `scripts/audit/test-coherence-manifest.test.mjs`     | `06d328749b270e50c0fcf3b6e98220bba845066aa4d4edac0ad2f037c12d98c8` |
| `.github/workflows/test-coherence.yml`               | `996e391659544a29232d956a82d8357d5117e1a00248f492f56163f3f484f2fd` |
| `evidence/v1-v9/AUD-007/test-coherence.generated.md` | `560876f8ad46a1a3cb58c03580c856638c36f1a018e39e211c59b34c2c2c432f` |
| `evidence/v1-v9/AUD-003/verification-record.json`    | `0356552d4184dec3e27eff215d3f447f70248cda3cf3334708b353aa83ab5689` |

## Automated and adversarial verification

- `pnpm generate:audit:v1-v9-verification-record` ran sequentially with Vitest
  `--maxWorkers=1` from the clean implementation source. It recorded V1 counts
  of 248/248, 289/289, 138/138, and 40/40; V8 at 351/351 across 18
  configurations; and V9 at 176/176 across 13 configurations. The known V8
  generated-contract drift remained measured red with exit 1.
- `pnpm verify:audit:v1-v9-verification-record` passed 8/8 adversarial tests and
  record/table/command/boundary/log/source coherence.
- `pnpm verify:audit:test-coherence` passed 6/6 adversarial tests plus the live
  workspace drift check.
- `pnpm verify:audit:capability-truth` passed 7/7 adversarial tests and all five
  generated-view checks after the new section-order integration.
- `pnpm verify:audit:machine-adjacent-docs` passed 3/3 tests and its workspace
  contract check.
- JSON Schema Draft 2020-12 validation of the coherence manifest, targeted
  ESLint with `--no-ignore --max-warnings=0`, targeted Prettier, JavaScript
  syntax checks, all three affected workflow YAML parses, and `git diff --check`
  passed.
- Resource supervision stayed near 12 GiB available memory with roughly 30 GiB
  free swap; no safety floor was approached.

## Claim boundary and inapplicable avenues

- This is one load-bearing assertion sample per credited configuration, not an
  assertion-by-assertion audit of every test in those suites. The generated
  weakness and uncovered-claim columns preserve that boundary.
- Passing local unit, contract, integration, or in-process E2E tests does not
  prove a deployed provider, Unreal placement, production release, player or
  learner outcome, efficacy, or live customer behavior.
- The negative-fixture count records source-bound, canonically executed
  fixtures; it does not claim exhaustive mutation coverage or a mutation score.
- No customer UI, mobile flow, media artifact, API contract, or persistence
  behavior changed. Playwright, mobile automation, accessibility scans, Unreal
  execution, provider probes, and media-quality evaluation are inapplicable.
- No human approval is required or manufactured by this work package.
