Work-package charter#
- Task ID:
AUD-007 - Owner: Oshun portfolio audit-integrity maintainer; implementation owner: Codex
- Accountable product: V1–V10 autonomous-content portfolio
- Priority: P0
- Target milestone: 2026-07-15 SOTA gap-closure program
- Risk tier: High audit-evidence and release-claim integrity
- Dependencies: AUD-003 canonical verification manifest and authenticated record, AUD-006 generated capability truth, and the 54 credited Vitest configurations
- Affected schemas and content slots: New
oshun.test-coherence-manifest.v1audit schema and generated coherence view; no shipped content slot or customer flow changed - Rollout strategy: Register one source-bound domain-value assertion per credited configuration, classify the proof actually exercised, require explicit external evidence before any high-evidence classification, and publish limitations beside the authenticated counts
- Rollback strategy: Revert the schema, manifest, generator, report, audit section, package scripts, and workflow changes together. Do not leave the canonical counts without their proof-class boundary.
- Evidence-pack location: This file and
evidence/v1-v9/AUD-007/
Implemented coherence contract#
2026-07-22 scope amendment#
The active contract now samples 54 credited configurations from
scripts/audit/v1-v10-verification-manifest.json. V10 contributes 19
configurations with 782/785 tests passed, 3 explicitly pending, and 0 failed.
The current canonical commands are
pnpm generate:audit:v1-v10-verification-record and
pnpm verify:audit:v1-v10-verification-record. Publication identities,
distribution counts, and command results below retain the initial 2026-07-15
evidence window as history.
scripts/audit/test-coherence-manifest.jsonmaps exactly one sampled test file and domain-value assertion to each of the 54 Vitest configurations credited byscripts/audit/v1-v10-verification-manifest.json. The generator rejects missing, duplicate, out-of-configuration, or out-of-filter samples.- Every sample is classified as
unit,contract,integration,e2e,real-provider,engine, orlive. Non-unit classes require a source-resident classification marker. Real-provider, engine, and live classes additionally require an external evidence reference, so a local suite cannot impersonate them. - The initial publication evidence distribution was 17 unit, 5 contract, 9 integration, and 4 local/in-process E2E configurations. The generated view carries the active distribution, including V10's 18 unit and 1 contract samples. Real-provider, engine, and live counts remain zero.
- Twenty-seven configuration samples bind a negative or deliberately broken fixture. This includes the V8 case evaluation mutation that drops a required deduction edge and must fail G4. The adversarial contract suite also removes a registered domain assertion in memory and proves the coherence gate fails, and it proves that assigning real-provider status without external evidence fails.
evidence/v1-v9/AUD-007/test-coherence.generated.mdpublishes the sampled claim, assertion marker, proof class, negative-fixture marker, weakness, and uncovered claims beside each recorded pass count. Canonical audit section 2.2 is generated from the same data..github/workflows/test-coherence.ymlenforces source-marker, record, and generated-report drift on the full credited source boundary. The existing audit-record workflow also runs the coherence verifier.
Publication identity#
- Coherence implementation source commit:
d76869b2ab861c4958f89c7447c2236c1ea7920f - Canonical verification-record publication commit:
977b335bbcb04a409c466c45d8c804e2505dd509 - Verification-record run interval:
2026-07-15T17:21:30.977Zthrough2026-07-15T17:26:12.076Z - Host/runtime:
dev-server, Linux6.8.0-110-genericx64, Node.jsv24.14.1, pnpm10.25.0, Vitest4.1.7, git2.43.0 - Provider/model gates: no OpenAI, Anthropic, RunPod, or database credential was present; no real-provider, engine, or live execution is claimed
| Artifact | SHA-256 |
|---|---|
scripts/audit/test-coherence-manifest.schema.json |
4240ee120dc660b937d6ba9670f6f6694474053581eb2615a5e4b68f5ecf1536 |
scripts/audit/test-coherence-manifest.json |
ffe203876c3dd1eafd5dd9f1d2c4d5d1ab526eafffbcdc1434ad3fc60ea95928 |
scripts/audit/generate-test-coherence.mjs |
8d324634d531a4dd392b60c0e33e3406e6ca7bd2fcbcc8af3bf090a00267606a |
scripts/audit/test-coherence-manifest.test.mjs |
06d328749b270e50c0fcf3b6e98220bba845066aa4d4edac0ad2f037c12d98c8 |
.github/workflows/test-coherence.yml |
996e391659544a29232d956a82d8357d5117e1a00248f492f56163f3f484f2fd |
evidence/v1-v9/AUD-007/test-coherence.generated.md |
560876f8ad46a1a3cb58c03580c856638c36f1a018e39e211c59b34c2c2c432f |
evidence/v1-v9/AUD-003/verification-record.json |
0356552d4184dec3e27eff215d3f447f70248cda3cf3334708b353aa83ab5689 |
Automated and adversarial verification#
pnpm generate:audit:v1-v9-verification-recordran sequentially with Vitest--maxWorkers=1from the clean implementation source. It recorded V1 counts of 248/248, 289/289, 138/138, and 40/40; V8 at 351/351 across 18 configurations; and V9 at 176/176 across 13 configurations. The known V8 generated-contract drift remained measured red with exit 1.pnpm verify:audit:v1-v9-verification-recordpassed 8/8 adversarial tests and record/table/command/boundary/log/source coherence.pnpm verify:audit:test-coherencepassed 6/6 adversarial tests plus the live workspace drift check.pnpm verify:audit:capability-truthpassed 7/7 adversarial tests and all five generated-view checks after the new section-order integration.pnpm verify:audit:machine-adjacent-docspassed 3/3 tests and its workspace contract check.- JSON Schema Draft 2020-12 validation of the coherence manifest, targeted
ESLint with
--no-ignore --max-warnings=0, targeted Prettier, JavaScript syntax checks, all three affected workflow YAML parses, andgit diff --checkpassed. - Resource supervision stayed near 12 GiB available memory with roughly 30 GiB free swap; no safety floor was approached.
Claim boundary and inapplicable avenues#
- This is one load-bearing assertion sample per credited configuration, not an assertion-by-assertion audit of every test in those suites. The generated weakness and uncovered-claim columns preserve that boundary.
- Passing local unit, contract, integration, or in-process E2E tests does not prove a deployed provider, Unreal placement, production release, player or learner outcome, efficacy, or live customer behavior.
- The negative-fixture count records source-bound, canonically executed fixtures; it does not claim exhaustive mutation coverage or a mutation score.
- No customer UI, mobile flow, media artifact, API contract, or persistence behavior changed. Playwright, mobile automation, accessibility scans, Unreal execution, provider probes, and media-quality evaluation are inapplicable.
- No human approval is required or manufactured by this work package.