Status: blocked on a fresh live provider/operator/evaluation/canary run; none of the six acceptance checks is closed.
Evidence-contract source tip: df17d74948d2d2428f83797609f96a10587d2905.
Evidence boundary#
The repository now has one machine-verifiable, fail-closed contract for the entire V1 reference vertical:
scripts/v1/verify-reference-vertical-evidence.mjsevidence/v1-v9/V1-003-reference-vertical-evidence.jsonevidence/v1-v9/V1-003-reference-vertical-readiness.json
The contract requires all six acceptance dimensions to pass together:
- A bounded set of unique text, image, audio, and video slots must be covered exactly once by real provider-backed artifacts. Every artifact must carry a provider request receipt, model identity, content hash, locator, rights record, and lineage.
- Authenticated UI receipts must show pause, edit, replace, approve, and rollback each executing once.
- Judge calibration must use at least 100 licensed human-gold samples and at least two distinct provider judges, with positive normalized weights, measured disagreement, and a calibration report.
- Every outcome must attribute to an exact generated artifact, and the primary optimization objective must be an explicit human outcome rather than raw engagement.
- A signed A/B or champion/challenger result against the authored baseline must have at least 100 samples per arm, an effect estimate, a p-value, and a verdict.
- The same run must have an observed canary placement, cohort, SLOs, costs, and an executed, verified rollback drill.
Completed evidence is fresh for no more than 24 hours. Missing, malformed, duplicated, stale, simulated, or incomplete evidence blocks the gate.
Current retained report#
The retained evidence input identifies the run as
v1-reference-vertical-not-yet-executed. Its readiness report is blocked,
with all six checks false. It does not substitute fixtures, mocks, or repository
capability claims for live proof.
The current execution environment exposed no live provider credential variable names in a name-only environment scan, and the retained audit registry has no current provider-probe artifact. The environment also cannot manufacture the licensed human-gold set, authenticated operator actions, controlled live experiment, production canary observations, or rollback drill required by this contract. Those are external execution inputs, so the V1-003 checklist remains open.
Verification#
On 2026-07-27:
node --check scripts/v1/verify-reference-vertical-evidence.mjspassed.node --test scripts/v1/verify-reference-vertical-evidence.test.mjspassed 9/9, including the valid reference case and adversarial cases for fake or missing providers, malformed slot bounds, duplicate artifacts, incomplete operator flow, invalid judge panels, engagement optimization, undersampled comparison, missing rollback, and stale evidence.- The default CLI rejected the retained blocked evidence with a nonzero exit.
--reportreproducedevidence/v1-v9/V1-003-reference-vertical-readiness.jsonbyte-for-byte after formatting.- Prettier, ESLint,
node --check, andgit diff --checkpassed.
Unblock procedure#
Execute one bounded live run with authorized provider credentials and licensed
evaluation data, capture immutable receipts from the operator UI, experiment,
canary, SLO/cost telemetry, and rollback drill, then replace the empty run
payload in V1-003-reference-vertical-evidence.json. Run:
node scripts/v1/verify-reference-vertical-evidence.mjs \
--evidence=evidence/v1-v9/V1-003-reference-vertical-evidence.json \
--report
Only a report with status: "passed" and all six checks true is sufficient to
close the corresponding checklist items.