Disciplines · Audits

V5 Autonomous Content Agents — Audit (2026-07-14)

claims around the generated content that were still fabricated.** Three seam clusters survived the 2026-06-12 remediation (it fixed the launch manifest, legal artifacts, and service health, but not audit category 2 in code).

5sections7 minread
Markdown source
On this page

Scope: every system in V5/ that creates content autonomously — procedural generators, run generators, AI commentary/recap/highlight pipelines, generated broadcast media, the adaptive persona, daily challenges, machine translation, and the automated director/editor pipelines. Method: main-loop sequential audit (no agent fan-out, per session-limit constraint); confirmatory read of every public generation function plus the project-mandated adversarial stub-indicator grep and a silent-stub scan (unseeded RNG, fabricated-success flags, circular gates). Prior context: V5/docs/audit/v5-verification-2026-05-31.md and V5/REMEDIATION_2026-06-12.md.

Verdict#

The generation logic itself is real and well-engineered; it was the trust claims around the generated content that were still fabricated. Three seam clusters survived the 2026-06-12 remediation (it fixed the launch manifest, legal artifacts, and service health, but not audit category 2 in code). All are now fixed evidence-gated and fail-closed, with the repo verifiers updated in the same pass and all 75 validator tests green. As an autonomous content platform V5 is not prod-ready end-to-end — the honest blockers are the same class as V3/V4: no cooked assets, provider seams at every actual model-inference moment, and human sign-off gates that now correctly read "pending" instead of pre-approving themselves.

What is genuinely REAL (audited this pass)#

  • V5Procgen — seeded deterministic generation throughout (FRandomStream over xor-combined seeds 73856093/19349663/83492791): 5 side-quest patterns with step placement, 320 weighted frontier encounter templates with honor-biased rescue weighting, 12 monster archetypes → contracts with rank/reward formulas, weighted loot rolls, per-partition city decoration, galaxy detail sets. Tests assert reproducibility (same seed ⇒ identical stable fingerprint) and divergence across seeds.
  • V5RoguelikeHunterRun — real run generator composing procgen contracts over 24 map templates and 20 rule-stacking modifiers (multiplicative loot/toxicity/damage, additive score bonus), permadeath lifecycle, and a fail-closed leaderboard score (only Completed + anti-cheat-clean runs are eligible).
  • AI Director & adaptive difficulty — honestly labeled declared intent in features/platform-performance-and-tech.md: no code anywhere, no fabricated claims, launch gate stays red. Not implemented, and says so.
  • services/replays — real content-addressed SHA-256 chunk store with refcounted dedup and HMAC-signed 900 s download grants (ReplaysDomain, live-Postgres path in realtime.ts); the not-configured path serves the honest 503/contract-echo seam from the 06-12 remediation. The three AI endpoints (TTS commentary, top-10 ranker, recap) return contract descriptors, not fabricated content — the actual inference moments are declared provider seams (CoquiTTS, v5-highlight-ranker-v1, v5-match-recap-summarizer-v1).
  • V5RadioTV — honest scheduler/selector over authored data (loop-clock segment selection, format/topic-filtered talk rotation, emergency-override vehicle mix). The 1200 talk lines are transparent template filler with no false provenance, and the §28 authoring tasks are honestly [ ].
  • UV5_SciFiPvP_BroadcastDirector — real priority-driven shot selection from live match state (impact replay ▸ missile track ▸ objective ▸ pursuit by distance), rule-driven blend times.
  • Kade Roe AI persona (V5SciFiGalacticSquad) — exactly what the docs claim: a deterministic, privacy-bounded trait-vector accumulator (5 traits, 6 authored signals, clamped trust/affinity, ≤0.10 deltas) with a hard cell/locality/no-cross-cell-training guard that returns state unchanged on violation. The honest opposite of a black-box model.
  • Mind Palace daily (V5DetectiveMindPalace) — date-indexed rotation over an authored puzzle catalog; solve scoring is fail-closed (solved + anti-cheat + positive time) with par-time bonus and hint penalties.
  • VTuber overlay / companion bridge — declared session-config payloads with honest guardian gating; no fabricated moderation claims.

Fabrication seams found and FIXED this audit#

HIGH — voice-chat "translation" that never translated#

UV5_Localization_VoiceChatTranslation::TranslateVoiceChatTranscript returned "[<target-culture>] " + SourceTranscript as TranslatedTranscript with bAllowed=true — the source text relabeled as target-language output while the route named V5RealtimeMT as provider (same class as V2's local_default seam). Fix: the function now requires a ProviderTranslatedTranscript; empty ⇒ fail-loud denial (DenialReason="mt_provider_result_missing", no text), non-empty ⇒ verbatim provider result plus MTProviderUsed recorded on the packet. Tests cover the provider path, the denial path, and the policy-gate denial.

HIGH — cross-cell replay editor pre-verified its own clips#

MakeReplaySourceClip hardcoded bReplayHashVerified / bPrivacyScrubbed / bRightsCleared / bSpliceEligible = true (struct defaults were also true), and validator + manifest + two test suites required the fabricated claims — a circular fabricate-then-assert gate (05-31 audit category 2, never fixed in code). Fix: defaults flipped to false; claims only flip through receipt functions — ApplyReplayHashVerification (non-empty expected==actual hash), ApplyRightsClearance (clearance ref + named reviewer), ApplyPrivacyScrubReceipt; bSpliceEligible is now derived from the three receipts. Splice bContinuityChecked is computed from actual timeline adjacency; bAudioNormalized requires a measured loudness pass inside −16 LUFS ±1 LU / ≤ −1 dBTP via ApplyAudioNormalizationMeasurement. Validation separates honesty errors (claim without receipt ⇒ invalid) from publish gating (bPublishReady=false + named PublishBlockers while receipts are missing — a valid draft is not a publishable export). The manifest (fullvision_cross_cell_replay_editor_manifest.json) now carries evidenceMode: "declared-sample", honest false claims, and a fail-closed exportPublishBlocked: true gate with reason; the Python validator computes continuity from rows, requires claims to match receipts, and requires the publish block to reflect actual receipt coverage.

MED — self-approving director gates#

  • MakeCameraSetup stamped bDirectorApproved=true on all 8000 generated camera setups; the validator required it. Now: setups start unapproved, RecordDirectorApproval(Library, DirectorId) applies a named sign-off (no-name approves nothing), validation reports bDirectorSignOffComplete=false + blocker until then, and an approval without a named director is an honesty error.
  • MakeCommentaryTrack stamped bCellApproved=true on director commentary tracks. Now: unapproved by default; RecordCellApproval(Track, ReviewerId) with a named reviewer flips it and records ApprovedByReviewerId.
  • MakeDirectorCut declared bContinuityChecked=true; now computed from real clip adjacency, and both the C++ validator and validate-cinematic-director-mode.py verify the flag against computed adjacency instead of trusting it.

MED — highlight "top-10" fixture posing as ranked output#

BuildTop10HighlightClips fabricates 10 clips from rank arithmetic (no replay input), and BuildYear1AIHighlightReel(ReplayId, …) embedded them as TopClips for a named replay, unlabeled; the declared v5-highlight-ranker-v1 model had no executable implementation. Fix: clips carry EV5HighlightClipProvenance — the fixture builder's clips are labeled SampleFixture (and must not claim a source replay), and a real ranker now exists: RankTop10HighlightClipsFromEvents implements the manifest's model — per-signal minimum thresholds (kill-multiplicity 2, others 1), 40/30/20/10 weighted scoring, objective-impact then earlier-timestamp tie-breaking, 6–18 s clip windows, top-10 truncation, RankedFromReplayEvents + SourceReplayId provenance. ValidateYear1AIHighlightReel enforces provenance honesty both ways; tests assert exact scores (3·40+2·30+1·20+1·10=210), threshold gating, tie-breaks, window clamping, and that a ranked claim without a source replay fails.

Cleanup — three validators still codified reverted checkboxes#

validate-period-documentary.py, validate-companion-ar-mode.py, and validate-companion-mind-palace-editor.py still required [x] marks that the 05-31/06-12 remediations honestly reverted to [ ] (documentary cinematic binary asset; companion §123/§142 with no buildable app) — so they failed red against the honest state. They are now evidence-gated both ways: the item must stay [ ] while the backing deliverable (cooked documentary assets; a buildable companion project) is absent, and must be [x] once it exists.

Honest remaining gaps (not locally closable)#

  • Every real model-inference moment is a provider seam: CoquiTTS synthesis, the recap summarizer, Whisper STT / V5RealtimeMT translation providers. The routing/policy/validation layers around them are real and fail-loud.
  • The AI Director is design intent with no code (disclosed as such).
  • Zero cooked binary assets exist anywhere in V5 (known since 05-31), so generated content references (/Game/... map/sequence paths) point at authored paths, not assets; content-volume tasks remain honestly [ ].
  • Human gates now read honestly pending: director sign-off, cell approval, rights clearance, replay-hash receipts, measured loudness passes.
  • No UE toolchain on this Mac: the C++ changes are verified by the repo's needle validators and idiom-careful review, not a compile. A V5Editor Linux Development build + Automation RunTests V5. on the UE box is the next hardening step (the 05-31 pass proved that pipeline works).

Verification record (2026-07-14)#

  • python3 -m pytest V5/tools — 75/75 passed (was 72/75 before this pass; the 3 reds were the stale-needle validators above).
  • All 15 generator-related validators run individually: PASS, including cross-cell-replay-editor, cinematic-director-mode, realtime-voice-chat-translation, broadcast, cinematics, period-documentary, companion-ar-mode, companion-mind-palace-editor, mind-palace-daily, scifi-ai-persona-companion, vtuber-companion-stream, hunter-run-modifiers, replay-modes, validate-v5-docs, validate-cells.
  • Adversarial grep over all audited generator directories: zero actionable hits (remaining matches are anti-cheat domain vocabulary and a comment documenting a previously fixed seam).
  • Unseeded-RNG scan over generation paths: zero hits (all generation uses seeded FRandomStream).