# V1 Autonomous Content Systems — SOTA Audit — 2026-07-02 **Scope:** every system that lets V1 autonomously produce content — the creative-autonomy director plane, the creative orchestrator + `@oshun/ai` agent-loop, the BFF generation executors/Isis release gate/autonomy bindings, the shared content-quality-judge engine + release gates + content-service, and the editorial/governance layer (studio-authoring, generation-control-isis, agentic-studio, agent-pipelines). **Method:** five deep-read adversarial audit agents (max 2 concurrent, per session limits), each reading every core source file with the repo's stub-indicator battery plus an explicit 2026-SOTA technique checklist; every HIGH finding was then independently re-verified by the coordinating session against the source before being recorded here. Prior ledgers (`CREATIVE_AUTONOMY_UNDERLYING_AUDIT_2026-07-02.md`, `AGENTIC_SYSTEMS_FULL_AUDIT_2026-07-02.md`, `V1_V9_AGENTIC_CONTENT_SOTA_ASSESSMENT_2026-06-14.md`) were used as the baseline so this pass verifies rather than re-discovers. ## Verdict up front **The text path is real and in several places ahead of common 2026 practice** — discounted Thompson sampling with exact Beta-preserving decay, Bradley–Terry tournaments with double-ordering position-bias mitigation, pessimistic-LCB best-of-N, corpus-level diversity gates, coverage-discounted panel confidence, fail-closed governance with content-hash-bound promotion. The adversarial stub scan across all five units returned **zero result-faking hits**; tests assert computed values throughout. **But five verified defects undermine the quality loop as deployed**, and the single biggest SOTA gap is structural: **the quality engine is text-only.** V1 generates images, narration, music, and video, yet no automated system ever looks at the pixels or listens to the audio — `qualityAggregate` is hardcoded `null` in all six provider adapters, the only "judge" media meets is a text LLM scoring the job's JSON metadata, and the plane's revise-with-feedback loop is therefore blind for media. Meanwhile the learning loop's back half is dormant: tournament preference pairs are discarded, promoted gold sets are never consumed for judge calibration, and human post-edit deltas — the strongest quality signal available — are captured but never learned from. --- ## §1 Verified defect ledger Severity ordering. Every HIGH was re-verified line-by-line by the coordinator; MED/LOW items carry the deep-read agent's evidence and were spot-checked where load-bearing. ### HIGH - **H1. Restart double-learning corrupts the bandit/calibration state.** `libs/oshun/creative-autonomy/src/director/director.ts:218` keeps `learnedItemIds` as a private in-memory `Set`; `apps/oshun/bff/src/agentic/autonomy-bindings/plane-persistence.ts:42-57` snapshots/restores the queue, sampler arms, and learner — but not that set. After every restart, `ingestResolutions()` (`director.ts:683-713`) re-feeds the entire resolved history into posteriors and calibration EMAs that already contain those updates. Posteriors artificially concentrate (killing exploration) a little more on each deploy. _(verified)_ - **H2. The production judge panel is three copies of the same model.** `apps/oshun/content-service/src/main.ts:33-43` builds all three `JudgePanel` members from one provider and one `judgeModel`, varied only by temperature (0.2/0.4/0.6). Median-of-correlated-draws ≈ a single judge, so the disagreement→`needsHuman` escalation, LCB pessimism, and active-learning routing all systematically underestimate uncertainty. The lib's own header (`judge-panel.ts:4-6`) prescribes "different model and/or rubric framing"; distinct model chains are already pinned in `model-routing.ts`. _(verified)_ - **H3. No prompt-injection fencing on judged text, and no judge red-team suite.** `libs/shared/content-quality-judge/src/judge-engine.ts:207-213` (and the pairwise path, and `player-proxy.ts`) embed the artifact after a bare `TEXT:` with no delimiters and no "never follow instructions in the text" clause; a grep for fencing language across the lib returns nothing. In a self-gating autonomy pipeline, a candidate that captures the judge gates its own release — and best-of-N then _preferentially selects_ injected candidates. _(verified)_ - **H4. Job-drain double-execution race.** `apps/oshun/bff/src/generation/jobs-route.ts:142-159` snapshots `status === 'queued'` once and never re-checks before flipping each job to `running`; the loop `await`s between jobs. Concurrent drains — operator tick, autonomy producer `runJob`, cycle driver are all callers — can both capture the same job: double provider spend, result/governance clobber, duplicate gallery records (`output-catalog.ts:182`), and the corollary `runJob` failure "unknown error" on reading a `running` job. _(verified)_ - **H5. The publish-readiness pipeline has zero production callers.** `libs/oshun/studio-authoring/src/editorial-lifecycle/pipeline.ts:50,155` (`evaluatePublishReadiness` — Sophia evidence pack, Lilith tone-review id, Isis release-gate ids, rights/provenance bundle, localization ratio — and `buildPublishingPipelineDecision`) are defined and tested but unbound; the autonomous publisher's default gate set is a single Lilith tone gate (`apps/oshun/bff/src/agentic/autonomy-bindings/editorial-publisher.ts:81`). An autonomous text item can publish with no evidence pack, no rights bundle, no localization check. _(verified — grep shows definitions only)_ ### MEDIUM **Creative-autonomy plane** - M1. Unbounded repeat feedback: `autonomy-route.ts:545-597` + `learner.ts:177-186` have no per-(user,item) dedupe or rate cap — one user can pump a category's posterior arbitrarily (found independently by two agents). - M2. Failed-topic cooldown keyed to `submittedAtUnixSeconds`, not resolution time (`director.ts:375`) — an item rejected 6.9 days after submission gets ~0 effective cooldown. - M3. Calibration leniency ratchet: with no explicit rating, `humanEquivalent` = the outcome reward constant (`learner.ts:155-159`); approving a mediocre 0.55-score item pushes the category's additive offset up by λ·0.45 — a lenient reviewer progressively opens the gate for everything in the category. - M4. Gate-retry can discard a judged champion: if the retry attempt yields zero complete variants, `director.ts:558-575` reports `production-failed` and the attempt-0 below-bar champion never parks for human review. (LOW-MED) Related accounting: `completeVariantCount` holds only the last attempt while `variantCount` accumulates all attempts (`director.ts:556-557`). - M5. LLM-synthesized topics bypass `sanitizeTopic` (`ideation.ts:241-250` admits `proposal.topic` verbatim; source-signal topics get the 160-char single-line defense at `signals.ts:100-101`). (LOW-MED) **Judge engine / content-service** - M6. Smart-apostrophe blindness in the slop detector: `slop.ts:64,84,90-91` patterns are byte-exact straight-apostrophe (`"couldn't help but"`), but LLM prose emits U+2019 — several high-weight tells silently never fire. - M7. Run-id collision overwrites audit records: `content-pipeline-service.ts:95-97` derives `run_${briefId}_1` deterministically; resubmission overwrites the prior persisted run. - M8. No compute ceiling on the §3.2 HTTP path: `http-router.ts:32-40` validates 4 strings; `candidateCount` has no upper bound and the `compute-budget` meter is never wired into the dispatcher — one POST can trigger thousands of writer+panel calls. - M9. The deployed content-service mount omits the grounding gate entirely (`main.ts:81-86`) — canon-inventing prose ships ungated in the one production wiring. - M10. `external-benchmark.ts:152-216` labels authored fixtures with real dataset identities (`eq-bench-creative-v3`, `litbench`) and invented `publishedScore`s — honest header, dishonest-at-a-distance reports. **Orchestrator / agent-loop** - M11. **Last-draft-wins revision:** `critique-revise.ts:66-83` returns the final iteration unconditionally — a revision scoring 0.4 ships over a 0.75 draft 0. Same pattern in `libs/shared/ai` `reflexion.ts:103-115`. Cheap fix (track argmax), high content-quality impact. _(verified)_ Note: the shared judge lib's `self-refine.ts:107-116` does this correctly; the orchestrator the plane actually uses does not. - M12. Self-grading deterministic critic: `default-critic.ts:74-185` reads `measuredLufs`, `goldSourceIds`, `referenceImage` from the artifact being graded — a generator that echoes the gold set passes trivially. Grading references belong on `node.params` or an independent measurement seam. - M13. Governance spend accounting is post-hoc and success-only: `router.ts:108-117` records a dispatch only on success and at a flat per-domain estimate; revision iterations and judge tokens are invisible to `maxDispatches`/`maxCostUsd`. Also `options.signal` is never checked in the dispatch loop (`router.ts:60-126`) — aborting a run keeps dispatching. - M14. Residual banned-pattern stubs in `libs/shared/ai/src/prompts/` `testing.ts:732,794-812,933-937` (placeholder quality eval, `pValue = isSignificant ? 0.01 : 0.5`) and `advanced/index.ts:456-460` ("In production, would use embeddings"). Off the orchestrator path but exactly the pattern the repo bans — should be gutted to fail-loud. **BFF generation / autonomy bindings** - M15. The media autonomy path is deterministically dead while advertised live: `server.ts:1158-1164` marks image/audio/video plannable on credential presence, but every adapter ships measurements that guarantee the Isis gate blocks or holds (provenance `false`, watermark `null` vs floors 0.99–1.0, `qualityAggregate: null` in all six adapters — _verified_), and `runJob` throws on both `blocked` and `needs_review` (`generation-producer.ts:52-57`). With real keys the plane pays providers every media cycle and discards every artifact. Honest fail-closed governance — but an economic drain plus a misleading "live domains" signal until the deploy-bound watermark/signing steps exist. - M16. `needs_review` conflated with failure in the autonomy producer — a gate verdict of "held for a human" aborts the variant instead of routing the held media into the plane's review queue. - M17. Media "judged" by JSON metadata: `create-plane.ts:125` gives all domains `createLlmJudgeCritic`, which serializes `{jobId, kind, url, provider}` into a text prompt — a real LLM call producing an epistemically meaningless score, which then drives the revise loop. - M18. Latent `analysis` gate bypass: `release-gate.ts:132-138` releases `releaseKind:'analysis'` outputs with no `url` ungated, but narration already returns inline `audioBase64` with no url — any future executor tagging inline media `analysis` skips the gate silently. (LOW-MED) - M19. Unvalidated executor casts (`request as IllustrationCardInputs`, `image-executor.ts:61`; admin raw-enqueue accepts any body, `jobs-route.ts:288-296`) → `"undefined — undefined style"` prompts sent to paid providers. (LOW-MED) Provider misattribution: `image-executor.ts:67` stamps `provider: 'runpod'` for the Stability client. (LOW) **Editorial / governance** - M20. Gold-set **consumption** dormant: `goldSetFromEntryRecords`, `heldOutSplit`, `queryEntries` have no production consumers — operators can promote entries but assembled sets never reach judge calibration (MED-HIGH: the learning loop's back half doesn't run). - M21. Editorial calendar machinery dormant: `evaluateEditorialPublishWindow`, embargo, recurrence (`lifecycle.ts:176-208`) have no callers; the director publishes immediately on approval (`director.ts:271-273`). The plane's own `planCalendar` (`selection.ts:193-228`) is likewise real, tested, and never called. - M22. Voice-disclosure gate missing: `release-gate-model.ts:20-23` promises `spoken-synthesis-disclosure-audible` / `voice-watermark-every-turn` / `watermark-every-frame`; only generic `watermark-coverage` floors exist and no audible-disclosure gate at all. - M23. Champion-challenger uses an independent two-proportion z-test on paired samples (`champion-challenger.ts:59-66`) — McNemar is the correct paired test; p-values are miscalibrated. (LOW-MED) Editorial records/transition history are in-process `Map`s — restart erases the audit trail of which gates each published item passed (`editorial-publisher.ts:83,110`). (LOW-MED) ### LOW (selected) - Pointwise "variance" conflates the temperature ladder (t, t+0.1, t+0.2) with judge noise (`judge-engine.ts:220-224`). - `cohenKappa`/`pearson` silently truncate mismatched arrays (`calibration.ts:66-67,86`). - Welch drift on n=1 windows → z=±Infinity → spurious quarantine (`drift.ts:72-81`); `delta===0` counted as 'better' in editorial drift (`drift-detection.ts:60-65`). - `structured-output.ts:119-154` ignores `stopReason` — truncated JSON burns all retries at the same `maxTokens`. - Plan schema has no `maxItems` on nodes; default governance gate is `ALLOW_ALL_GATE` (`plan-schema.ts:21-24`, `router.ts:54`). - `formattingDensity` counts plain newlines as formatting markers (`reward-model.ts:82-88`). - Review queue grows unboundedly and the full store (variants included) is serialized on every persist (`review-queue.ts:114-183`, `plane-persistence.ts:42-50`). - `aggregateFeedback` blends mixed-bundle ratings silently (`feedback.ts:50-64`); `@oshun/agent-pipelines` is a 2-line re-export facade; `EditorialChecklistEntry` has zero consumers. --- ## §2 SOTA gaps — where further techniques/layers/algorithms would raise quality Consolidated across the five units, deduplicated, ranked by expected content-quality impact. "Now" = implementable on this box today; "gated" = needs live creds/data/GPU. ### Theme 1 — A media quality plane (the largest gap: nothing looks at the pixels) 1. **VLM-as-judge for images** — score composition/artifacts/text-legibility/ prompt-fidelity by sending the produced URL through the existing OpenRouter multimodal path. Slots: compute inside `adaptImageProviderGenerate` (`image-provider-env.ts`) feeding the currently-null `qualityAggregate`, plus a domain-routed `criticFor(node.domain)` map replacing the blind JSON critic in `create-plane.ts:125`. **M, now** (same key that gates the plane). 2. **Cross-modal consistency check** — VLM caption/VQA of the produced image → embedding cosine vs the brief + dependency text; a new media-aware `CompellingnessJudge` (all five current judges throw on media-only variants — `variant-text.ts:16-52`). **M, now.** 3. **Audio quality probes** — ffmpeg is installed on-box: `ebur128` loudness, `silencedetect`, `ffprobe` duration-vs-request on narration (`audioBase64`) and music URLs; emit `measuredLufs`/duration so the orchestrator's existing deterministic critic consumes an _independent_ measurement (also fixes M12 for audio). ASR round-trip WER: **gated**. Rest: **M, now.** 4. **Video probes** — `ffprobe` duration/resolution vs request; sample N frames → per-frame moderation with the _existing_ image safety classifier (closes video's `safetyScanScore: null`); frame-pair SSIM variance for flicker. Slot: `adaptVideoProviderGenerate`. **M-L, mostly now.** 5. **Best-of-N for media with seed control** — enqueue K=2-4 jobs with varied seeds in the image/video `DomainGenerator`, select by the VLM scorer; record losers as non-released candidates. Add `seed` to provider requests for reproducibility (nothing threads one today). **M, now; spend-gated in degree.** 6. **Structured media prompt construction** — replace `"${subject} — ${stylePreset} style"` (`image-executor.ts:43`) with a declared per-`stylePreset` style-guide table (palette, composition, negative prompts — deterministic brand consistency) plus an optional LLM "prompt director" expansion pass in the autonomy media generators. **S-M, now.** 7. **Provider quality routing** — `provider-measurement.ts` builds envelopes but measures nothing and keeps no history; one hardcoded provider per kind, no fallback. Per-kind registry with measured win-rate/latency/cost feeding weighted routing. **L, partially data-gated.** 8. **Caption/dub faithfulness** — ASR the dubbed track → back-translation → semantic similarity vs source. **L, gated (ASR creds).** ### Theme 2 — Close the learning loop's back half 9. **Preference-pair persistence (DPO/RM-ready)** — tournament `ComparisonRecord`s and losing variants are computed then discarded (`director.ts:577-579`), and best-of-N `ranked` pairs likewise; persist `{brief, chosen, rejected, margin, source: tournament|human, provenance}` JSONL. This is free training data already being paid for. Slots: after `runVariantTournament` + a `preference-export.ts` in the judge lib. **S-M, now** (training itself GPU/creds-gated). 10. **Wire the gold-set → judge-calibration loop** (M20) — scheduled job: assemble per-family sets, `heldOutSplit`, re-score holdout with the panel, compute κ via the existing `calibration.ts`, recalibrate per-category gate bars. All machinery exists; nothing runs it. **M, now.** 11. **Human post-edit diff channel** — `editedContentRef` and `computeOutputDiff` exist but the learner has no edit channel; on `edit`-kind decisions compute changed-fraction + section diffs, feed as shaped reward and store as prompt-revision exemplars. The strongest quality signal there is. **S-M, now.** 12. **Per-dimension human verdicts** — judges emit 7 dimensions; humans return one scalar (`autonomy-route.ts:265-267`). Add per-dimension agree/override chips → per-dimension calibration instead of one additive EMA. **S, now.** 13. **Replace the additive calibration offset with Platt/isotonic scaling** per category once observations ≥ N (fixes M3's leniency ratchet structurally); keep the EMA as cold-start. Slot: `learner.ts:152-190`. **M, now.** 14. **Engagement-outcome attribution** — published-item live metrics → fractional bandit reward + calibration observations; full-auto categories currently learn nothing at all (`director.ts:695`). Slot: a poller binding beside `createAutonomyCycleDriver`. **M, data-gated.** ### Theme 3 — Judge robustness 15. **Heterogeneous judge panels + agreement weighting** — route the three panel members through distinct pinned model chains (fixes H2; routes already exist in `model-routing.ts`); later add per-member κ-derived weights (inverse-variance pooling). **S now; weights data-gated.** 16. **Injection fencing + a judge red-team regression suite** — delimit judged text, add instruction-immunity clauses, and ship a `judge-red-team.ts` probe library (score-inflation injections, rubric echo, flattery, homoglyph slop evasion, markdown stuffing) run as a CI gate with scripted providers (fixes H3). **S-M, now.** 17. **Turn on multi-sample judging in production** — the discipline exists (`samples` option, temperature-perturbed means) but the BFF wires the default `samples: 1` (`create-plane.ts:130`). Also fix the temperature ladder so variance is measured at a fixed operating point. **S, now.** 18. **Cost-aware cascaded judging** — cheap screen (single fast judge + slop/lexical heuristics) auto-resolves scores far from threshold; full panel only near the bar or on high variance; pairs with the active-learning band. **M, now.** 19. **Exemplar-anchored judging** — append 2-3 scored gold exemplars per content type (already curated in `benchmark.ts`) to `renderRubric`; optional retrieval-augmented judging (canon passages in judge context). **S, now.** 20. **Reconciliation/debate round before human escalation** — on `disagreement > threshold`, each judge sees the others' scores+reasons and re-scores once; escalate only if still split. Directly lowers human-touch rate. **M, now.** 21. **Rubric-version regression gating** — rubric semver exists but nothing forces a κ-non-regression run on the gold benchmark when a rubric/prompt/slop-list bumps; reuse champion-challenger with the _judge_ as the variant. Pin rubric versions into `GateVerdict`, editorial history, and gold entries (one field each). **M, now.** ### Theme 4 — Production-time search (plan → draft → revise) 22. **Plan-quality judging before spend** — plans are only structurally validated (`planner.ts:97-104`); an LLM plan-critic (coverage vs brief, node atomicity, dependency sanity) rejecting/re-planning below bar is the cheapest quality lever in the loop. **S-M, now.** 23. **Multi-plan sampling** — sample 2-3 decompositions at temperature, score with the plan critic, pick argmax (ToT-lite). **S-M, now.** 24. **Best-draft tracking + node-level best-of-N** — fix M11 (argmax draft), then K parallel first-pass samples with the critic picking the revision seed. Slot: `critique-revise.ts`. **S, now.** 25. **Dimension-targeted critics** — `JUDGE_SCHEMA` is `{score, feedback}`; extend to per-dimension scores + evidence quotes + compare-to-prior so revision feedback is targeted, and thread the full critique trajectory (`feedbackHistory`) instead of last-critique-only. **S, now.** 26. **Retrieval tools for writer nodes** — the fully-built `AgentLoop`/`ToolRegistry` is never used for generation; an `AgentLoop`-backed text generator with retrieval (Sophia BM25 exists in-repo) lets writers ground and cite honestly — and fixes half of M12 by making citations verifiable. **M, now.** 27. **Replanning on node failure** — failed nodes cascade-fail dependents with no retry/repair; add `repairPlan(plan, failures)` re-decomposition and `requireMetDependencies` policy. **M, now.** 28. **Parallel frontier execution + context budgeting + cache/resume** — Kahn-frontier scheduling with `maxParallelNodes`; budgeted `summarizeDependencies` per node token allowance (yemaya adapter JSON-dumps full upstream outputs today); content-hash cache on the generator registry and `routePlan(plan, {seedArtifacts})` resume. **M each, now.** 29. **Per-node model routing** — strong model for prose nodes, cheap for outline/metadata; `modelFor(node)` in the orchestrator config. **S, now.** ### Theme 5 — Ideation & portfolio intelligence 30. **Multi-candidate ideation** — one synthesizer call per cycle today; N independent samples at varied temperature, deduped by `topicSimilarity`, union into candidates. **S, now.** 31. **Semantic embedding novelty** — acknowledged seam (`ideation.ts:318-320`); trigram Jaccard misses paraphrase duplicates ("blood moon" vs "total lunar eclipse"). Injectable `TopicEmbedder` (MiniLM-class CPU embeddings run on this box) for novelty scoring, cooldown matching, and cluster guarding. **S, now.** 32. **OPRO-style rubric/brief evolution** — mandatory rejection reasons and tweak directions are stored but never mined; a periodic reflection job clusters them per category into proposed rubric addenda and brief-template mutations, emitted as `SteeringSuggestion`s (human-owned, tighten-only preserved). **M, now.** 33. **Cross-cycle style memory** — per-category `StyleMemo` (LLM-summarized contrast of approved vs rejected champions, versioned like steering) injected into `composeBrief`; tweaks are currently one-shot and champion traits are never distilled. **M, now.** 34. **Persona/audience simulation judges** — all judges score generic quality against one hardcoded register ("curious general audience"); persona-conditioned judges ("would this reader click/finish/share?") per category audience card slot directly into the partial-panel machinery. **S, now.** 35. **Uncertainty-aware compute routing** — confidence currently only fails the gate; escalate confidence-only failures to more samples/judges, route persistent low confidence to humans rather than parking, and add per-category `stakes` scaling variants/samples (value-of-compute). **S-M, now.** 36. **Contextual bandits + exploration floor** — LinTS/logistic TS over signal-kind/timeliness/novelty features behind the existing `CategorySampler` interface once volume justifies it (**M-L, data-hungry**); meanwhile a min-trial quota so low-prior categories are never starved (**S, now**). 37. **Wire calendar-level portfolio planning** — `planCalendar` is built, tested, and never called (M21); schedule approved publications and add a week-level angle-mix/variety objective. **S wire / M objective, now.** ### Theme 6 — Editorial & post-publish 38. **Bind the publish-readiness pipeline** (H5) so autonomous publishes require the evidence pack/rights/localization gates by content type; bind per-content-type `EditorialChecklistEntry` data. **S-M, now.** 39. **A/B or holdout publishing** — publish 2 gate-passing variants to segments, measure, feed back (reuse champion-challenger with the McNemar fix). **M-L, data-gated.** 40. **Staleness re-review sweeps** — reuse the dormant recurrence machinery to create re-review calendar entries for aging published artifacts. **S-M, now.** 41. **Inter-rater reliability of humans** — sampled dual-assignment in the review queue + Cohen's κ per reviewer pair (no human-vs-human agreement is measured anywhere). **M, data-gated.** 42. **Post-publish quality monitoring** — corrections, complaints, engagement regressions → `ReviewOutcome` + auto re-review. **M-L, data-gated.** --- ## §3 What is already at or beyond SOTA (keep and build on) - Discounted Thompson sampling with Beta-preserving decay + two-level (category, angle) arms + exact Marsaglia–Tsang sampling (`bandit.ts:136-144`, `selection.ts:91-107`). - Bradley–Terry MM tournament + both-orderings position-bias mitigation with inconsistency→tie collapse (`tournament.ts:58-96`, `llm-judges.ts:179-186`, `judge-engine.ts:292-326`). - Pessimistic-LCB best-of-N resisting reward-hacking, plus excess-correlation reward-hacking detection that auto-hardens selection (`best-of-n.ts:60-65`, `reward-model.ts:173-190`). - Corpus-level diversity gates failing homogeneous batches whose items individually pass, plus generation-time novelty pressure (`corpus-gate.ts`, `diversity-pressure.ts`). - Partial specialist panels with coverage-discounted confidence and typed refusal of uncovered dimensions (`rubric.ts:147-164`). - Fail-closed single-path governance: one executor→Isis-gate→catalog drain for operator, curated, and autonomous jobs; evidence-mandatory release gates with content-hash-bound promotion and named signoff; human-review trigger catalog where an unassigned reviewer blocks release. - Honest fail-loud posture everywhere the audit looked: zero result-faking stubs on the production paths (the only exceptions found are the M14 files, which are unreachable from these paths). - AIMD concurrency governor honoring provider retry-after; durable snapshots for jobs/catalog/plane; full provenance chains (signal → idea → scores → tournament → gate) on every review item. - Statistically real drift (direction-aware Welch z), κ-calibration with textbook-value tests, champion-challenger promotion gates, salted-SHA-256 deterministic holdout splits. --- ## §4 Priority worklist **P0 — verified defects (fix before further feature work)** 1. H4 drain race (re-check `status === 'queued'` per job; or a claim flag). 2. H1 restart double-learning (persist `learnedItemIds`, or derive from learner state). 3. H2 heterogeneous panel (route members through the pinned distinct chains). 4. H3 fencing + minimal red-team probes. 5. H5 bind publish-readiness gates for autonomous publishes. 6. M11 best-draft argmax; M16 route `needs_review` media to the review queue; M6 apostrophe normalization; M9 grounding gate in the deployed mount; M1 feedback dedupe; M14 gut the residual `prompts/` stubs to fail-loud. **P1 — highest quality-per-effort, implementable now** Media plane first (items 1-6): VLM image judge + cross-modal check + ffmpeg audio/video probes + media best-of-N + style-guide prompts — this converts media from "safety-gated only, judged blind" to first-class judged content. Then learning-loop back half (9-13): preference-pair logging, gold-set→κ job, per-dimension verdicts, post-edit diffs. Then judge robustness (15-21) and production search (22-29): plan judging, multi-plan, retrieval-grounded writers. Then ideation/portfolio (30-35, 37). **P2 — data/creds/GPU-gated (build seams now, bind at deploy)** Engagement attribution (14), contextual bandits (36), DPO/RM training on the logged pairs, ASR-based WER/dub checks (8), A/B publishing (39), inter-rater κ (41), post-publish monitoring (42), live κ calibration on real provider traffic. --- ### Method notes - Five audit units: `libs/oshun/creative-autonomy`; `libs/oshun/creative-orchestrator` + `libs/shared/ai` + `libs/shared/content-eval`; `apps/oshun/bff/src/generation` + `src/agentic`; `libs/shared/content-quality-judge` + `libs/shared/content-release-gates` + `libs/oshun/content-service`; `libs/oshun/studio-authoring` + `libs/oshun/generation-control-isis` + `libs/oshun/agentic-studio` + `libs/oshun/agent-pipelines`. - Each unit: full read of core sources, stub-indicator grep battery, verification of prior-ledger claims (49/49 prior claims checked: 45 CONFIRMED, 4 PARTIAL — the PARTIALs are recorded above as M-items or caveats; 1 claim REFUTED at the orchestrator layer: "revision kept only on improvement" = M11). - All five HIGHs re-verified line-by-line by the coordinating session before inclusion. MED/LOW findings are single-deep-read unless marked verified. - No code was modified in this pass — audit only.