Disciplines · Audits

V1 Autonomous Content Agents — Verification Audit — 2026-07-13

autonomous content stack for V1 is real, SOTA-grade, adversarially clean, and fully green in tests and typecheck.

7sections6 minread

On this page

Question audited: are all the agents that autonomously create content for V1/ SOTA and production-ready?

Method: main-loop verification (no agent fan-out, per session limits) against the prior ledgers — V1_AUTONOMOUS_CONTENT_SOTA_AUDIT_2026-07-02.md (defect ledger H1–H5, M1–M23, L1–L10, SOTA items R1–R42), V1_SOTA_AUDIT_REMEDIATION_TODOS_2026-07-02.md (42/42 marked done), and V1_AUTONOMOUS_CONTENT_LEDGER_APPLICATION_2026-07-04.md (P0–P2 bindings). This pass does not take the ledgers on faith: every HIGH fix was re-read in current source, the adversarial stub battery was re-run over every audited unit, and every core suite was re-executed on this box today.

Verdict#

Yes at the code level, with an honest, explicitly-drawn deploy boundary. The autonomous content stack for V1 is real, SOTA-grade, adversarially clean, and fully green in tests and typecheck. What remains open is exactly what the repo's own ledgers say remains open: live-provider credentials and a handful of deploy-/policy-bound seams that fail loud rather than fake success. Nothing found in this pass contradicts a single [x] in the remediation ledger.

1. The agent inventory (what "the agents" are)#

Layer System Role
Director plane @oshun/creative-autonomy signals → bandit ideation → K variants → ε-BT tournament → calibrated 7-dim gate → review queue → publisher → preference learner
Production search @oshun/creative-orchestrator (+ @oshun/ai) plan-DAG decomposition, plan judging, node best-of-N, critique-revise with argmax draft, replanning, per-node model routing
Judge plane @shared/content-quality-judge + apps/oshun/content-service heterogeneous 3-model panels, injection-fenced prompts, cascaded/reconciliation judging, slop/reward-hack detectors, gold-set κ calibration
Execution & governance apps/oshun/bff/src/generation + src/agentic, @oshun/generation-control-isis, @shared/content-release-gates 7 media executors behind the single fail-closed Isis dispatch seam; watermark + C2PA + safety + quality probes; quotas, budgets, drain loop
Composed-work ledgers @isis/workflows ledger planes (07-04 wave) session/briefing producers, score⇄ledger adapter, lesson spine, claim retakes, persona→bank binding, node-addressed retakes
Editorial plane @oshun/studio-authoring + autonomy bindings publish-readiness gates, editorial calendar, checklists, McNemar champion-challenger, staleness sweeps

2. Re-verified in current source (this session)#

All five HIGH defects from 07-02 are fixed in HEAD:

  • H1 — director.ts:325-331 snapshotLearnedItemIds/restoreLearnedItemIds, wired through plane-persistence.ts:55,80-83 with a pre-migration fallback. No restart double-learning.
  • H2 — apps/oshun/content-service/src/main.ts:47 builds the panel through resolveJudgePanelModels(3, isAvailable): distinct models per member with degraded-diversity auditing.
  • H3 — judge-engine.ts:32-53: UNTRUSTED_BEGIN/END fencing, delimiter-breakout sanitization, immunity clause; red-team regression suite in the lib.
  • H4 — jobs-route.ts:182-183 re-checks status === 'queued' per job before claiming (drain race closed).
  • H5 — editorial-publisher.ts:24,90: default gate set is the per-content-type publish-readiness pipeline (createPublishReadinessGates), not a lone tone gate.

The 07-02 audit's biggest structural gap — "the quality engine is text-only, media is deterministically dead" — is closed by the post-audit isis wave (commits 812ad33e…a038318c…): real image watermarking + C2PA signing + safety scan + ffmpeg quality prober (image now auto-releases through full governance), real narration/video/music governance (the last media kinds clear the release gate), signed C2PA manifests across image/video/narration/music, audio/video watermark modules, per-user daily quota gate, durable generation budget gate, real spend capture, media best-of-N with seed control, provider quality routing, dub-faithfulness scoring. The two remaining qualityAggregate: null sites were read in context: both are honest fail-closed defaults (pre-governance initial signals in image-provider-env.ts:124; deploy-bound dub watermarker/signer in caption-dub-provider-env.ts:343 with blocking behavior documented inline).

SOTA technique layer spot-checked present and tested: cascade.ts (cost-aware cascaded judging), reconciliation/debate round in judge-panel.ts/judge-engine.ts, plan-judge.ts (plan-quality gate before spend), preference-log.ts (DPO/RM-ready preference-pair persistence), persona-simulation judges, Platt/isotonic calibration, exploration floors + LinTS contextual bandit, McNemar paired champion-challenger, rubric-version regression gating.

3. Adversarial stub scan — zero actionable hits#

The mandatory grep battery plus the silent-stub scan (Math.random in deterministic paths, fabricated-success catch blocks, excuse comments) was re-run across all audited units (creative-autonomy, creative-orchestrator, content-quality-judge, content-release-gates, content-service lib+app, bff/generation, bff/agentic, studio-authoring, generation-control-isis, agentic-studio, agent-pipelines, shared/ai). Every hit was read in context; all are false positives (reward-hacking detector vocabulary, persona-simulation judges, annotated non-secret id suffixes, Fisher–Yates/A-B stochastic gating). The M14 fabrication stubs (prompts/testing.ts fake p-values, advanced/index.ts embeddings excuse) are confirmed gutted — testing.ts now computes a real Welch t-test with Student-t two-tailed p-values and fails honest on n<2.

4. Test + typecheck evidence (executed on this box, 2026-07-13)#

Suite Result
@oshun/creative-autonomy 248/248 (16 files)
@shared/content-quality-judge 280/280 (34 files)
@shared/content-release-gates 19/19
@oshun/generation-control-isis 410/410 (20 files)
BFF src/generation + src/agentic 587 passed, 1 creds-gated skipIf(!ENGINE) skip (66 files)
@oshun/creative-orchestrator 98/98
@oshun/studio-authoring 138/138
@oshun/content-service (lib) 26/26
@isis/workflows (ledger planes/producers) 376 passed, 1 gated skip (34 files)
apps/oshun/bff tsc --noEmit exit 0

5. The honest boundary (what "prod ready" does NOT include)#

These are not defects; they are the deliberately drawn seams, all fail-loud:

  1. Live provider credentials are deploy-bound (V1/TODOS.md:2942) — the plane only advertises domains whose outputs can actually release (M15 fix); with no creds it refuses rather than fabricates.
  2. Deploy-bound media steps — caption-dub watermark/C2PA signer (blocks on the floors until it lands), ASR/WER transcriber seam, FVD evaluator seam for Living-Scene continuity (explicit {absent, policyValue} sentinel that blocks any finite threshold).
  3. Data-gated learning — engagement-outcome attribution poller, per-member κ panel weights, inter-rater dual-review κ, A/B live traffic: real machinery, accrues at deploy.
  4. Publish autonomy is per-surface POLICY, kept on purpose — Veritas named attestation, Nisaba scholarly authority, persona consent, crisis screens (pass rate pinned 1.0), minor/likeness/cloned-voice human triggers. Autonomy raises draft throughput, not publication authority.
  5. BFF ledger stores are BFF-lifetime in-memory per the house store pattern (durable snapshots exist for jobs/catalog/plane).
  6. Residual V1 open items are deploy/ops (creds, Stripe live mode), other-studio verification sweeps (Bellona/Hathor), a human lineage-sensitivity review, and two mobile mentor-presence delivery refinements — none are content-agent code gaps.

6. Bottom line#

  • SOTA: yes — the stack implements the 07-02 audit's full R1–R42 catalog (media VLM judging, cross-modal consistency, ffmpeg probes, preference-pair logging, gold-set→κ calibration, cascaded + reconciliation judging, plan judging + multi-plan sampling, retrieval-grounded writers, OPRO-style steering mining, style memos, contextual bandits with exploration floors, publish-readiness binding, staleness sweeps), several of which remain ahead of common 2026 practice (discounted Thompson sampling with exact Beta-preserving decay, both-orderings BT tournaments, pessimistic-LCB best-of-N with excess-correlation hack detection, coverage-discounted partial panels, content-hash-bound fail-closed governance).
  • Prod-ready: yes at the library + wiring level — everything compiles, typechecks, and passes its suites today; zero result-faking stubs; every absent capability is an explicit fail-loud seam owned by a named deploy step. Flipping it live is a credentials-and-ops exercise, not a code exercise.