Disciplines · Audits

V1 Residual Audit — Sophia Grounding / Isis Generation / Audience Tiers / Media / Gallery / Search

output runs evaluateGenerationRelease (jobs-route.ts:238), fail-closed on absent/invalid releaseMeasurement; provider adapters source real governance signals (image-provider-env.ts:102, honest null→block for unscanned /

2sections10 minread

On this page

Slice: V1/features.md lines 2815–3609. Read-only static audit, 2026-06-11. Evidence = current source only. Recorded V1.x deferrals (search-discovery retirement E5), D1-covered /domains/*, and env-gated fail-closed providers awaiting creds are NOT findings.

The 2026-06-10 ground-truth headline for this slice — "Honest gate keystone real; verification loops + gallery unwired" — is largely accurate. C6 wired the admin/operator gallery write path; the customer gallery and the verification loops remain unwired, and an ungated job front-door slipped in.


Generation job lifecycle routes are unauthenticated (enqueue / drain / read-by-id)#

Severity: P0-SEC

Evidence: apps/oshun/bff/src/generation/jobs-route.ts:204 (POST /v1/generation/jobs), :216 (POST /v1/generation/jobs/process), :264 (GET /v1/generation/jobs/:jobId) — none carries a preHandler. The sibling operator routes in the same file (GET /v1/generation/jobs :145, POST …/:jobId/retry :176) ARE gated with { preHandler: [abuseProtection, authProtection] } + hasAdminScope, proving auth was intended on this surface and these three were missed. app.ts has no global onRequest auth hook (only an onResponse hook at :707).

Spec promise: "Raw provider machinery … is never customer-facing — Isis is the only path" (features.md 3134-3136); entitlement-gated curated cards (3225-3229).

What the code actually does: Any unauthenticated caller can (a) POST /v1/generation/jobs with an arbitrary {kind, request} — bypassing the curated-card entitlement gate, raw-graph leak audit and field validation in curated-card-route.ts entirely, since those only guard the in-process enqueueGenerationJob function path, not this HTTP front-door; (b) POST …/process to drain the queue, firing real provider executors (Stability / Suno / ElevenLabs / Meshy / fal cost + DoS); (c) GET …/:jobId to read any job's result (asset URLs), governance, and request (prompts, userId, tenantId) by guessable id.

Fix sketch: Add { preHandler: [abuseProtection, authProtection] } + hasAdminScope/owner check to the enqueue, process, and get-by-id routes; scope get-by-id to the job's invoking user or an admin.


Operator Sophia evidence / contradiction / unsupported-claim queue is a hardcoded fixture#

Severity: P0-HONESTY

Evidence: apps/oshun/web/src/app/operator/admin/page.tsx:14 renders AdminInbox with getAdminInbox(); apps/oshun/web/src/lib/lilith-data/operator-depth.ts:340 (evidenceReview: { queue, sourceSets, retrievalSets, citationValidation, unsupportedClaims, contradictions }) is entirely hand-authored constants (TR-047 drift review, "Metis core-six", VR-088, fabricated unsupportedCount, contradictionCount, source excerpts). getAdminInbox (:263) merges live abuse-reports into items only — evidenceReview is passed through untouched from getAdminInboxFixture(). AdminEvidenceQueuePanel (components/evidence-admin/AdminEvidenceQueuePanel.tsx) makes zero fetch / /v1/ calls; it just renders props.data.

Spec promise: "Admin: evidence queue, source-set inspection, retrieval-set inspection, citation validation, unsupported-claim queue, contradiction queue, grounded-report promotion to publication" (features.md 2939-2941).

What the code actually does: Presents fabricated evidence-review records as live operator data on a real page route (/operator/admin), with no seedData/disclosure label (unlike the E1-covered Isis consoles). An operator cannot tell these aren't real cases.

Fix sketch: Either wire the panel to a real BFF evidence-queue route backed by @sophia/verification outputs, or production-gate the fixture behind an explicit seedDataNotice like the E1 Isis consoles.


Severity: P0-HONESTY

Evidence: apps/oshun/web/src/app/studio/generation-gallery/GenerationGalleryClient.tsx:708 mounts <GeneratedArtifactProvenanceInspector surfaceId="client-generation-gallery" …/> with no artifacts prop, so it defaults to GENERATED_ARTIFACT_PROVENANCE_RECORDS (components/provenance/GeneratedArtifactProvenanceInspector.tsx:57 / :426) — canned records (saraswati-raga-dawn-mix, fixed SHA-256s). The component never fetches the real C6 route GET /v1/generation/outputs/:outputId/provenance (generation/output-catalog.ts:244) and is not bound to the selected gallery record.

Spec promise: "Provenance-bundle inspector for every output (consent ID, prompt, model, watermark, timestamp, invoking user, tenant, workflow class, cost ledger entry)" (features.md 3434-3436).

What the code actually does: Shows the same hardcoded Saraswati/raga provenance bundle on every gallery surface regardless of which output is selected — disconnected from the live provenance store C6 actually populates.

Fix sketch: Fetch /v1/generation/outputs/:outputId/provenance for the selected record and pass the real bundle to the inspector; drop the static default on the live gallery surface.


Severity: P1

Evidence: apps/oshun/web/src/app/studio/generation-gallery/page.tsx:21 calls loadGalleryViewForRequest(); loader.ts:99 falls back to defaultContext (anon / contemplative / empty entitlement, :51) and defaultStore ({ records: [], lineage: [] }, :67) because bindGalleryStore / bindGalleryContext (:43 / :47) are never called anywhere in the web app (grep returns only the loader definitions + tests). In GenerationGalleryClient.tsx the action buttons data-action="launch-branch" (:647), "send-to-editorial" (:659), "send-to-asset-library" (:674), "takedown" (:689) have no onClick — the only handlers are setSelectedId/setBranchMode/stopPropagation.

Spec promise: "Unified output gallery in customer Studio … per-output lineage tree … replay-with-tweak … send-to-editorial, send-to-asset-library, and takedown flows" (features.md 3415-3441).

What the code actually does: Every customer sees an empty gallery (their own released outputs never appear), and the lineage/branch/replay/compare/bulk-action controls are inert. planBranch computes a client-side cost estimate but "Launch" submits nothing. The admin gallery (BFF /isis/output-gallery + bulk actions route) is real; the customer surface is not.

Fix sketch: Call bindGalleryStore/bindGalleryContext from a server integration that reads the per-user records from the C6 catalog; wire the action buttons to BFF endpoints (branch → enqueue, send-to-editorial / takedown → bulk-actions route).


Sophia fact-check / contradiction / unsupported-claim loops are unwired#

Severity: P1

Evidence: @sophia/verification (libs/sophia/verification/src/index.ts exports fact-check-loop, contradiction, unsupported-claim-loop) has no BFF or web consumer (grep for @sophia/verification across apps/oshun/bff/src

  • apps/oshun/web/src is empty). The BFF evidence adapter's getFactCheckReport returns an honest-empty report (results: [], overallCredibility: 0, sophia-read-adapters.ts:197) and getClaimVerification returns null (:194). buildSophiaGroundedAnswer (routes/domain-stubs.ts:411) does retrieval + extractive/abstractive composition with a groundingState/abstained flag but runs no claim extraction → independent retrieval → claim-to-source matching → unsupported flag loop, and no contradiction detection.

Spec promise: Fact-Check Loop (2866-2882), Contradiction Loop (2884-2897), Unsupported-Claim Loop (2899-2910) — all framed as launch-critical.

What the code actually does: Honest fail-loud seams (not result-faking stubs), but the promised verification loops and their queues do not run on any live path. This is the 06-10 "verification loops unwired" residual, still open.

Fix sketch: Wire @sophia/verification fact-check/contradiction loops behind a BFF route that consumes the grounded-answer claim list; surface the unsupported-claim/contradiction queues to the operator panel (see P0-HONESTY #2).


Civitai intake approval never produces a managed-models record (two sources of truth)#

Severity: P1

Evidence: apps/oshun/bff/src/isis/civitai-intake-store.ts:115 applyDecision only advances the queue entry's review state via applyReviewDecision; it never writes to any model registry. The creator-facing registry ManagedModelRegistryStore (apps/oshun/bff/src/isis/managed-models-store.ts:92) is read-only over a hardcoded SEED_MODELS array (SeedManagedModelStore.list, :82) — there is no addModel/promotion method, and the route admin-isis-model-registry.ts exposes only a GET.

Spec promise: "Normalization into Isis-owned model records and Oshun-controlled storage … Managed-models browser exposing only post-intake approved models" (features.md 3256, 3262-3263); "promotion decisions … review-to-staging".

What the code actually does: Intake approvals and the managed-models browser are disjoint stores. An approved Civitai model can never appear in the registry the creator surfaces read; the registry only ever shows its two seed LoRAs.

Fix sketch: On applyDecision('approve'/'conditional') write a normalized ManagedModelSummary into a mutable managed-models store keyed by the staging recipe; have the registry browser read it.


Generation jobs have no autonomous worker — customer submissions never complete on their own#

Severity: P1

Evidence: apps/oshun/bff/src/server.ts:951 registers the job route, but the only drain path is POST /v1/generation/jobs/process (jobs-route.ts:216), triggered manually from the Studio Job Management workspace (apps/oshun/web/src/components/studio/StudioIsisJobManagementWorkspace.tsx:18 JOBS_PROCESS_ENDPOINT / :268 "Drained the queue"). No setInterval/timer ticks the generation queue (unlike the reminder worker at server.ts:622 and deletion worker at :721).

Spec promise: Customer curated cards / music / sky-briefing / narration are contemplative-product generation surfaces (features.md 3220-3229, 3312-3343) implying the submission completes.

What the code actually does: A customer card submit returns 202 accepted and the job sits queued indefinitely until an operator manually drains it from the admin workspace — the customer-facing generation pipeline never finishes autonomously.

Fix sketch: Add an opt-in interval worker (mirroring OSHUN_REMINDER_WORKER_INTERVAL_MS) that calls the same drain logic, or drain inline at enqueue when a provider is configured.


Output-lineage operator console: unlabeled seeds + second lineage source of truth#

Severity: P2

Evidence: apps/oshun/bff/src/isis/output-lineage-store.ts:29-65 seeds SEED_OUTPUTS/SEED_EDGES (May-25 out_base_illustration_v2 → out_veritas_brief_001 → …) unconditionally — no production gate and no consoleSeedNotice (the route admin-isis-output-lineage.ts emits none, unlike the six E1-covered consoles). Separately, C6's recordReleasedGenerationOutput writes lineage into its own OutputLineage instance inside generation/output-catalog.ts:88, served via the provenance route — never into output-lineage-store.

Spec promise: Per-output lineage tree + takedown-cascade reach (features.md 3422-3433); E1 seed-data labeling on seeded operator consoles.

What the code actually does: The /isis/lineage-graph console shows seeded fixture edges as production lineage with no disclosure, and a genuinely released output's lineage (in the C6 catalog) never appears there — two disjoint lineage graphs.

Fix sketch: Production-gate + seedDataNotice the lineage store; have C6 register its edges into the same output-lineage-store the console reads.


Customer Sophia answer surface drops server-computed grounding state and claim labels#

Severity: UX

Evidence: buildSophiaGroundedAnswer (routes/domain-stubs.ts:464-478) returns groundingState (grounded/partial/abstained), claims (each with a retrieved/synthesized label), citationMap, and abstained. The customer form apps/oshun/web/src/app/sophia/SophiaAskForm.tsx:9-14 types the response as only { query, answer, confidence, citations } and renders only confidence/citations — the grounding-state indicator and per-claim labels are discarded. (Also a confidence enum mismatch: BFF emits grounded/cautious/unavailable; the web type expects high/medium/cautious/unknown, so the chip prints raw unavailable.)

Spec promise: "non-noisy grounding state indicator (color-and-label)" (2938); per-answer grounding state surfaced to UI (2844-2845); retrieved vs synthesized claim labels (2858-2860).

What the code actually does: Renders confidence + citation list only; the grounded/partial/abstained state and retrieval-vs-synthesis labels the backend already computes are invisible to the customer.

Fix sketch: Extend the web AnswerResponse to read groundingState + claims + abstained and render a labeled grounding-state chip and per-claim retrieved/synthesized badges.


Voice runtime watermark verifier absent on the customer playback path#

Severity: P2

Evidence: No customer playback component runs a watermark verify before audio plays (grep for watermark verify/tamper/intact in apps/oshun/web/src finds only operator/TTS-workspace controls and the static NarrationCard disclosure string NarrationCard.tsx:41 "Provenance and watermark are embedded"). The substantive gate is the release-time watermarkCoverage floor (generation/release-gate.ts:80, provider-measurement.ts:129), which is real; the playback-time verifier is not implemented.

Spec promise: "Runtime watermark verifier embedded in playback that flags missing or tampered watermarks before audio reaches user ears" (features.md 3361-3362).

What the code actually does: Relies on the release gate only; no playback-time tamper check. (The release-gate coverage makes this lower priority, but the promised verifier is a code task that is absent, not a deploy-cred gap.)

Fix sketch: Add a playback-entry watermark-verify call (against the provenance bundle's watermark hash) that blocks/flags before the audio element starts.


Positives confirmed real (not findings)#

  • Isis release gate keystone is genuinely wired end-to-end: every produced output runs evaluateGenerationRelease (jobs-route.ts:238), fail-closed on absent/invalid releaseMeasurement; provider adapters source real governance signals (image-provider-env.ts:102, honest null→block for unscanned / un-watermarked / unsigned outputs). No fabricated passing constants.
  • C6 catalog write (generation/output-catalog.ts) writes real gallery+lineage+provenance only on complete; blocked/held outputs never cataloged; idempotent on retry; prod starts empty.
  • C7 user-object search (search/user-object-candidates.ts) really pools the member's saved items / collections / notebooks / habits, domain-authorized, honest-empty.
  • Audience-tier surface boundary is properly enforced deny-by-default at the proxy: applyStudioBoundary (proxy.ts:247) hard-blocks (404) or Yemaya-CTA redirects AAA-only /studio/isis/* routes for non-AAA sessions via the real @isis/entitlements resolveStudioBoundary.
  • Curated-card / music / nyx-3d submission routes are real, server- authoritative entitlement + envelope + leak-audit gated, fail-closed 503 without a provider.
  • E1 seed-data notices are correctly applied to the six named Isis consoles.

Severity counts#

Severity Count
P0-SEC 1
P0-HONESTY 2
P0-STRUCT 0
P1 4
P2 2
UX 1
DEPLOY 0
Total 10