# Creator Surfaces, Voice, Music, and 3D Generation This page documents the creator-facing generation surfaces that sit on top of the V1 generation pipeline: the curated creator cards (illustration, narration, ambient audio, explainer, caption/dub, accessibility, Living Scene), the voice and music subsystems, and 3D generation and post-processing. It serves curated creators on the contemplative product and — for the heavier machinery — AAA-tier creators in Yemaya Studio, while the underlying execution providers stay operator-controlled. It is the surface layer above [Isis Generation Control](./isis-generation-control.md), [Generation Audience Tiers and Surface Boundaries](./generation-tiers-and-surfaces.md), and [External Model Intelligence and Execution Providers](./external-models-and-execution.md); where every produced asset lands afterward is the [Output Gallery, Lineage, Branch, and Replay](./output-gallery-lineage.md). The single rule that shapes every surface on this page: **raw generation machinery never appears on the contemplative product, and no live provider call ever happens without a deploy-time credential.** Every executor is fail-closed by design — with no key the provider resolver returns `null` and the job fails with `provider_not_configured` rather than fabricating an output. See §24 for the generation-pipeline backlog. ## Where these surfaces sit in the tier system The generation pipeline is gated by a deterministic tier resolver in `libs/isis/entitlements/src/generation-tier.ts`. The implemented tier union is: ```ts type GenerationTier = | 'operator-admin' | 'aaa-creator' | 'curated-creator' | 'contemplative'; ``` > **Accuracy note (corrects the prose at features.md "four canonical tier > names").** The hub document elsewhere describes the four tiers as `Customer`, > `Curated-Creator`, `AAA-Creator`, and `Operator`, "used verbatim." Two of > those four are _display labels_, not the code identifiers: the implemented > `GenerationTier` values (`generation-tier.ts:23-28`) are `contemplative` (the > `Customer`/contemplative product), `curated-creator`, `aaa-creator`, and > `operator-admin` (the `Operator` tier). When reading code, match on the > hyphenated lowercase identifiers; the title-case names are the human-readable > taxonomy. `GENERATION_TIERS` is a frozen array of the four code values. `resolveGenerationTier(entitlement)` returns a `ResolvedTier` (`{ entitlement, tier, surfaceAllowlist }`); it derives the tier from the entitlement's `tags` via `resolveTierFromTags` using a strict precedence (`operator-admin` > `aaa-creator` > `curated-creator` > `contemplative`). `checkSurfaceAccess({ entitlement, surface })` returns a `SurfaceAccessVerdict` that is `allow` (with tier) or `deny` (with tier + a UI `action` of `404-hard-block`) — deny-by-default. Every creator surface on this page is named in the 28-value `GenerationSurface` enum and gated by the per-tier `TIER_ALLOWLISTS`. See [Generation Audience Tiers and Surface Boundaries](./generation-tiers-and-surfaces.md) for the full surface vocabulary and the host-boundary CTA logic. ## Curated Creator Generation Surfaces The curated tier (`curated-creator`) exposes a small, opinionated set of cards. Each card binds to an _approved workflow class_ — there is no raw model picker, no LoRA hash selector, no scheduler choice, and no graph editor. The card inputs are deliberately narrow (subject, style preset, aspect, locale, mood, tempo), and the picker contents resolve from `(tier, tenant entitlement)`. These surfaces are the customer-path entry points that feed the BFF generation executors under `apps/oshun/bff/src/generation/`. | Card | Bound to | Inputs (curated) | Backing BFF executor | | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- | -------------------------------------------------------- | | Illustration | Approved Sophia source set + Lilith persona + Isis workflow class | subject, style preset, aspect, locale | `image-executor.ts` | | Narration (TTS) | Approved voice profiles, per-locale prosody, Lilith disclosure copy | text, voice profile (allowlisted), locale | `narration-executor.ts` | | Ambient audio | Approved music workflow classes | mood, tempo, duration, loopability | `music-executor.ts` / `music-enqueue-executor.ts` | | Explainer (Veritas) | Grounded claim + source set, mandatory Sophia evidence pin | claim, source pin | `explainer-executor.ts` | | Caption + dub | Localization queues | source track, target locale | `caption-dub-executor.ts` | | Accessibility-pass | Alt-text / contrast / caption / transcript generation | source asset | `accessibility-pass-executor.ts` | | Living Scene | Approved score templates (Tara Contemplative Arcs, Nyx Sky Briefings, Veritas Grounded Explainers, Metis Lesson Visualizers, Arete Living Offerings) | voice + tap + text cue on every template | `sky-briefing-executor.ts` and the Living Scenes runtime | Cross-cutting guarantees every card carries: - **Per-card pre-flight** cost/latency estimate, provenance preview, consent confirmation, and a "regenerate-with-direction" affordance. - **Side-by-side variant compare** and a **send-to-editorial** pipeline — the same `send-to-editorial` bulk action surfaced in the [Output Gallery](./output-gallery-lineage.md). - **Takedown-on-revoke**: a consent revocation cascades to every output that cited it. - **Per-card kill-switch** and **per-tenant entitlement gate** with graceful degradation when the underlying workflow class is frozen. - **No raw graph leaks** via DOM, network, or URL — the surface boundary is tested at component, route, and BFF layers (§24.11). The Living Scene card in particular exposes _no_ raw model picker, LoRA hash selector, scheduler choice, or graph editor; the template picker resolves entirely from tier + tenant entitlement. See [Living Scenes — Concept and Customer Promise](./living-scenes-overview.md). ### Safety classifiers gate every curated output Before any curated output is released, the BFF runs an output-content safety scan: `image-safety-classifier.ts` for visual outputs and `text-safety-classifier.ts` for spoken/text outputs. These are **fail-closed**: when the classifier is absent or the scan fails, the release-gate `safetyScanScore` is `null` and the gate _blocks_ — an output is never released unscanned, and a passing score is never fabricated. The narration path is the canonical example: TTS faithfully speaks exactly its input text, so moderating that text _is_ a faithful output-content safety scan (see the comment in `narration-provider-env.ts`). ### The autonomous / send-to-editorial pipeline (creative orchestrator) The contemplative/curated "send-to-editorial" flow and the autonomous generation pipeline are powered by `@oshun/creative-orchestrator` (`libs/oshun/creative-orchestrator`), wired into the BFF agentic layer (`apps/oshun/bff/src/agentic/creative-generator-tools.ts`, `agentic-governance-gate.ts`) and `libs/yemaya`. It is a real, fail-loud pipeline built on `@oshun/ai/agent-loop` (`runStructuredOutput`, `runReflexion`): - `decomposeBrief(...)` turns a brief into a schema-validated, acyclic `CreativePlan` DAG, validated against `CREATIVE_PLAN_SCHEMA` (cycle detection via `detectCycle` / `validateDagStructure` / `topologicalOrder`). - `routePlan` / `CreativeOrchestrator` dispatch plan nodes to a `GeneratorRegistry` of `DomainGenerator`s under real governance — a `BudgetGovernanceGate` (budget / kill-switch / throttle) with an `ALLOW_ALL_GATE` for tests. - `reviseArtifact` runs a bounded generate → critique → revise (Reflexion) loop around each artifact, scored by an `ArtifactCritic` (`createMetricCritic`, `createLlmJudgeCritic`, or the non-provider-gated `createContentEvalCritic`). - `createYemayaAgentGenerator` adapts Yemaya's specialized agents into the registry; `createMetisNarrator` adds streaming narration. Per its own package description, the orchestrator "fails loud when no provider / generator is wired; never fabricates artifacts" — the same fail-closed posture as every executor below. See [Agent Registry, Job Orchestration, and Multi-Agent Plans](./agentic-registry-jobs-plans.md). ## Voice Providers, Voice Cloning, and Audio Integrity Voice is the highest-trust generation surface because a cloned voice is identity-bearing. V1 wraps it in a provider abstraction plus a heavy consent, rights, and watermark regime. ### The live voice path (ElevenLabs, fail-closed) The narration executor expects an injected `NarrationProviderGenerate` (synthesize → inline base64 audio). The deployable boundary `narration-provider-env.ts` resolves the real **ElevenLabs** provider via `createElevenLabsProvider` from `@psyche/voice-synthesis` and adapts its `synthesize` to that seam. It is fail-closed and requires **both** environment variables: | Variable | Purpose | | --------------------------- | --------------------------------- | | `OSHUN_ELEVENLABS_API_KEY` | ElevenLabs credential | | `OSHUN_ELEVENLABS_VOICE_ID` | The deploy-selected voice profile | With either missing, the resolver returns `null` and a narration job fails closed — a deployment without credentials never fabricates audio. The ElevenLabs HTTP path is, per the source comment, "exercised at deploy time with a live key"; the env→provider wiring, request/result adaptation, and the release-gate measurement emission are unit-tested in-repo. The provider adapter itself lives at `libs/isis/ai-providers/src/providers/tts/elevenlabs-provider.ts` (audio-side client at `libs/isis/audio-generation/src/voice/elevenlabs-client.ts`), and `libs/oshun/persona-registry/src/voice-provider-abstraction.ts` keeps additional providers swappable. Per deps§9, **no `elevenlabs` SDK is pinned** — V1 calls ElevenLabs through a direct HTTP client. ### Voice integrity, cloning, and abuse The voice subsystem (the `voice-cloning-tool` surface, AAA/operator only) covers: - Voice-provider abstraction, voice profile registry, consent rules, rights rules, cloned-voice safeguards, disallowed behaviors, watermarking, quality scoring, provider fallback/failover, and synthetic disclosure. - Admin review for voice profiles: watermark verification, abuse-risk scores, and review outcomes. - Customer-facing **voice provenance** for premium voiced experiences. - The voice-clone authoring workflow: consent capture with identity verification → sample upload + read-aloud script → naturalness/prosody scoring → watermark verification → abuse-risk scorecard → multi-party signoff → voice-profile registration → release toggle → revocation cascade. - A **voice-pack browser** for curated-tier creators (filter by language, persona, lineage; preview; entitlement gate; provenance display) with _no access to the cloning machinery_ — the curated tier never sees the `voice-cloning-tool` surface. - A **runtime watermark verifier** embedded in playback that flags missing or tampered watermarks _before audio reaches user ears_. - An abuse-detection surface (impersonation alerts, denylist matches, revocation-cascade triggers, takedown coordination with consent revocation). Voice evaluations cover validation, provider fallback, abuse detection, watermark, naturalness, and voice-coherence — including impersonation prompts and identity-protection breach tests, with revocation-cascade reach measured against SLA. See [Persona, Avatar, and Voice Packs](./persona-avatar-voice-packs.md) and [Lilith Persona Policy](./lilith-persona-policy.md). ## Music and Audio Generation Music generation serves ritual/meditation soundscapes (Tara), ambient explainers (Veritas), study beds (Metis/Nisaba), and sky-event briefing audio (Nyx). The `@isis/music-generation` library (`libs/isis/music-generation`) provides the "provider abstraction, workflow classes, watermark + provenance" (its package description, §24.8). ### Workflow classes and guardrails `@isis/music-generation` defines a `MusicWorkflowClass` and a frozen `DEFAULT_GUARDRAILS` record with per-class allowed MIME types (`libs/isis/music-generation/src/guardrails.ts`). The implemented classes and their format guardrails: | Workflow class | Allowed MIME types | | ---------------- | ---------------------------------------------------- | | `ambient` | `audio/wav`, `audio/mpeg`, `audio/ogg` | | `ritual` | `audio/wav`, `audio/mpeg`, `audio/ogg`, `audio/flac` | | `meditation-bed` | `audio/wav`, `audio/mpeg`, `audio/ogg` | | `study-bed` | `audio/wav`, `audio/mpeg`, `audio/ogg` | | `score` | `audio/wav`, `audio/flac` | | `stem-render` | `audio/wav`, `audio/flac` | `checkGuardrails(...)` enforces format, length, and file-size constraints per class and returns typed `GuardrailIssue`s; `checkDisallowedContent(...)` returns a `DisallowedContentVerdict` for the content guards below. Each class also carries the curated-card inputs in the doc's surface: mood, tempo, duration, loopability, key, intensity-curve, with Lilith tone-policy enforcement and Sophia source attribution where reference-track-driven. The AAA tier adds a stems & mix surface (per-stem isolation, gain, fade, loop-point editor, mix-bus routing) hosted in Yemaya Studio. ### Two distinct music execution modes V1 has **two** music paths, and only one is wired live on the customer path today: **(1) Batch music (submit → poll → download) — Suno only.** The `music-executor` expects an injected `MusicProviderGenerate` (`(ProviderMusicRequest) => Promise<{ id, url? }>`). The deployable boundary `music-provider-env.ts` resolves the real **Suno** provider (`SunoProvider` from `@isis/audio-generation/generation`) from environment credentials: | Variable | Purpose | | -------------------------- | ------------------------------------------------------------------------------- | | `OSHUN_SUNO_API_KEY` | Suno credential (required; absent → resolver returns `null`) | | `OSHUN_SUNO_MODEL_VERSION` | One of `v3` \| `v3.5` \| `v4` \| `v5` (validated against `SUNO_MODEL_VERSIONS`) | | `OSHUN_SUNO_BASE_URL` | Optional base-URL override | With no `OSHUN_SUNO_API_KEY` the resolver returns `null` and a music job fails closed. Curated music is **instrumental** (no lyrics/vocals — see `translateMusicRequest`), and there is no in-repo audio-content safety scanner, so the emitted release measurement carries `safetyScanScore: null` and the gate _blocks at the safety floor_ until a deploy binds a real audio safety scan + watermark + C2PA signing step. The Suno HTTP submit→poll→download path is exercised at deploy time with a live key. > **Accuracy note (corrects the music provider list at features.md "MusicGen, > Suno, Udio, Stable-Audio, custom-on-Comfy").** At the file level, > `libs/isis/audio-generation/src/generation/` contains `suno-provider.ts`, > `udio-provider.ts`, `self-hosted.ts`, `sfx-provider.ts`, and > `music-generator.ts` (plus `audio-analysis.ts`) — there is no standalone > `MusicGen` or `Stable-Audio` _provider file_. `music-generator.ts` declares a > `MusicProviderType = 'suno' | 'udio' | 'stable-audio' | 'local'` union, and > `self-hosted.ts` is the in-house path that _implements_ Stable Audio 2.0 + > MusicGen as local backends — so those names are real internals, not separate > hosted adapters. Critically, the **BFF batch music executor wires only Suno > today**; `udio-provider.ts` exists as an adapter but is **not** customer-path > wired. deps§9 lists Suno _and_ Udio as V1-used music adapters, but that > implies a parity the BFF wiring does not yet have — Udio is library-present, > not executor-bound. **(2) Realtime / streaming music (frame-by-frame socket) — Magenta RT.** A distinct, long-lived interactive mode lives at `apps/oshun/bff/src/generation/realtime-music-route.ts` over the WebSocket path `/v1/generation/realtime`. Control envelopes flow _in_ as JSON (`generation-control`); decoded PCM flows _out_ as binary frames; unified `stream.*` events flow out as JSON. The source is the on-device `MagentaRtProvider` (Google Magenta RealTime 2) resolved by `realtime-music-provider-env.ts` from `@euterpe/realtime-gen` and `@euterpe/providers` (`magentaRt`). It is fail-closed exactly like Suno: it returns `null` (and the route serves a plain `503 mrt2_not_configured`, never opening a socket) unless `OSHUN_MRT2_ENABLED=true` **and** the on-device runtime actually loads — native engine + weights present + device feasible. On a host without the linked inference backend, `runtime.load()` reports `not_configured`. The codec sample rate is `48_000` Hz. This streaming mode is omitted from the hub document's music section entirely; it is a genuinely separate generation pattern (live stream vs. batch submit→poll). ### Watermark, provenance, and content guards Every produced track carries a watermark and a provenance bundle (consent ID, prompt, model, license, generator, timestamp). The disallowed-content guards refuse: copyrighted-style imitation without rights, melody/voice clones without consent, and contemplative-tone violations (over-stimulation, rhythmic patterns flagged for crisis-state harm). Tests cover workflow-class enforcement, watermark presence, copyrighted-style detection against a fixture set, provenance completeness, stem alignment, and contemplative-tone regression. SFX is handled separately by `sfx-provider.ts`, whose `SFXProviderType = 'elevenlabs' | 'audioldm' | 'audiocraft' | 'freesound' | 'local'` union covers foley and surface-material categories. ## 3D Generation and Post-Processing 3D generation is primarily AAA-tier (the `3d-generation`, `gaussian-splatting`, and `auto-rigging` surfaces in `TIER_ALLOWLISTS['aaa-creator']`); the contemplative product exposes only narrow read-mostly surfaces such as Nyx sky-event 3D briefings. The provider library is `libs/isis/ai-providers/src/providers/three-d/`, whose modules export the 3D provider abstraction: `types`, `topology-verification`, `texture-quality`, `pipeline-class-registry`, `provenance`, and a `router`. - **3D provider abstraction** covers text-to-3D, image-to-3D, gaussian-splatting capture-to-mesh, and auto-rigging, bound to the existing 3D node set. - **Post-processing pipeline classes** for topology verification, texture enhancement, RIFE interpolation (`rife` surface), and Topaz upscaling (`topaz` surface) — chained via approved pipeline classes only, never ad-hoc at runtime. The `PipelineClassRegistry` registers `PipelineClassDefinition`s and computes a `declaredGraphHash(...)` so a class binding is verifiable. - **Operator pipeline-class authoring** (graph + classes + cost class + rehearsal-fixture set); creator surfaces only _select_ an approved pipeline. - **Nyx sky-event 3D briefing card** (curated tier) producing a lightweight scene viewer for celestial events, bound to an approved 3D pipeline class. - **Provenance**: every 3D output carries source 2D refs, pipeline-class hash, model version, and a watermark embedded in mesh metadata. Tests cover pipeline-class binding, topology validity, texture-quality scoring, provenance completeness, and watermark preservation through enhancement passes. Per deps§9, the broader 3D stack also spans `libs/isis/gaussian-splatting` and `libs/isis/3d-generation`; live training/capture execution of these pipelines is provider-gated and not exercised in default CI. ## The provider adapter library and live-call gating All of the above route through the provider adapter library at `libs/isis/ai-providers/src/providers/`, which contains: `animation`, `civitai`, `comfy`, `comfy-cloud`, `controlnet`, `conversational-ai`, `florence`, `image-generation`, `instantid`, `ip-adapter`, `live-preview-streaming`, `llm`, `model-registry`, `multi-gpu-orchestration`, `music-generation`, `three-d`, `tts`, `usage`, `video-generation`, `video-processing`, and `workflow-versioning`. > **Accuracy note (corrects the request-flow provider label).** The hub document > and the ARCHITECTURE flow describe the execution provider as "ComfyUI · > ElevenLabs · Suno." That label is incomplete for the live customer path. > ComfyUI/RunPod are the _operator substrate_, but the live customer-facing > executors call **hosted HTTP providers**: image → **Stability SD3.5** > (`StabilityProvider`), voice → **ElevenLabs**, batch music → **Suno**, and > video → **fal.ai-hosted LTX-Video**. None of those four live executors calls > ComfyUI directly. The live image and video boundaries follow the same fail-closed contract: | Surface | Live provider | Env contract | Notes | | ------- | -------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Image | Stability SD3.5 (`StabilityProvider` from `@isis/ai-providers/providers/image-generation`) | `OSHUN_STABILITY_*` (model via `OSHUN_STABILITY_MODEL`, default `sd3.5-large`) | Valid aspect ratios: `1:1`, `16:9`, `9:16`, `4:3`, `3:4`, `21:9`, `9:21`, `3:2`, `2:3`. Stability does not sign C2PA. | | Video | fal.ai-hosted LTX-Video (`LTXProvider` from `@isis/ai-providers/providers/video-generation`) | fal.ai key (`video-provider-env.ts`); model via `OSHUN_VIDEO_MODEL`, default `ltx-video-2.0` | submit + poll + download. No in-repo video safety scanner/watermarker/C2PA signer → blocks at the gate until deploy-bound. Aspect: `16:9`/`9:16`/`1:1`/`4:3`; resolution `720p`/`1080p`. | These deploy-time env vars are the real toggles that flip a surface from fail-closed to live; absent them the resolver returns `null` and the executor fails with `provider_not_configured`. Every executor routes its output through the canonical release measurement (`buildReleaseMeasurement` / `provider-measurement.ts`) before release. See [External Model Intelligence and Execution Providers](./external-models-and-execution.md) for the ComfyUI/RunPod substrate and Civitai intake. ### The AAA-tier render scheduler (`@oshun/render-farm`) The execution scheduler the docs gesture at but never name is `@oshun/render-farm` (`libs/oshun/render-farm`), imported by `apps/yemaya/studio-web` (confirmed in its `vite.config.ts`, `vitest.config.ts`, `tsconfig.json`, and `package.json`). Its package description: "Shared render-farm scheduling primitives for job submission, worker dispatch, dependency execution, preemption, and checkpoint resume." It exports a `RenderFarmScheduler` (via `createRenderFarmScheduler`) plus a rich type vocabulary: `RenderJob`, `RenderJobSubmission`, `RenderTask`, `RenderAssignment`, `RenderWorkerNode` / `RenderWorkerCapabilities` / `RenderWorkerHeartbeat`, `RenderGpuCapability` / `RenderGpuRequirement`, `RenderCheckpoint` (for checkpoint/resume), `PreemptionDecision`, `RenderCloudBurstPlan` / `RenderCloudBurstDecision` / `RenderCloudBurstProvider` (cloud-burst), `RenderCostEstimate` / `RenderCostRates`, `RenderQuotaBreach` / `RenderQuotaEvaluation`, and `RenderDashboardSnapshot`. This is the priority queue + worker-node capability/GPU matching + preemption + checkpoint + cost scheduler that backs AAA-tier render jobs. ## Surface boundary: the legacy `/studio/isis/*` tree `apps/oshun/web/src/app/studio/` hosts the real customer surfaces `generation` and `generation-gallery`, alongside a legacy `isis` subtree (`3d-generation`, `video-generation`, `audio-generation`, `lora-training`, etc.). The legacy tree is **hard-blocked**: `STUDIO_ISIS_ALLOWED_ROUTE_SEGMENTS` in `libs/isis/entitlements/src/studio-boundary.ts` is an **empty frozen array** (`Object.freeze([])`), so `isAaaOnlyRoute(segment)` is true for every legacy segment. The `proxy.ts` middleware applies `resolveStudioBoundary` to any `/studio/isis/*` request: - For an entitled `aaa-creator` / `operator-admin` session reaching the surface from the contemplative shell → `render-yemaya-cta` (a `307` redirect to `/aaa-upgrade` with `X-Studio-Boundary: aaa-cta` and disclosure copy). - For everyone else (contemplative / curated) → `hard-block-404` (a `404` rewrite to `/_not-found` with `X-Studio-Boundary: hard-block`, through the root document shell for WCAG document-title compliance). So the doc's claim that "AAA-tier routes return disclosure + Yemaya signup gate for entitled users, hard-block for the rest" is borne out by the `render-yemaya-cta` vs `hard-block-404` split — with the refinement that the approved customer surfaces have _moved_ to `/studio/generation/*` and `/studio/generation-gallery`, and the entire legacy `/studio/isis/*` tree is now behind that boundary check rather than served directly. AAA-tier power-user surfaces (full graph editor, full Civitai browser, LoRA training, model merging, music stems, voice cloning, full 3D pipeline editor) live in Yemaya Studio at `apps/yemaya/studio-{web,desktop}`. ## Related - [Isis Generation Control](./isis-generation-control.md) - [Generation Audience Tiers and Surface Boundaries](./generation-tiers-and-surfaces.md) - [External Model Intelligence and Execution Providers](./external-models-and-execution.md) - [Output Gallery, Lineage, Branch, and Replay](./output-gallery-lineage.md) - [Living Scenes — Concept and Customer Promise](./living-scenes-overview.md) - [Persona, Avatar, and Voice Packs](./persona-avatar-voice-packs.md) - [Lilith Persona Policy](./lilith-persona-policy.md) - [Agent Registry, Job Orchestration, and Multi-Agent Plans](./agentic-registry-jobs-plans.md) - [../features.md](../features.md)