Creator Surfaces, Voice, Music, and 3D Generation

This page documents the creator-facing generation surfaces that sit on top of the V1 generation pipeline: the curated creator cards (illustration, narration, ambient audio, explainer, caption/dub, accessibility, Living Scene), the voice and music subsystems, and 3D generation and post-processing. It serves curated creators on the contemplative product and — for the heavier machinery — AAA-tier creators in Yemaya Studio, while the underlying execution providers stay operator-controlled. It is the surface layer above Isis Generation Control, Generation Audience Tiers and Surface Boundaries, and External Model Intelligence and Execution Providers; where every produced asset lands afterward is the Output Gallery, Lineage, Branch, and Replay.

The single rule that shapes every surface on this page: raw generation machinery never appears on the contemplative product, and no live provider call ever happens without a deploy-time credential. Every executor is fail-closed by design — with no key the provider resolver returns null and the job fails with provider_not_configured rather than fabricating an output. See §24 for the generation-pipeline backlog.

Where these surfaces sit in the tier system#

The generation pipeline is gated by a deterministic tier resolver in libs/isis/entitlements/src/generation-tier.ts. The implemented tier union is:

type GenerationTier =
  | 'operator-admin'
  | 'aaa-creator'
  | 'curated-creator'
  | 'contemplative';

Accuracy note (corrects the prose at features.md "four canonical tier names"). The hub document elsewhere describes the four tiers as Customer, Curated-Creator, AAA-Creator, and Operator, "used verbatim." Two of those four are display labels, not the code identifiers: the implemented GenerationTier values (generation-tier.ts:23-28) are contemplative (the Customer/contemplative product), curated-creator, aaa-creator, and operator-admin (the Operator tier). When reading code, match on the hyphenated lowercase identifiers; the title-case names are the human-readable taxonomy. GENERATION_TIERS is a frozen array of the four code values.

resolveGenerationTier(entitlement) returns a ResolvedTier ({ entitlement, tier, surfaceAllowlist }); it derives the tier from the entitlement's tags via resolveTierFromTags using a strict precedence (operator-admin > aaa-creator > curated-creator > contemplative). checkSurfaceAccess({ entitlement, surface }) returns a SurfaceAccessVerdict that is allow (with tier) or deny (with tier + a UI action of 404-hard-block) — deny-by-default. Every creator surface on this page is named in the 28-value GenerationSurface enum and gated by the per-tier TIER_ALLOWLISTS. See Generation Audience Tiers and Surface Boundaries for the full surface vocabulary and the host-boundary CTA logic.

Curated Creator Generation Surfaces#

The curated tier (curated-creator) exposes a small, opinionated set of cards. Each card binds to an approved workflow class — there is no raw model picker, no LoRA hash selector, no scheduler choice, and no graph editor. The card inputs are deliberately narrow (subject, style preset, aspect, locale, mood, tempo), and the picker contents resolve from (tier, tenant entitlement). These surfaces are the customer-path entry points that feed the BFF generation executors under apps/oshun/bff/src/generation/.

Cross-cutting guarantees every card carries:

• Per-card pre-flight cost/latency estimate, provenance preview, consent confirmation, and a "regenerate-with-direction" affordance.
• Side-by-side variant compare and a send-to-editorial pipeline — the same send-to-editorial bulk action surfaced in the Output Gallery.
• Takedown-on-revoke: a consent revocation cascades to every output that cited it.
• Per-card kill-switch and per-tenant entitlement gate with graceful degradation when the underlying workflow class is frozen.
• No raw graph leaks via DOM, network, or URL — the surface boundary is tested at component, route, and BFF layers (§24.11). The Living Scene card in particular exposes no raw model picker, LoRA hash selector, scheduler choice, or graph editor; the template picker resolves entirely from tier + tenant entitlement. See Living Scenes — Concept and Customer Promise.

Safety classifiers gate every curated output#

Before any curated output is released, the BFF runs an output-content safety scan: image-safety-classifier.ts for visual outputs and text-safety-classifier.ts for spoken/text outputs. These are fail-closed: when the classifier is absent or the scan fails, the release-gate safetyScanScore is null and the gate blocks — an output is never released unscanned, and a passing score is never fabricated. The narration path is the canonical example: TTS faithfully speaks exactly its input text, so moderating that text is a faithful output-content safety scan (see the comment in narration-provider-env.ts).

The autonomous / send-to-editorial pipeline (creative orchestrator)#

The contemplative/curated "send-to-editorial" flow and the autonomous generation pipeline are powered by @oshun/creative-orchestrator (libs/oshun/creative-orchestrator), wired into the BFF agentic layer (apps/oshun/bff/src/agentic/creative-generator-tools.ts, agentic-governance-gate.ts) and libs/yemaya. It is a real, fail-loud pipeline built on @oshun/ai/agent-loop (runStructuredOutput, runReflexion):

• decomposeBrief(...) turns a brief into a schema-validated, acyclic CreativePlan DAG, validated against CREATIVE_PLAN_SCHEMA (cycle detection via detectCycle / validateDagStructure / topologicalOrder).
• routePlan / CreativeOrchestrator dispatch plan nodes to a GeneratorRegistry of DomainGenerators under real governance — a BudgetGovernanceGate (budget / kill-switch / throttle) with an ALLOW_ALL_GATE for tests.
• reviseArtifact runs a bounded generate → critique → revise (Reflexion) loop around each artifact, scored by an ArtifactCritic (createMetricCritic, createLlmJudgeCritic, or the non-provider-gated createContentEvalCritic).
• createYemayaAgentGenerator adapts Yemaya's specialized agents into the registry; createMetisNarrator adds streaming narration.

Per its own package description, the orchestrator "fails loud when no provider / generator is wired; never fabricates artifacts" — the same fail-closed posture as every executor below. See Agent Registry, Job Orchestration, and Multi-Agent Plans.

Voice Providers, Voice Cloning, and Audio Integrity#

Voice is the highest-trust generation surface because a cloned voice is identity-bearing. V1 wraps it in a provider abstraction plus a heavy consent, rights, and watermark regime.

The live voice path (ElevenLabs, fail-closed)#

The narration executor expects an injected NarrationProviderGenerate (synthesize → inline base64 audio). The deployable boundary narration-provider-env.ts resolves the real ElevenLabs provider via createElevenLabsProvider from @psyche/voice-synthesis and adapts its synthesize to that seam. It is fail-closed and requires both environment variables:

With either missing, the resolver returns null and a narration job fails closed — a deployment without credentials never fabricates audio. The ElevenLabs HTTP path is, per the source comment, "exercised at deploy time with a live key"; the env→provider wiring, request/result adaptation, and the release-gate measurement emission are unit-tested in-repo. The provider adapter itself lives at libs/isis/ai-providers/src/providers/tts/elevenlabs-provider.ts (audio-side client at libs/isis/audio-generation/src/voice/elevenlabs-client.ts), and libs/oshun/persona-registry/src/voice-provider-abstraction.ts keeps additional providers swappable. Per deps§9, no elevenlabs SDK is pinned — V1 calls ElevenLabs through a direct HTTP client.

Voice integrity, cloning, and abuse#

The voice subsystem (the voice-cloning-tool surface, AAA/operator only) covers:

• Voice-provider abstraction, voice profile registry, consent rules, rights rules, cloned-voice safeguards, disallowed behaviors, watermarking, quality scoring, provider fallback/failover, and synthetic disclosure.
• Admin review for voice profiles: watermark verification, abuse-risk scores, and review outcomes.
• Customer-facing voice provenance for premium voiced experiences.
• The voice-clone authoring workflow: consent capture with identity verification → sample upload + read-aloud script → naturalness/prosody scoring → watermark verification → abuse-risk scorecard → multi-party signoff → voice-profile registration → release toggle → revocation cascade.
• A voice-pack browser for curated-tier creators (filter by language, persona, lineage; preview; entitlement gate; provenance display) with no access to the cloning machinery — the curated tier never sees the voice-cloning-tool surface.
• A runtime watermark verifier embedded in playback that flags missing or tampered watermarks before audio reaches user ears.
• An abuse-detection surface (impersonation alerts, denylist matches, revocation-cascade triggers, takedown coordination with consent revocation).

Voice evaluations cover validation, provider fallback, abuse detection, watermark, naturalness, and voice-coherence — including impersonation prompts and identity-protection breach tests, with revocation-cascade reach measured against SLA. See Persona, Avatar, and Voice Packs and Lilith Persona Policy.

Music and Audio Generation#

Music generation serves ritual/meditation soundscapes (Tara), ambient explainers (Veritas), study beds (Metis/Nisaba), and sky-event briefing audio (Nyx). The @isis/music-generation library (libs/isis/music-generation) provides the "provider abstraction, workflow classes, watermark + provenance" (its package description, §24.8).

Workflow classes and guardrails#

@isis/music-generation defines a MusicWorkflowClass and a frozen DEFAULT_GUARDRAILS record with per-class allowed MIME types (libs/isis/music-generation/src/guardrails.ts). The implemented classes and their format guardrails:

checkGuardrails(...) enforces format, length, and file-size constraints per class and returns typed GuardrailIssues; checkDisallowedContent(...) returns a DisallowedContentVerdict for the content guards below. Each class also carries the curated-card inputs in the doc's surface: mood, tempo, duration, loopability, key, intensity-curve, with Lilith tone-policy enforcement and Sophia source attribution where reference-track-driven. The AAA tier adds a stems & mix surface (per-stem isolation, gain, fade, loop-point editor, mix-bus routing) hosted in Yemaya Studio.

Two distinct music execution modes#

V1 has two music paths, and only one is wired live on the customer path today:

(1) Batch music (submit → poll → download) — Suno only. The music-executor expects an injected MusicProviderGenerate ((ProviderMusicRequest) => Promise<{ id, url? }>). The deployable boundary music-provider-env.ts resolves the real Suno provider (SunoProvider from @isis/audio-generation/generation) from environment credentials:

With no OSHUN_SUNO_API_KEY the resolver returns null and a music job fails closed. Curated music is instrumental (no lyrics/vocals — see translateMusicRequest), and there is no in-repo audio-content safety scanner, so the emitted release measurement carries safetyScanScore: null and the gate blocks at the safety floor until a deploy binds a real audio safety scan + watermark + C2PA signing step. The Suno HTTP submit→poll→download path is exercised at deploy time with a live key.

Accuracy note (corrects the music provider list at features.md "MusicGen, Suno, Udio, Stable-Audio, custom-on-Comfy"). At the file level, libs/isis/audio-generation/src/generation/ contains suno-provider.ts, udio-provider.ts, self-hosted.ts, sfx-provider.ts, and music-generator.ts (plus audio-analysis.ts) — there is no standalone MusicGen or Stable-Audio provider file. music-generator.ts declares a MusicProviderType = 'suno' | 'udio' | 'stable-audio' | 'local' union, and self-hosted.ts is the in-house path that implements Stable Audio 2.0 + MusicGen as local backends — so those names are real internals, not separate hosted adapters. Critically, the BFF batch music executor wires only Suno today; udio-provider.ts exists as an adapter but is not customer-path wired. deps§9 lists Suno and Udio as V1-used music adapters, but that implies a parity the BFF wiring does not yet have — Udio is library-present, not executor-bound.

(2) Realtime / streaming music (frame-by-frame socket) — Magenta RT. A distinct, long-lived interactive mode lives at apps/oshun/bff/src/generation/realtime-music-route.ts over the WebSocket path /v1/generation/realtime. Control envelopes flow in as JSON (generation-control); decoded PCM flows out as binary frames; unified stream.* events flow out as JSON. The source is the on-device MagentaRtProvider (Google Magenta RealTime 2) resolved by realtime-music-provider-env.ts from @euterpe/realtime-gen and @euterpe/providers (magentaRt). It is fail-closed exactly like Suno: it returns null (and the route serves a plain 503 mrt2_not_configured, never opening a socket) unless OSHUN_MRT2_ENABLED=true and the on-device runtime actually loads — native engine + weights present + device feasible. On a host without the linked inference backend, runtime.load() reports not_configured. The codec sample rate is 48_000 Hz. This streaming mode is omitted from the hub document's music section entirely; it is a genuinely separate generation pattern (live stream vs. batch submit→poll).

Watermark, provenance, and content guards#

Every produced track carries a watermark and a provenance bundle (consent ID, prompt, model, license, generator, timestamp). The disallowed-content guards refuse: copyrighted-style imitation without rights, melody/voice clones without consent, and contemplative-tone violations (over-stimulation, rhythmic patterns flagged for crisis-state harm). Tests cover workflow-class enforcement, watermark presence, copyrighted-style detection against a fixture set, provenance completeness, stem alignment, and contemplative-tone regression. SFX is handled separately by sfx-provider.ts, whose SFXProviderType = 'elevenlabs' | 'audioldm' | 'audiocraft' | 'freesound' | 'local' union covers foley and surface-material categories.

3D Generation and Post-Processing#

3D generation is primarily AAA-tier (the 3d-generation, gaussian-splatting, and auto-rigging surfaces in TIER_ALLOWLISTS['aaa-creator']); the contemplative product exposes only narrow read-mostly surfaces such as Nyx sky-event 3D briefings. The provider library is libs/isis/ai-providers/src/providers/three-d/, whose modules export the 3D provider abstraction: types, topology-verification, texture-quality, pipeline-class-registry, provenance, and a router.

• 3D provider abstraction covers text-to-3D, image-to-3D, gaussian-splatting capture-to-mesh, and auto-rigging, bound to the existing 3D node set.
• Post-processing pipeline classes for topology verification, texture enhancement, RIFE interpolation (rife surface), and Topaz upscaling (topaz surface) — chained via approved pipeline classes only, never ad-hoc at runtime. The PipelineClassRegistry registers PipelineClassDefinitions and computes a declaredGraphHash(...) so a class binding is verifiable.
• Operator pipeline-class authoring (graph + classes + cost class + rehearsal-fixture set); creator surfaces only select an approved pipeline.
• Nyx sky-event 3D briefing card (curated tier) producing a lightweight scene viewer for celestial events, bound to an approved 3D pipeline class.
• Provenance: every 3D output carries source 2D refs, pipeline-class hash, model version, and a watermark embedded in mesh metadata.

Tests cover pipeline-class binding, topology validity, texture-quality scoring, provenance completeness, and watermark preservation through enhancement passes. Per deps§9, the broader 3D stack also spans libs/isis/gaussian-splatting and libs/isis/3d-generation; live training/capture execution of these pipelines is provider-gated and not exercised in default CI.

The provider adapter library and live-call gating#

All of the above route through the provider adapter library at libs/isis/ai-providers/src/providers/, which contains: animation, civitai, comfy, comfy-cloud, controlnet, conversational-ai, florence, image-generation, instantid, ip-adapter, live-preview-streaming, llm, model-registry, multi-gpu-orchestration, music-generation, three-d, tts, usage, video-generation, video-processing, and workflow-versioning.

Accuracy note (corrects the request-flow provider label). The hub document and the ARCHITECTURE flow describe the execution provider as "ComfyUI · ElevenLabs · Suno." That label is incomplete for the live customer path. ComfyUI/RunPod are the operator substrate, but the live customer-facing executors call hosted HTTP providers: image → Stability SD3.5 (StabilityProvider), voice → ElevenLabs, batch music → Suno, and video → fal.ai-hosted LTX-Video. None of those four live executors calls ComfyUI directly.

The live image and video boundaries follow the same fail-closed contract:

These deploy-time env vars are the real toggles that flip a surface from fail-closed to live; absent them the resolver returns null and the executor fails with provider_not_configured. Every executor routes its output through the canonical release measurement (buildReleaseMeasurement / provider-measurement.ts) before release. See External Model Intelligence and Execution Providers for the ComfyUI/RunPod substrate and Civitai intake.

The AAA-tier render scheduler (@oshun/render-farm)#

The execution scheduler the docs gesture at but never name is @oshun/render-farm (libs/oshun/render-farm), imported by apps/yemaya/studio-web (confirmed in its vite.config.ts, vitest.config.ts, tsconfig.json, and package.json). Its package description: "Shared render-farm scheduling primitives for job submission, worker dispatch, dependency execution, preemption, and checkpoint resume." It exports a RenderFarmScheduler (via createRenderFarmScheduler) plus a rich type vocabulary: RenderJob, RenderJobSubmission, RenderTask, RenderAssignment, RenderWorkerNode / RenderWorkerCapabilities / RenderWorkerHeartbeat, RenderGpuCapability / RenderGpuRequirement, RenderCheckpoint (for checkpoint/resume), PreemptionDecision, RenderCloudBurstPlan / RenderCloudBurstDecision / RenderCloudBurstProvider (cloud-burst), RenderCostEstimate / RenderCostRates, RenderQuotaBreach / RenderQuotaEvaluation, and RenderDashboardSnapshot. This is the priority queue + worker-node capability/GPU matching + preemption + checkpoint + cost scheduler that backs AAA-tier render jobs.

Surface boundary: the legacy /studio/isis/* tree#

apps/oshun/web/src/app/studio/ hosts the real customer surfaces generation and generation-gallery, alongside a legacy isis subtree (3d-generation, video-generation, audio-generation, lora-training, etc.). The legacy tree is hard-blocked: STUDIO_ISIS_ALLOWED_ROUTE_SEGMENTS in libs/isis/entitlements/src/studio-boundary.ts is an empty frozen array (Object.freeze([])), so isAaaOnlyRoute(segment) is true for every legacy segment. The proxy.ts middleware applies resolveStudioBoundary to any /studio/isis/* request:

• For an entitled aaa-creator / operator-admin session reaching the surface from the contemplative shell → render-yemaya-cta (a 307 redirect to /aaa-upgrade with X-Studio-Boundary: aaa-cta and disclosure copy).
• For everyone else (contemplative / curated) → hard-block-404 (a 404 rewrite to /_not-found with X-Studio-Boundary: hard-block, through the root document shell for WCAG document-title compliance).

So the doc's claim that "AAA-tier routes return disclosure + Yemaya signup gate for entitled users, hard-block for the rest" is borne out by the render-yemaya-cta vs hard-block-404 split — with the refinement that the approved customer surfaces have moved to /studio/generation/* and /studio/generation-gallery, and the entire legacy /studio/isis/* tree is now behind that boundary check rather than served directly. AAA-tier power-user surfaces (full graph editor, full Civitai browser, LoRA training, model merging, music stems, voice cloning, full 3D pipeline editor) live in Yemaya Studio at apps/yemaya/studio-{web,desktop}.

Related#

• Isis Generation Control
• Generation Audience Tiers and Surface Boundaries
• External Model Intelligence and Execution Providers
• Output Gallery, Lineage, Branch, and Replay
• Living Scenes — Concept and Customer Promise
• Persona, Avatar, and Voice Packs
• Lilith Persona Policy
• Agent Registry, Job Orchestration, and Multi-Agent Plans
• ../features.md