Saraswati Stage: Music Generation & Live Performance

Saraswati Stage is V3's virtual-artist tenant: a roster of six persistent AI musicians who release a catalog, hold scheduled concerts to a stadium audience, and run drop-day singles, listening parties, and quarterly festivals inside the Lilith metaverse. The architecture splits cleanly in two, and the split is the whole story. Music generation is engine-agnostic — it runs as a chain of TypeScript gates and an Isis-orchestrated worker pool with no Unreal dependency, producing reviewed, provenance-stamped tracks ahead of release. Live performance is the opposite: a UE5 Sequencer asset is the canonical concert, driving MetaSounds music streams, Niagara stage FX, a MetaHuman performer, and between-song speech against an NTP-style music clock. The seam between them is a released track plus a persona — everything the catalog pipeline produces becomes either a row in the in-world discography or a cue track in a concert Sequencer.

This page is grounded in real code in four places: the catalog gates in libs/v3/isis-music/, the tenant logic (persona, voice, concert authoring) in libs/v3/saraswati-stage/, the concert export/quality gates in libs/v3/concert-quality/, and the UE side under V3/ue/Source/V3Cinematics, V3/ue/Source/V3Audio, and the concert-master template under V3/ue/Content/Saraswati/Templates/. It is the architecture-side companion for the "Avatars, Audio, and Tenant Experiences" set; the section hub is ../V3_ARCHITECTURE.md.

The complete concert-design reference is @calliope/stage: its typed concept, setlist, stage, lighting, screen, choreography, venue, camera, VFX, spatial-audio, audience, streaming, and VOD-package services are specification sources for this tenant. V3 does not yet import Calliope at runtime. V3 remains the owner of tenant records, rights/provenance, recorded editorial evidence, export quality, and the Unreal contract; a composition root should select Calliope design outputs at cook time and then run V3's gates. Neither Calliope's delivery plan nor a V3 Sequencer asset id proves that rendered media bytes exist.

What ships, honestly#

The generation, rights, persona, and concert-authoring gates are real, domain-specific TypeScript, each with a validation suite that asserts computed behaviour rather than shape. isis-music ships multi-provider routing with a cost/quality comparator and a failover drill, a Sophia-grounded lyric factuality check that catches an unverifiable award claim, an auto-mix/master pass with real LUFS/true-peak targets and a release-blocking editorial review queue, and an Audible-Magic/Pex fingerprint check that routes a major-label match to review. saraswati-stage ships a voice-signature builder that blocks public-figure clones, a persona policy lock with an adversarial prompt suite, a ten-gate concert authoring pipeline that refuses to publish until every gate passes, and a real Ed25519 C2PA track signer. concert-quality adds a judge-panel scene quality + diversity gate spliced into the same release-gate suite. The UE side ships the V3.Saraswati.ConcertMasterTemplate automation test over a canonical concert Sequencer contract.

Four honest qualifications. First, the generation pipeline does not call live music or TTS providers. routeSaraswatiMusicProvider scores a static in-repo registry of six providers; buildSaraswatiGroundedLyricDraft returns a deterministic lyric fixture and the tested logic is the factuality grounding gate around it; the auto-mix/master pass is a settings-and-review model, not a DSP renderer; the fingerprint matcher is token-overlap scoring against a synthetic 50-track reference catalog. These are honest gate harnesses with the real providers (Suno/Udio, Cartesia/ElevenLabs, Audible Magic/Pex) as provider-gated seams. Second, V3Cinematics and V3Audio (the modules) are thin registration contracts — each exposes only GetModuleName(), GetOwnedSurfaceTag(), SupportsRuntimeLoad() (V3/ue/Source/V3Cinematics/Public/V3Cinematics.h). The concert runtime the monolith describes (a Sequencer playing MetaSounds subgraphs and Niagara cues) is engine content, not C++ in this repo. The V3Audio module is a stub, but the same folder also ships the real Steam Audio VR profile in V3SteamAudioVr.* (covered in ./avatar-animation-and-audio.md). Third, the "concert Sequencer asset" (V3/ue/Content/Saraswati/Templates/ConcertMaster.uasset) is JSON text, not a binary cooked LevelSequence — a descriptor paired with a ConcertMaster.v3sequence.json contract. The automation test validates that contract's shape and bindings with LoadFileToString + Contains; it does not load a Sequencer in-editor. Fourth, the off-platform distribution (Spotify / Apple Music), PRO registrations, and the Adobe CAI trust root are provider-gated. The sections below say where each claim is backed.

Music generation: an engine-agnostic gate chain#

The catalog pipeline runs entirely off-engine: an editorial brief enters a job queue, Isis orchestrates a music provider, and the draft track passes through a fixed chain of gates before it can become a release. Each gate is a pure function with a validation suite; nothing fabricates a "success" it didn't compute.

Provider routing and failover#

provider-routing.ts registers six music providers — suno, udio, stable-audio, musicgen, audiocraft, mubert — each carrying costPerMinuteCents, a qualityScore, latencyP95Ms, maxDurationSeconds, and supportsStems/supportsLyrics flags. routeSaraswatiMusicProvider filters out providers that are unavailable, already failed, can't meet the requested duration, or lack a required capability, then sorts the survivors with compareProviders: in cost mode by cents ascending (quality breaks ties), in quality mode by score descending (cost breaks ties). The first survivor is selected; the rest become a failoverChain. drillSaraswatiMusicProviderFailover proves the chain is real — it routes once, marks the winner failed, re-routes, and asserts the second pick differs from the first. The drill report passes only when all six providers register and both cost and quality drills fail over successfully. Suno (quality 0.92) wins quality routing; Mubert (80¢/min) wins cost routing — the comparator is exercised against actual numbers, not a placeholder.

Grounded lyrics and tone#

grounded-lyric-generation.ts is where the honest seam is sharpest. buildSaraswatiGroundedLyricDraft selects a model family (claude-opus for a release candidate, claude-sonnet for a sketch) and emits a deterministic four-line draft — but the substance is evaluateSaraswatiLyricFactuality, which splits the lyric into lines, decides which lines look like factual claims (a cue-phrase scan for is, won, born, studied with, …), and requires each one to be supported by a phrase in a Sophia evidence pack. A line with no supporting citation becomes a blocking issue and the report status flips to blocked. The validation set proves the discrimination: a Sophia-grounded lyric about the fictional persona "Anaya Veil" passes with zero unverifiable claims, while "Anaya Veil won the 2025 Grammy for Best New Artist…" is correctly counted as one unverifiable claim and blocked — caughtKnownUnverifiableClaim is the gate's headline assertion. Tone is a separate pass (lyric-tone-policy.ts) scored against the Lilith persona policy, routing a violation to an editorial queue.

Auto-mix, master, and the editorial review gate#

mix-master-review-gate.ts masters to streaming-loudness targets: SARASWATI_MUSIC_MASTER_TARGET_INTEGRATED_LUFS = -14, SARASWATI_MUSIC_MASTER_TRUE_PEAK_DBTP = -1, 48 kHz / 24-bit, with per-stem settings (createStemSettings assigns real gain/pan/compressor-ratio/high-pass values across up to five stem kinds — lead-vocal, harmony-vocal, percussion, melodic-bed, ambience). The architectural point is the review queue: every track enters status: 'queued' with releaseBlocked: true, and only an explicit approve-release action flips it to approved and unblocks it; request-changes and reject-release leave it blocked. Every action writes a SaraswatiMusicReviewAuditEvent. evaluateSaraswatiAutoMixMasterReviewGate runs three tracks through approve / request-changes / reject and asserts that exactly one is release-allowed, the other two stay blocked, all three were surfaced before release, and one audit event was logged per action — a gate that would fail if release ever leaked past review.

Fingerprint pre-release check#

fingerprint-pre-release-check.ts is the rights firewall. SARASWATI_FINGERPRINT_REFERENCE_CATALOG is 50 reference tracks tagged major-label / independent / public-domain / oshun-cleared, each with an Audible Magic and a Pex fingerprint. scoreFingerprintMatch computes a real similarity — token-set coverage weighted 0.9 plus a duration-proximity term weighted 0.1 — and SARASWATI_FINGERPRINT_MATCH_THRESHOLD = 0.82 gates it. A match against a major-label reference sets status: 'editorial-review-required', releaseBlocked: true, and mints a queue item requiring lilith-rights-review, license-documentation, and editorial approval before release. The validation report asserts the discrimination both ways: an original track clears, an independent-catalog match does not trigger major-label review, and both an Audible-Magic and a Pex major-label match route to editorial.

Provenance, C2PA, and the catalog#

A released track carries an Isis provenance bundle (track-provenance-bundle.ts) and a C2PA manifest (track-c2pa-manifests.ts) whose six required assertions are c2pa.actions.v2, org.oshun.v3.track-release, org.oshun.v3.generated-artifact-provenance, org.oshun.v3.rights-waterfall, org.oshun.v3.pro-registration, and org.oshun.v3.adobe-cai-verification. The signature is a real Ed25519 signature over the canonical claim payload — the SaraswatiTrackC2paSigner seam takes an injected KMS key in production and a deterministic dev key otherwise, so verification (and tamper detection) is genuine, "not an FNV-1a string mislabeled sha256" (the source says so in as many words). On release, discography-release-flow.ts propagates the track to the in-world catalog (SARASWATI_IN_WORLD_CATALOG_MAX_PROPAGATION_SECONDS = 60), which the UE5 client and the Tier-2 fallback both read, and off-platform-distribution.ts fans out to Spotify / Apple Music with PRO-society registration coverage.

The artist behind the catalog: persona, voice, and policy#

A Saraswati artist is not a model checkpoint; it is a persona dossier, a consent- bound voice signature, and a runtime policy lock — all in saraswati-stage.

Voice signatures and the public-figure block#

voice-signature-build.ts builds a persona's TTS voice signature only from consent-ledger source material. reviewSaraswatiVoiceSignatureBuild requires a Saraswati persona id, a voice-sig:consent-ledger: ref, at least 60 clean consented source minutes across grants, and rejects any grant whose scope isn't saraswati-persona-only or that has been revoked. The safety-critical check is findPublicFigureMatch: a probe score at or above SARASWATI_VOICE_SIMILARITY_MATCH_THRESHOLD_BASIS_POINTS = 8600 (86%) against a known public figure is a blocking issue, and the validation-corpus false-positive rate must stay at or under SARASWATI_VOICE_SIMILARITY_MAX_FALSE_POSITIVE_BASIS_POINTS = 100 (1%). signOffSaraswatiGAVoiceSignatures builds all six GA personas, runs an adversarial probe of a 91.2%-similar public-figure voice, and asserts it is blocked before any signature builds — fanVoiceUploadsAccepted: false is baked into every consent dossier, and a Lilith-rights sign-off is required per persona.

The persona policy lock#

persona-policy-lock.ts governs what an artist persona will say at runtime (lyrics, between-song speech, fan interaction). Each of the six GA personas locks three advice classes — medical, legal, financial — and enforceSaraswatiPersonaPolicyLock classifies a prompt into one of three actions: stay-in-persona, graceful-noop, or operator-handoff. A locked- advice-class hit, a public-figure-imitation request, or targeted political persuasion routes to operator-handoff (queue lilith-safety-saraswati-persona, 10-minute SLA); a role-swap jailbreak or a system-prompt-disclosure attempt is a graceful-noop that stays in character without leaking. policyBreached is the fail-loud signal — it is set only if a prompt that did match a reason somehow returned stay-in-persona. The adversarial suite runs six attack prompts across all six personas (36 cases) and passes only when policyBreachCount === 0 and no advice request escaped the operator handoff.

Live performance: Sequencer + MetaSounds#

A concert is a UE5 Sequencer asset, and the canonical shape of one is the concert-master template. V3/ue/Content/Saraswati/Templates/ConcertMaster.v3sequence.json is the contract: schema oshun.v3.saraswati.concert-master-template.v1, owned by the V3Mode_SaraswatiConcert Game Feature and the V3Cinematics module, binding six cue-track families — each mapped to a real UE MovieScene track class:

The music-sync track carries an ntp-style-concert-clock with maxDriftMs: 25 keyed off a songStartNtpMs server timestamp — the same ≤ 25 ms music-sync budget the per-mode tick table sets for a Saraswati performer (20 Hz transform, 60 Hz expression, 30 Hz Sequencer cues), and the NTP clock-recovery math itself lives in spatial-audio (see ./avatar-animation-and-audio.md). The audience-cam track encodes privacy as data — excludePrivacySeats: true, excludeCrowdBandIndividualHandles: true — so a camera sweep cannot frame a non-consenting attendee. The between-song-speech track sets personaPolicyLock: true, names Sophia as the grounding provider and Cartesia/ElevenLabs as TTS, and points at a pre-rendered MetaSound speech graph. V3SaraswatiConcertMasterTemplateTests.cpp (V3.Saraswati.ConcertMasterTemplate) asserts the .uasset resolves to /Game/Saraswati/Templates/ConcertMaster as a LevelSequence, that the contract reports templateLoadsCleanlyInEditor and previewPlayRendersAllCueTrackTypes, that all six cue tracks exist with their exact bindings, and that the 420-second preview fires all six cue types.

The concert authoring pipeline#

concert-authoring-pipeline.ts models the editorial workflow that fills that template, as ten release gates (SaraswatiConcertAuthoringGateId): persona-setlist proposal, per-track generation review, motion-preset bindings, lighting/Niagara cues, Sequencer assembly, speech-bank review, audience choreography, dress rehearsal, provenance publish, and GA cadence. Each gate has a real predicate: the setlist must hold exactly 12 songs within a 2-working-day proposal window; the debut catalog must be reviewed within 6 working weeks; every song must bind a motion preset and a lighting+Niagara cue pair within their 12-day windows; the speech bank must hold 8–12 slots of 3–5 reviewed alternates each; at least two staging dress rehearsals must be signed off; the publish receipt's Isis provenance bundle must be visible in the inspector; and the GA cadence must hold (6 artists × 2 concerts/month = 12/month over ≥ 60 days, all editorial-signed-off). publishSaraswatiConcertSequencer throws unless every gate passes — the publish receipt's sequencerAssetId resolves to /Game/Saraswati/Concerts/<concert>/ConcertMaster, mirroring the template path.

Between-song speech#

Between-song speech is pre-generated and queued, never live-generated — live generation can't meet the beat-sync budget and would expose a policy violation to a live audience without a review gate. Each setlist slot gets 3–5 editorially- reviewed alternates (each run through Sophia factuality, the persona policy lock, and the persona voice signature, stored as a MetaSound stream), plus a generic neutral cold-fallback. At showtime the concert director (or an AI-director rule) picks the alternate that best fits the moment — e.g., the take that names a fan actually in attendance. Live unscripted Q&A is not supported at GA; it moves to a post-show green-room (V3Mode_SaraswatiClub) where pre-generation plus a retry budget is viable.

Scene quality and the concert export suite#

concert-quality adds the quality signal the authoring gates don't measure. concert-scene-quality-gate.ts scores each scene/speech draft with the §1 judge panel (assessArtifactQuality) against a default bar of 70, and measures the whole concert's draft set for corpus diversity (cluster coverage ≥ 0.6, mean pairwise distance ≥ 0.25) and slop density (≤ 25 per 1000 words) — so a concert can't be "eight phrasings of one line" even if each phrasing passes on its own. These are GateDefinitions for the same @oshun/content-release-gates suite as the C2PA/consent gates; v3-concert-export.ts composes the authoring release state, the provenance/consent gates, the scene quality+diversity gates, and an optional stage-director sign-off into one ReleaseGateService run, so a low-quality or homogeneous scene blocks publishing exactly like a revoked consent or a bad signature. The C2PA signature comes from the shared @oshun/content-signing Ed25519 signer adapted in concert-track-signer.ts — one crypto core signing both isis 3D-asset manifests and V3 concert tracks. The gates fail loud on an empty draft set rather than wave an empty concert through.

Edge cases and failure modes#

• Generation gates fail closed. A lyric with an unsupported factual claim, a track that never got an approve-release, or a major-label fingerprint match each blocks the track and routes it to a human queue — none silently passes.
• A public-figure voice never builds. buildSaraswatiVoiceSignature throws if the review status isn't ready; a ≥ 86%-similar public-figure probe is a blocking issue before any model artifact is produced.
• The persona lock can't leak. An advice/imitation/persuasion request is an operator-handoff; a jailbreak is a graceful-noop; policyBreached is set only when a matched-reason prompt wrongly stays in persona — the test bar is zero breaches across all 36 cases.
• A concert can't publish half-authored. publishSaraswatiConcertSequencer throws unless all ten gates pass; the GA-cadence gate additionally requires every scheduled concert to be editorial-signed-off.
• Audience cameras are privacy-bound by data. The audience-cam cue track ships excludePrivacySeats / excludeCrowdBandIndividualHandles, so the constraint travels with the asset, not a runtime afterthought.
• The asset/runtime seam is a JSON descriptor. ConcertMaster.uasset is JSON text and the automation test validates the contract; the binary LevelSequence, MetaSound graphs, Niagara systems, the MetaHuman performer, and motion/lighting presets are authored UE content, not code in this repo. Reading V3Cinematics/V3Audio expecting a concert engine will mislead — those modules are registration stubs.

How it connects#

Saraswati Stage is a consumer of the embodiment stack and a producer for the tenant experiences. Its MetaHuman performers, costume packs, and the ≤ 25 ms music-sync clock are rendered and spatialized by the avatar/animation/audio stack and the Tier-1 UE5 client; the concert audience and performer tick budgets, the SFU voice path, and the music-sync clock recovery are the same ones that page sizes. The persona policy lock and grounded-lyric factuality reuse the Lilith persona-policy and Sophia surfaces shared with Tara classes, Aja, and the Commons, the other tenant in this set. Provenance bundles, C2PA signing, and the @oshun/* package map (Isis control plane, content-release-gates, content-signing, the tenant packages) are catalogued in ../../platform/oshun-domain-libraries.html.

Related#

• Avatar, Animation & Spatial Audio — the MetaHuman performers, MetaSounds/Steam Audio, and the music-sync clock this pipeline drives
• Tara Classes, Aja, and the Commons — the sibling tenant sharing the persona-policy and Sophia surfaces
• Tier-1 UE5 Client — the native runtime that loads V3Mode_SaraswatiConcert and plays the concert Sequencer
• Oshun Domain Libraries — where the Isis, content-release-gate, signing, and tenant packages sit
• The section hub: ../V3_ARCHITECTURE.md