Living Scenes

Living Scenes is the V1 cross-cutting product surface that turns a stated intention into a 4–8 minute, watermarked, re-renderable artifact the customer keeps in a personal gallery. It serves contemplative customers (a Tara breath arc), curious ones (a Nyx sky briefing, a Veritas grounded explainer, a Metis lesson visualizer), and creators (an Arete living offering). It sits across the platform substrates rather than inside any single one: Yemaya owns the rendering substrate and the runtime, Psyche carries the live-direction envelope, Isis governs generation, Lilith enforces tone, and Sophia grounds the fact-bearing templates. This page is hubbed at ../ARCHITECTURE.md alongside the substrate deep-dives (Yemaya/Psyche runtime, Isis, Lilith, Sophia). Product scope: V1/features.md § Living Scenes. Backlog: §25.

Read this page for what is deterministic decision logic vs. pixel-level generation. The Living-Scenes "brain" is real, typed, and unit-tested: the Conductor state machine (@yemaya/living-scenes-runtime, v0.1.0), the Blend Kernel transition contracts and continuity gate (@yemaya/blend-kernel, v0.1.0), the 12-technique catalog, the per-template allowlists with crisis-collapse, the compatibility scorer, the compose-assist budgets and gold-set eval, the template fixture eval-gates, and the Psyche scene-event stream. What is aspirational / downstream is everything frame-level: the actual latent video generation, optical-flow warps, FVD computation, and GPU determinism. Those live outside these modules — the pure functions here consume continuity scores and conditioning hashes as inputs; they never compute pixels. The Conductor file says it plainly: "Actual GPU dispatch lives outside this module." The transitions file says the same: "Implementations of the actual DSP / shader code live downstream."

The promise, and what makes it keepable#

A customer states an intention and receives a 4–8 minute Living Scene tuned to it. The artifact is watermarked, kept in their gallery as a personal artifact, and re-renderable forever from the underlying Score. That last clause is the load-bearing one. The artifact is not the canonical object — the Score is. The Score is versioned, deterministic, and re-renderable, so the same intention can be re-cut at a higher resolution, against a newer catalog version, or in a reduced-motion variant, without the customer re-authoring anything.

Two product nouns recur:

• Domain template — the canonical Living-Scene shape for a domain. Each template fixes pacing, the allowed cue verbs, the blend-kernel inputs, the permitted sources, and the tone constraints. The five V1 templates have canonical machine IDs (the docs historically used only friendly names): tara-contemplative-arc, nyx-sky-briefing, veritas-grounded-explainer, metis-lesson-visualizer, arete-living-offering (libs/isis/workflow-classes/src/living-scene/template-catalog.ts).
• Contemplative Arc (the Tara template) and Living Offering (the Arete template) are the two consumer-facing names. The Contemplative Arc paces to a breath-cycle clock and honors Lilith's strictest tone band; the Living Offering requires intent capture before scoring and is kept private, opt-in-to-share.

Conductor flow — intention to artifact#

The Scene Score schema#

The Score is the durable, JSON-serializable contract. It is defined twice on purpose: the cross-process Zod contract in libs/contracts/src/living-scene/score.ts and the runtime mirror in @yemaya/living-scenes-runtime/score/score-schema, so that the BFF, queues, and persistence validate payloads against the same schema the runtime trusts. deepParseScore(value) is the canonical entry point: it parses the schema and enforces three cross-field invariants — segment-id uniqueness, cue-target resolution (appliesToSegmentId must be * or an existing segment), and the release-gate lookahead floor (renderEnvelope.minLookaheadSegments ≥ 2).

A Score is { scoreId, template, tenantId, authorUserId, intent?, segments[], cuePlan[], renderEnvelope, createdAtUnixSeconds }.

Segment — the real fields#

Accuracy fix. Earlier docs (features § Scene Score Schema) described a Segment as a 13-field object — {intent, public_redacted_intent, duration_band, style_anchor, motion_descriptor, audio_role, narration_script_pin, sophia_grounding_pin, lilith_tone_band, persona_binding, transition_in, transition_out, accessibility_role}. None of those named fields exist verbatim. The real SegmentSpec (score.ts:47-60) has eight fields. Crucially: intent lives on the Score, not the Segment (it is the ScoreIntentLayer); transitions are not stored on segments (they are boundary decisions resolved by compose-assist / the blend kernel); and grounding, persona, and accessibility are not Segment fields.

SegmentKind (score.ts:15-25): opening, breath-anchor, visual-meditation, narration-bed, sky-event-arc, lesson-visualizer, grounded-explainer, living-offering, closing.

Intent layer — private vs. public, never co-mingled#

Intent is its own object on the Score (ScoreIntentLayer, score.ts:121-129): { intentId, privateText (≤20k), publicRedactedText (≤20k), privateIntentHash (/^[0-9a-f]{64}$/), capturedAtUnixSeconds, redactionCategoriesHit[] }. The private text and its SHA-256 hash are what the shareability rules protect: the private intent layer never leaves the originating user, even when the rendered artifact is shared. redactionCategoriesHit draws from Lilith's redaction taxonomy (named-person, deceased, medical-condition, abuse-disclosure, location-of-safety, minor-identity, financial-account, legal-status), so the public-redacted text can be audited for what was stripped.

CarryState — the continuity protocol, with field names#

The docs describe continuity hand-off qualitatively ("CLIP-style style anchor, last-frame conditioning, motion-vector descriptor, color-LUT alignment, audio tail"). The real protocol is a concrete schema with constraints (CarryStateSchema, score.ts:38-44):

This is the artifact the Conductor propagates across a boundary, and the one planReconnect hands a returning client so it can resume mid-scene.

RenderEnvelope — far more than "engine version + model hashes"#

The envelope pins the deterministic render target. Every field is a literal union or bounded integer, so an envelope cannot ask for an off-spec resolution or frame rate (RenderEnvelopeSchema, score.ts:91-107):

minLookaheadSegments is the bridge between the static envelope and the runtime: it is both a literal union and re-asserted at parse time (deepParseScore throws "renderEnvelope must require ≥ 2 lookahead segments" at score.ts:165-167). The Conductor reads exactly this number to size its pre-warm window.

Cue plan — live direction as data#

Live direction is not free-form text; it is a typed cue plan. A CueSpec (score.ts:78-86) is { cueId, verb, args, appliesToSegmentId, channels[] }, where channels is a serialized array (voice | tap | text) because a Set is not JSON-serializable. The nine cue verbs (ScoreCueVerb, score.ts:62-72, mirrored as CUE_VERBS_VOCABULARY in the runtime's cue-parser.ts) are:

linger · advance · morph · recall · shift-style · shape-by · save-moment · abandon · kill

Which verbs are legal depends on the template — see the per-template allowed-cue sets below.

The Conductor — a real segment-slot state machine#

@yemaya/living-scenes-runtime (conductor.ts) is the scheduling brain. It is pure logic: it models render scheduling as deterministic state transitions and never dispatches a GPU. Each segment is a SegmentSlot whose state walks a six-state machine:

SegmentRenderState = pending → pre-warming → ready → streaming → done
                                                              ↘ aborted

The conductor state is { scoreId, envelope, slots[], playheadIdx, backpressureActive }. The lifecycle functions are:

Lookahead-bounded pre-warm#

The pre-warm window is [playheadIdx + 1, playheadIdx + minLookaheadSegments]. Pending slots inside that window become pre-warming; slots beyond it stay pending (conductor.ts:79-102). This is what lets the next 2–4 segments warm their latents before the playhead reaches them, hiding generation latency behind the currently-streaming segment.

Backpressure truncation to lookahead = 1#

When downstream is slower than realtime, setBackpressure flips backpressureActive, and planPreWarm collapses the lookahead to 1 — pre-warming only the immediate next slot (conductor.ts:86). This is the deliberate degradation path: under load the system stops speculatively warming four segments ahead and warms exactly one, trading smoothness headroom for not falling further behind.

Carry-state handoff#

advancePlayhead is where continuity propagates. It writes the outbound CarryState onto the slot being left, then promotes the next slot — but only if that next slot is already ready or streaming (otherwise it throws invalid-transition: cannot advance: next slot ... is in state ...). The promoted slot inherits the leaving segment's outbound carry-state as its inbound carry-state (carryStateInbound: input.carryStateOutbound ?? next.carryStateInbound, conductor.ts:159). That single line is the seam through which lastFrameConditioningHash, motionDescriptor, lutId, and the rest flow from one segment's tail into the next segment's warm start.

Reconnect resumability#

planReconnect answers "the client dropped at segment X — can it resume, and with what state?". A segment is resumable only if its slot is streaming or ready; otherwise it returns { resumable: false, reason: "segment in non-resumable state ..." }. When resumable, it hands back the slot's carryStateInbound as resumeCarryState, so the rejoining client is re-seeded with exactly the continuity state the segment started from (conductor.ts:180-209).

The Blend Kernel — typed transition contracts and a continuity gate#

@yemaya/blend-kernel (transitions.ts) owns the mechanical primitive layer. It is, again, contracts plus validation — "Implementations of the actual DSP / shader code live downstream." There are nine TRANSITION_KINDS (transitions.ts:13-23), not the five the docs list. Each composes outbound carry-state with inbound carry-state at a boundary:

Staleness fix. The features doc listed five transition specs (latent-warm-start, optical-flow-morph, color-LUT-match, audio-crossfade, narrative-pivot). The real array adds four more: motion-descriptor-handoff, motion-descriptor-reset, audio-level-jump, variable-rate-sequencer.

Out-of-range parameters raise TransitionValidationError with a code (out-of-range | unknown-kind | missing-required) — the kernel refuses to construct an off-spec transition rather than silently clamping.

Continuity scorecard gate#

The kernel also owns the gate the §25.18 release machinery calls. A ContinuityScorecard is { fvd, flickerScore, colorJump, motionVectorContinuity, audioRmsDeltaDb }, and isBlendContinuityAcceptable({ scorecard, thresholds }) returns { acceptable, reasons[] }. It fails closed: any non-finite value is a reason, and each metric is checked against its threshold (FVD/flicker/colorJump must be under their max, motionVectorContinuity must be over its min, and |audioRmsDeltaDb| must be under its max). Crucially, these are inputs the pure function consumes — FVD and flicker are computed by the downstream renderer, not here. The gate's job is to decide, deterministically, given those numbers.

The Cinematographic Technique Catalog (12 named techniques)#

On top of the nine mechanical transitions sits a versioned catalog of twelve named, creator-intent techniques (cinematographic-catalog.ts:16-29):

hard-cut · match-cut · l-cut · j-cut · dissolve · smash-cut · whip-pan · jump-cut · match-action · cross-cut · montage · fade-to-black

fade-to-black is the universal fallback — it is the recovery transition for crisis↔non-crisis boundaries and the reduced-motion fallback for the most aggressive techniques. Each catalog entry (CinematographicTechnique) pins its kernel composition (which of the nine transitions it chains), an overlapWindowRange, requires/forbids tags, the toneBands it can carry, its templateAllowlist, a reducedMotionFallback, a provenanceTag matching technique:[a-z0-9-]+, and a per-technique evalThresholds block. Those thresholds are concrete numbers, not qualitative descriptions:

The catalog is version-pinned per kept artifact (PinnedTechnique, catalogVersion) so deprecating a technique never breaks playback of an artifact that used it. canReshareDeprecatedTechnique returns true only when the pinned version equals the current version — reshare of a deprecated technique requires re-rendering against the current catalog.

Composition flow and technique selection#

The compatibility scorer (7 dimensions)#

scoreSegmentPair (scorer.ts) is a pure function that grades an adjacent Segment pair across seven dimensions and returns one of three verdicts: hard-incompatible, soft-incompatible (with a PairScore), or compatible. The seven dimensions (COMPATIBILITY_DIMENSIONS, scorer.ts:21-29) are style-anchor, motion-descriptor, audio-role, narration, tone-band, grounding, and persona; the composite is the arithmetic mean of all seven (compositeOf).

The per-dimension scoring is deliberate and asymmetric:

Accuracy fix. The features doc described style-anchor distance as "CLIP-embedding distance between the outgoing Segment's terminal style anchor and the incoming Segment's opening anchor." That is not what this pure module computes. styleScore (scorer.ts:75-77) is exact-ID equality (1 vs 0.35), and motionScore is likewise exact equality (1 vs 0.5). A real CLIP-embedding metric would be a downstream input, not a property of this deterministic scorer; the doc overstated an embedding-based metric that is not implemented here.

The hard-incompatible policy gate runs first#

Before any mechanical scoring, policyGate (scorer.ts:118-165) can short-circuit to hard-incompatible regardless of how smooth the pair would be:

• Crisis ↔ non-crisis without a recovery transition. If exactly one of the two segments is non-safe and the transition is not fade-to-black, the pair is blocked ("crisis ↔ non-crisis pair requires recovery transition (fade-to-black)").
• Pedagogical ↔ entertainment intra-Score — a pedagogical tone band next to an entertainment one is forbidden.
• Tara ↔ forbidden technique — on tara-contemplative-arc, any of smash-cut, whip-pan, jump-cut is hard-blocked.
• Veritas mutually-retracted pins — on veritas-grounded-explainer, if both segments cite only Sophia source pins that are in retractedSophiaPinHashes, the pair is blocked. This is the Sophia-retraction interlock: a grounded explainer cannot stand on sources that have been pulled.

The gate's verdict is final; composite thresholds (compositeThresholdSoft, compositeThresholdCompatible) only decide the soft boundary between compatible and soft-incompatible for pairs that survive the gate.

Tone gating, crisis collapse, and per-template allowlists#

tone-gating.ts (@yemaya/blend-kernel) enforces the allowlists at three points: the customer Technique Picker (don't surface disallowed entries), compose-assist candidate generation (don't suggest them), and Score promotion (block on inclusion). assertTechniqueAllowed runs the checks in priority order:

1. Crisis-frame collapse takes precedence. Within one frame of crisis-frame activation, the allowlist collapses, for every template, to the safe trio CRISIS_COLLAPSE_ALLOWLIST = {hard-cut, dissolve, fade-to-black} (tone-gating.ts:28-32). Anything else throws crisis-frame-collapse.
2. Themis interlock for Metis. If a metis-lesson-visualizer segment node is in an in-progress assessment (metisAssessmentLockedNodes contains it), every technique is blocked (themis-assessment-locked).
3. Per-template allowlist from the catalog's templateAllowlist.
4. Arete operator-curated transitions. The three "energetic" techniques — OPERATOR_CURATED_ARETE_TECHNIQUES = {smash-cut, whip-pan, jump-cut} — are only available on arete-living-offering, are never allowed under Lilith's strictest tone band (strictest-lilith-tone-block), and otherwise require an explicit operator-curated entitlement grant (operator-curated-technique-required).

Reconciliation — two allowlists kept in sync. There are two per-template technique allowlists in the codebase that must agree: the per-technique templateAllowlist[] on each catalog entry (cinematographic-catalog.ts) and TEMPLATE_TECHNIQUE_ALLOWLISTS in compose-assist.ts:86-125. They currently agree for all five templates, but they are a duplicated source of truth that can drift — the docs describe only one conceptual allowlist. Treat the catalog as canonical and verify the compose-assist copy against it whenever either changes.

Templates also constrain cue verbs. Each LivingSceneTemplate (template-catalog.ts) carries an allowedCueVerbs set, and Veritas sets morphForbidden: true so templateAllowsVerb({ verb: 'morph' }) is always false for it:

Tara Contemplative Arc — paced to breath#

The Tara template has five sub-variants (TARA_SUBVARIANTS), each with a concrete BreathCycle (tara.ts) so visuals and captions pace to an inhale/ hold/exhale/hold clock:

Compose Assist — an AgentRun with budgets and a gold-set#

Compose Assist runs as an Isis AgentRun: a bounded agent invocation with a budget (maxTokens, maxCostCents, maxLatencyMs) and grants (toolIds, allowEgress). enforceBudget fails the run if observed tokens/cost/latency exceed the cap (compose-assist.ts). Candidate techniques are filtered through the same gating policy as the picker, and each suggestion carries a confidence band [low, high] ∈ [0,1] and a continuityPreScore.

Tier caps#

validateCompositionDraft enforces hard ceilings per tier (TIER_CAPS, compose-assist.ts:447-452):

A crisis-active draft is rejected outright with reason crisis-frame-composition-lock, regardless of tier.

Promotion continuity gate#

Promoting a composition draft to a Score is gated on continuity. Both the draft-level check and the per-boundary check require continuityPreScore ≥ 0.85 — below that, promotion throws continuity-pre-score-out-of-range (compose-assist.ts:585, 688). Each boundary must also connect adjacent segment cards and pass the workflow-class policy gate. Promotion returns a PromotedCompositionScore with both a techniquesByBoundary map and a parallel reducedMotionTechniquesByBoundary map, so the reduced-motion variant is computed in lockstep at promotion time, not as an afterthought.

Gold-set evaluation harness#

evaluateGoldSet returns { precision, recall, refusalRate, correctRefusals, falseRefusals } over a labeled gold set (compose-assist.ts:292-345). Precision is tp/(tp+fp), recall is tp/(tp+fn), and the harness distinguishes correct refusals (the assist declined where it should) from false refusals (it declined where it shouldn't) — the latter being the metric that catches an over-cautious assistant. A champion/challenger comparison gates deployment of a new assist model behind minPrecision, minRecall, and maxRefusalRate.

Template fixture eval-gates#

Before a template can be promoted to production, its fixture set must clear TEMPLATE_EVAL_GATES (template-catalog.ts:147-199). All five templates require at least 30 fixtures and a crisisTrigger pass rate of 1.0 (a single missed crisis fixture blocks promotion). The other thresholds vary by template:

evaluateFixtureSet returns { pass, failures[] }, listing every threshold a report missed; a template whose gate requires groundingCorrectness but whose report omits it fails with "groundingCorrectness required but not reported." Tara and Arete have no grounding gate because they are not fact-bearing templates; Veritas — the grounded explainer — carries the strictest bar.

Live Direction Channel — the Psyche scene-event stream#

The runtime telemetry surface is a typed stream of eight scene events on the Psyche envelope (libs/oshun/embodiment-psyche/src/events/scene-events.ts). These are entirely absent from the prose docs but are the real Live Direction Channel:

scene.segment-start · scene.segment-end · scene.transition-start · scene.transition-end · scene.live-direction-applied · scene.policy-intervention · scene.crisis-frame · scene.fallback-engaged

Each event extends SceneEventEnvelope (eventId, sessionId, sequence, emittedAtUnixMillis, traceId, spanId, parentSpanId), and the payloads are fully typed — e.g. scene.segment-end carries an endReason ∈ {planned, cue-advance, cue-abandon, policy-cut}; scene.transition-end carries the realized continuity scorecard; scene.policy-intervention records an interventionKind (pse-strobe-cap, luminance-cap, motion-cap, tone-band-shift, persona-cap) with pre/post values; scene.crisis-frame carries a crisisLabel and probability; and scene.fallback-engaged names both the fallbackKind (reduced-motion, still-image-only, transcript-only, audio-with-still, pre-rendered-substitute) and its trigger (provider-degraded, policy-rejection, eval-gate-fail, shareability-restriction).

verifyEventStream(events) enforces three invariants over a session's feed (scene-events.ts:180-211): a single trace id end-to-end (a mid-stream trace change throws), monotonic sequence (no out-of-order), and gap-free sequence (sequence must increment by exactly 1). This is what lets the contemplative viewer and the Score Editor trust the feed as a complete, ordered record of what happened during a render.

Tone, crisis policy, and provenance#

Every render binds against persona-policy-lilith; a crisis frame downgrades audacity, collapses the technique allowlist to the safe trio, and blocks sharing. Provenance is multi-layered: C2PA Content Credentials across channels, visible and audio watermarks, a ProvenanceBundle per artifact, and an out-of-band public attestation page at oshun.app/scene/<id> that re-asserts provenance even when embedded marks have been stripped.

Shareability matrix#

Every Living Scene is kept-private by default. Sharing requires explicit opt-in and a ToS check; revocation and takedown cascade across every copy — kept, shared, and reshared. The private intent layer never leaves the originating user even when the artifact is shared.

Surfaces — where Living Scenes appears#

Accuracy fix — the AAA editor claim. The architecture prose said the Scene Score Editor "does not appear on the contemplative product under any entitlement." That is only partly accurate. The AAA editor does live in Yemaya Studio (apps/yemaya/studio-{web,desktop}), as stated — but a second, Tara-focused scene editor also lives on the Oshun web app at apps/oshun/web/src/app/lilith-studio/scenes/TaraSceneEditor.tsx (and /lilith-studio/scene/new). This "Lilith Studio" surface on the contemplative product complicates the absolute "does not appear on the contemplative product" claim; treat the contemplative app as carrying a constrained Tara editor, with the full AAA score editor reserved for Yemaya Studio.

The BFF route's keep/share/idempotency logic is exercised by e2e suites — scene-viewer-deepening.spec.ts, living-scene-card.spec.ts, and public-scene-abuse-report.spec.ts (per WALKTHROUGH/results/v1-triage-2026-06-23.md).

Real vs. aspirational — the honest line#

Real, typed, and unit-tested (deterministic decision logic): the Score / Segment / CarryState / RenderEnvelope contracts and deepParseScore invariants; the Conductor's six-state machine with lookahead pre-warm, backpressure truncation, carry-state handoff, and reconnect planning; the nine typed transitions with parameter-range validation and the continuity-scorecard gate; the 12-technique catalog with per-technique eval thresholds and provenance tags; the 7-dimension compatibility scorer with the hard-incompatible policy gate; the three-point tone gating with crisis collapse and the Themis interlock; the compose-assist budgets, tier caps, 0.85 promotion gate, and gold-set precision/recall harness; the template fixture eval-gates; and the eight Psyche scene events with stream verification.

Aspirational / downstream (everything frame-level): the actual latent video generation, optical-flow warps, FVD/flicker computation, and GPU determinism. These are not computed in these modules — they are modeled as score inputs the pure functions consume (a ContinuityScorecard to gate, a lastFrameConditioningHash to carry, a gpuClass to target). The Conductor itself says "Actual GPU dispatch lives outside this module" and the transitions file says "Implementations of the actual DSP / shader code live downstream." That separation is the point: the brain is deterministic and testable; the pixels are produced by a governed Isis generation backend (Isis substrate) that is provider-gated and swapped in at deploy time.

Related#

• Agentic AI Studio
• Psyche — Real-Time Runtime Substrate
• Isis — Generation Control Substrate
• Lilith — Contemplative Policy Substrate
• Sophia — Grounding Substrate
• Generation Audience Tiers
• Trust, Safety, and Privacy
• Persona, Avatar, and Voice Packs
• Subsystem Glossary
• V1/features.md § Living Scenes
• Hub: V1 Architecture