# Latency, Accessibility, Eval Sets, and Tests

This page documents the _quality spine_ of Living Scenes: the latency and
backpressure budgets that keep a live scene responsive, the backend resolver
that chooses between the chained-clip pipeline and the feature-flagged Phase 176
world-model, the reduced-motion and accessibility companions every Score ships
with, and — most concretely — the per-template eval sets, release gates, and the
deterministic test contracts that decide whether a workflow class is allowed to
promote to the contemplative tier at all. It serves the AAA-Creator and operator
authoring consoles (who watch these gates before they ship), trust-and-safety
review (whose crisis and abuse signals feed the eval corpus), and the
contemplative product (whose users feel the latency and accessibility budgets
directly). In the V1 stack it sits _downstream_ of the Yemaya generation
substrate and the governance spine documented in
[Scene Safety, Determinism, Provenance, and Cue Privacy](./scene-governance.md):
where governance decides _whether_ a frame is safe, this page decides _whether a
whole workflow class renders well enough, fast enough, accessibly enough, and
deterministically enough to ship_. Much of it is real, tested decision logic in
`libs/yemaya/living-scenes-runtime/src/` (the published package
`@yemaya/living-scenes-runtime`) and `libs/oshun/trust-safety/src/`; the latency
numbers, the continuity-metric models (FVD, optical-flow morph), and the real
pixel-level renders are upstream of these gates, and this page says so plainly
wherever it matters.

## Latency, Quality, and Backpressure Budgets

A Living Scene is a continuous AI-generated video that _listens_ — the user
shapes it live by voice, tap, or text — so the latency budget is the difference
between a contemplative experience and a stuttering one. The V1 targets are:

| Budget                                  | P50                            | P95      | Notes                                               |
| --------------------------------------- | ------------------------------ | -------- | --------------------------------------------------- |
| First-frame latency                     | ≤ 400 ms                       | ≤ 800 ms | measured from session start                         |
| Segment generation (per 4–12 s segment) | ≤ 6 s                          | ≤ 12 s   | with parallel pre-warm so the playhead never stalls |
| Tap cue                                 | ≤ 1 frame at the next boundary | —        | the cheapest cue path                               |
| Voice cue                               | —                              | ≤ 800 ms | ASR + Lilith pre-screen + cue mapping end-to-end    |
| Text cue                                | —                              | ≤ 300 ms | no ASR leg                                          |

- **Blend-window quality**: each blend window carries a per-frame
  temporal-flicker score and a Fréchet Video Distance (FVD) across the window.
  When either crosses the workflow class's threshold, the runtime re-renders the
  boundary or falls back to a _longer_ blend window rather than shipping a
  visible seam.
- **Backpressure** is a graceful-degradation ladder, not a failure: when GPU
  latency degrades, the runtime first extends the current segment, then reduces
  the next segment's resolution within tolerance, then lengthens the blend
  window, and only then falls back to still-image-plus-narration — it engages
  these _before_ failing the session. The terminal rungs of this ladder are the
  same fallbacks the Psyche envelope surfaces as `scene.fallback-engaged` events
  (see below).
- **Quality dashboards** track P50/P95/P99 per region, per provider, per
  template, and per persona, fire SLO-breach alerts, and map a breach to the
  corpus of active sessions it impacts so an operator can see _who is affected_,
  not just _that a number moved_.

> Honest status: the latency _targets_ and the backpressure _ladder_ are
> product-spec contracts. The numbers (P50 ≤ 400 ms, FVD thresholds,
> optical-flow morph quality) are met by upstream rendering infrastructure that
> is not part of the decision-logic libraries this page is grounded in; the
> governance and eval code below is the genuinely-implemented, tested part of
> the story.

## Backend Strategy and the Phase 176 Bridge

The backend that actually produces frames is chosen by `resolveBackend` in
`libs/yemaya/living-scenes-runtime/src/backend/backend-resolver.ts`. There are
exactly two backends (`BACKEND_IDS`):

- **`chained-clip`** — the V1 default and the _only_ backend on the
  contemplative tier at launch. It is the continuity-disciplined pipeline:
  parallel pre-warm of N-second segments, latent warm-start, optical-flow morph,
  LUT match, and audio crossfade across overlap windows. `planChainedClip`
  constructs its `ChainedClipPlan` (`preWarmSegments`, `blendKernelVersion`,
  `mode: 'materialize-then-stream' | 'stream-live'`) and requires
  `minLookaheadSegments ≥ 2` so there is always a segment warming behind the
  playhead.
- **`phase-176-nous`** — a feature-flagged alternate that swaps the chained
  backend for a live latent world-model from the Nous domain's Phase 176 serving
  program. The customer surface is unchanged; only the substrate differs.

The selector is keyed on a four-value entitlement tier (`ENTITLEMENT_TIERS`):

| Tier              | Can resolve to         | Rule                                                                                |
| ----------------- | ---------------------- | ----------------------------------------------------------------------------------- |
| `contemplative`   | `chained-clip` only    | hard guard: **never** `phase-176-nous`, even if the workflow class declares support |
| `curated-creator` | highest-rank serveable | as below                                                                            |
| `aaa-creator`     | highest-rank serveable | as below                                                                            |
| `operator-admin`  | highest-rank serveable | as below                                                                            |

For non-contemplative tiers, the resolver sorts the workflow class's
`supportedBackends` by `BACKEND_QUALITY_RANK` (`chained-clip: 0`,
`phase-176-nous: 1`) and picks the highest-rank backend that _both_ the
`phase176FeatureFlag` enables (Phase 176 is skipped when the flag is off) _and_
the per-backend SLO `canServeNow` confirms it can serve under headroom. The
contemplative branch is a separate, explicit code path: it throws
`contemplative-cannot-resolve-phase-176` / `unknown-backend` rather than ever
letting a contemplative session touch the world-model backend. When nothing can
serve, it throws `no-backend-can-serve` instead of silently degrading. This is
exactly the _Backend swap_ test contract below: AAA-tier sessions can flip
between the two backends without breaking the customer surface, while
contemplative-tier sessions can never resolve to Phase 176.

## Reduced-Motion and Accessibility Renders

Accessibility is not a post-hoc variant; every Score produces **two** renditions
in lockstep. The runtime enumerates them as `LIVING_SCENE_RENDITIONS` in
`libs/yemaya/living-scenes-runtime/src/personal-artifacts/personal-artifacts.ts`:

- **`full-motion`** — the continuous video.
- **`reduced-motion`** — a still-image carousel that advances at the _same_
  per-segment cadence, with the same narration and ambient audio. It is modeled
  as `ReducedMotionSceneSegment` records (`segmentId`, `durationSeconds`,
  `stillFrameUrl`, `narrationAudioCue`, `ambientAudioCue`). Reduced-motion is an
  independent shareable artifact _and_ a vestibular-friendly accessibility
  companion — it is not a degraded second-class copy.

On top of the two renditions:

- **Captions**: a sidecar caption track per locale, with optional burn-in for
  channels that strip sidecars (default off, user-toggle).
- **Audio description (AD)**: a per-segment AD track narrating the visual
  content for screen-reader users; it ships with every template.
- **Transcript**: a full transcript of narration + AD, linked from the public
  viewer and downloadable.
- **Color-contrast and font-size**: the public viewer respects OS-level
  accessibility settings and offers high-contrast and large-text chrome
  variants.
- **Keyboard and assistive-tech navigation**: cue verbs are reachable by
  keyboard, and the scrubber, share, and accessibility controls are usable
  without a pointer.
- **Accessibility release gate**: promoting _any_ workflow class to the
  contemplative tier requires the reduced-motion variant + AD + captions +
  transcript to pass parity QA. This is one of the gates enumerated below.

The fallback ladder and accessibility renditions are observable on the Psyche
envelope: `SceneFallbackEngaged` in
`libs/oshun/embodiment-psyche/src/events/scene-events.ts` carries a
`fallbackKind` of
`reduced-motion-render | still-image-only | transcript-only | audio-with-still | pre-rendered-substitute`
and a `trigger` of
`provider-degraded | policy-rejection | eval-gate-fail | shareability-restriction`.
A reduced-motion fall-down under provider degradation and a reduced-motion
fall-down under an `eval-gate-fail` are _the same render path_ surfaced with
different triggers, which is why the accessibility artifact has to be a
first-class, always-present rendition rather than something synthesized on
demand.

## Eval Sets, Release Gates, and Rollback

A workflow class earns the contemplative tier by passing a battery of evals
against a per-template fixture corpus. The corpus is **≥ 30 fixtures per
template at V1**, spanning six fixture families: golden-path, adversarial cue,
crisis-trigger, locale-parity, accessibility-parity, and grounding-correctness
(the last for Nyx / Veritas / Metis templates — see
[Veritas — Grounded Stories and Claims](./domain-veritas.md),
[Nyx — Sky Events and Perspective](./domain-nyx.md), and
[Metis — Education and Tutoring](./domain-metis.md)).

### The eval families and the code each gate runs

| Eval family                           | What it measures                                                                                               | Grounded in                                                                                                                    |
| ------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| **Continuity** (workflow-class scope) | boundary FVD, temporal-flicker score, color-jump magnitude, motion-vector continuity, narration-time alignment | scorecard shape: `SceneTransitionEnd.continuityScorecard` (`fvd`, `flicker`, `colorJump`, `motionVector`) in `scene-events.ts` |
| **Determinism**                       | same Score + Envelope on N approved nodes ⇒ pixel-equivalence within tolerance                                 | `checkDeterminism` (below)                                                                                                     |
| **Watermark robustness**              | per-channel survival of {visible-mark short-code, audio watermark, C2PA sidecar, attestation-page resolution}  | `evaluateAllChannels` (below)                                                                                                  |
| **Cue policy**                        | cue-rejection precision/recall against the fixture cue corpus across launch locales                            | `evaluateCorpus` redaction eval + `parseCue` fail-closed (below)                                                               |
| **Crisis-detection**                  | scene-stream crisis detector at parity with the conversational detector across the Lilith taxonomy             | `checkLocaleParity` (below)                                                                                                    |
| **Accessibility**                     | reduced-motion parity, AD coverage, caption accuracy, screen-reader navigability                               | accessibility release gate (above)                                                                                             |

**Release gate**: every workflow class must pass _all six_ — continuity,
determinism, watermark, cue policy, crisis, and accessibility — before promotion
to the contemplative tier. Failures _block_ promotion and are surfaced in the
Isis release-gate dashboard (see
[Isis Generation Control](./isis-generation-control.md)). This is the backstop
§25 contract; the backlog framing is `§25` in [../TODOS.md](../TODOS.md).

**Rollback**: any production workflow class can be rolled back to a prior
version. In-flight sessions complete on the current version; new sessions
resolve to the rolled-back version; and kept artifacts pinned to a version
remain playable under their pinned render envelope. This is the _Determinism_
test's third leg — a rollback must leave pinned artifacts rendering identically.

### Determinism harness — the per-frame SHA-256 gate

`checkDeterminism` in
`libs/yemaya/living-scenes-runtime/src/determinism/determinism-harness.ts` is
the actual decision function behind the determinism gate. It is a _pure_
function over per-`(node, frame)` SHA-256 digests — the rendering happens
upstream; the harness only judges the hashes. Each `NodeFrameDigest` is
`{ nodeId, frameNumber, sha256 }`; the harness groups digests by frame, and any
frame where the set of node hashes has more than one distinct value is a
mismatch. It returns a `DeterminismReport`:

```ts
interface DeterminismReport {
  nodeIds: readonly string[];
  frameCount: number;
  mismatchedFrames: readonly number[];
  perNodeMismatchCount: ReadonlyMap<string, number>;
  pass: boolean;
}
```

Two edge cases are load-bearing. First, a run with **fewer than 2 nodes
auto-passes** (`pass: true`) — there is nothing to disagree with, so a
single-node render can never _fail_ determinism, only be inconclusive. Second,
the gate is `pass` iff _every_ node's mismatch count stays within
`mismatchToleranceFrames`; the per-node count is measured against the first
sorted node as the reference hash. A single node that drifts beyond tolerance
fails the whole gate. This is intentionally a tolerance gate, not bit-exact
equality, because GPU nondeterminism produces a small bounded number of
off-by-an-LSB frames that should not block a promotion that is otherwise
deterministic.

### Watermark-robustness — at least one provenance signal per channel

`evaluateAllChannels` in
`libs/yemaya/living-scenes-runtime/src/provenance/channel-robustness/channel-eval.ts`
runs the watermark-survival gate over the seven share channels in
`SHARE_CHANNELS`: `telegram`, `whatsapp`, `twitter-x`, `instagram`, `discord`,
`email`, `generic-web`. For each channel it counts fixtures where at least one
of the three redundant signals survived the channel's transcode / strip behavior
— `visibleMark`, `audioWatermark`, or `c2paSidecar`. A channel `passes` only
when **every** fixture surfaced ≥ 1 signal (`atLeastOneSignalSurvivedCount`
equals the fixture count), and `allChannelsPass` additionally requires that no
channel is _missing_ fixtures (`missingChannels` is empty). This is the
eval-side companion to the four redundant provenance channels documented in
[Scene Safety, Determinism, Provenance, and Cue Privacy](./scene-governance.md):
the point of three signals plus a resolvable attestation page is that _no single
channel can strip provenance_, and this gate proves it empirically per channel
rather than asserting it.

The visible-mark side of that survival contract is real: `buildAttestationUrl`
in `provenance/visible/visible-mark.ts` constructs the attestation page as
`${origin}/scene/${encodeURIComponent(shortCode)}` from a 6-character
Crockford-style short-code (confusable `0/O/1/I/L` removed) — keyed on the
_short-code_, not an artifact `<id>`, and with the `origin` injected rather than
a hardcoded `oshun.app`. Earlier doc drafts that wrote the attestation page as
the literal `oshun.app/scene/<id>` were stale on both counts.

### Cue-policy evals — fail-closed parsing + a precision/recall corpus

The cue-policy gate has two halves, both real. The structural half is `parseCue`
/ `sanitizeForCueParse` in
`libs/yemaya/living-scenes-runtime/src/cues/cue-parser.ts`: free text and ASR
transcripts are sanitized (quotes, control characters, code-fence markers, and
HTML tags stripped; whitespace collapsed; truncated to 240 chars) and then
mapped to a small constrained verb vocabulary. Anything that is not a recognized
verb returns `verdict: 'unrecognised'`, and a voice cue below
`LOW_CONFIDENCE_THRESHOLD = 0.6` returns `verdict: 'low-confidence'` — so
out-of-vocabulary or low-confidence cues _fail closed_ and **raw text never
reaches the segment generator**. That constrained-verb mapping is the
prompt-injection defense, which is why the _Cue verb mapping_ test below asserts
exactly this fail-closed behavior.

The policy half is the redaction precision/recall evaluator. `evaluateCorpus` in
`libs/yemaya/living-scenes-runtime/src/cue-privacy/intent-redaction.ts` runs the
eight `LILITH_REDACTION_CATEGORIES` (`named-person`, `deceased`,
`medical-condition`, `abuse-disclosure`, `location-of-safety`, `minor-identity`,
`financial-account`, `legal-status`) — each a deterministic regex + replacement
token — over a labeled fixture corpus and reports per-category and macro
precision/recall. This is a deterministic pattern-matcher robust for the test
taxonomy, _not_ an ML classifier; the module header is explicit that real-world
deployment will swap in a model-backed classifier with the same signature.

> A note on which launch locales the cue-policy and crisis evals run over: the
> codebase carries **two divergent launch-locale sets**, and any cross-locale
> eval checklist must pick one and flag the divergence. The platform set
> `OSHUN_LAUNCH_LOCALES` (`libs/oshun/i18n/src/index.ts`) is _eight_ locales —
> `en-US, es-US, fr-FR, de-DE, ar, he, ja-JP, pt-BR`. The Living-Scenes
> locale-parity set `V1_LAUNCH_LOCALES`
> (`libs/isis/workflow-classes/src/living-scene/locale-parity.ts`) is _ten_ —
> `en-US, en-GB, es-ES, es-419, fr-FR, de-DE, pt-BR, ja-JP, ar-EG, he-IL` (with
> `RTL_LOCALES = {ar-EG, he-IL}`). The LS-scoped evals (cue policy, crisis
> parity, locale-parity fixtures) are authoritative over the _ten_-locale
> region-tagged set; treat the eight-locale i18n catalog as the platform
> string-coverage set, not the LS eval target. See
> [Taxonomy, Localization, and Versioning](./taxonomy-localization-versioning.md)
> for the broader localization model.

### Crisis-detection evals — locale parity with a hard imminent floor

`checkLocaleParity` in
`libs/yemaya/living-scenes-runtime/src/cues/crisis-classifier.ts` is the
scene-stream crisis gate: it requires the scene detector to be at parity with
the conversational detector across the Lilith taxonomy. The taxonomy is the
eight `LILITH_CRISIS_LABELS` — `self-harm-imminent`, `self-harm-ideation`,
`harm-to-others`, `severe-dissociation`, `panic-acute`, `overwhelm`, `distress`,
`safe` — and `decideCrisisAction` maps them to a severity of `imminent`
(`self-harm-imminent`, `harm-to-others`), `severe` (`self-harm-ideation`,
`severe-dissociation`, `panic-acute`), or `moderate` (`overwhelm`, `distress`).
The parity gate is a `LocaleParityReport` that is `atParity` only when `macroF1`
clears the threshold _and_ the **`imminentFalseNegativeRate` is ≤ 0.02** — a
hard 2% ceiling on missed imminent-class crises per locale, because a missed
imminent signal is the most dangerous failure mode in the system. The classifier
_model_ is downstream; this module owns the decision shape, the taxonomy
mapping, and the parity contract so any compatible classifier output can be
gated against the same floor.

## Tests — the contracts each gate is verified against

The following are the V1 test contracts. Most are real, executed Vitest suites
in `libs/yemaya/living-scenes-runtime/src/`; the BFF-level ones are Playwright
e2e specs over `apps/oshun/bff/src/routes/living-scenes.ts` (a 1,871-line
route).

- **Score round-trip**: serialize → store → re-render produces a
  pixel-equivalent video within tolerance on N approved nodes (the determinism
  harness over the round-tripped Score).
- **Cue policy enforcement**: voice/text cues that violate Lilith policy are
  rejected at the cue gate, never reach the segment generator, and trigger the
  soft-rejected-cue UX. Backed by `parseCue`'s fail-closed `unrecognised` /
  `low-confidence` verdicts.
- **Cue verb mapping**: free text and ASR transcripts map correctly to the
  constrained verb vocabulary; out-of-vocabulary cues fail closed.
- **Crisis-frame entry**: a crisis-signal cue, a conversational signal, or a
  re-detection-sweep result triggers an immediate fade-to-still + plain operator
  voice + Iris memory-write suspension + safety-incident record. This is exactly
  the cross-surface cascade in
  `libs/oshun/trust-safety/src/crisis/crisis-frame-cascade.ts`: a single
  `lilith.crisis_frame.activated` event (`LILITH_CRISIS_FRAME_ACTIVATED_EVENT`)
  fans out to the five `CRISIS_FRAME_SURFACES` — `psyche`, `lilith-video`,
  `tara`, `iris`, `assistant` — with `haltSynthesis` and `suspendMemoryWrites`
  pinned true and non-overridable, and `crisis-frame-worker.ts` binds the real
  Redis bus. See [Lilith Persona Policy](./lilith-persona-policy.md) and
  [Iris Memory and Identity](./iris-memory-identity.md).
- **Watermark survival**: artifacts pushed through every supported share channel
  recover at least one provenance signal per channel (`evaluateAllChannels`,
  seven channels, ≥ 1-signal-survival pass).
- **Determinism**: same Score + Envelope ⇒ pixel-equivalence; an engine upgrade
  surfaces a "v2 available" notice; a rollback keeps pinned artifacts rendering
  under their pinned envelope.
- **Shareability matrix**: per-component restrictions are enforced at the share
  endpoint, fallback shares present the correct degradation, and crisis-touched
  artifacts are private-only regardless of other components — the
  `resolveShareability` matrix (nine components, four reach levels,
  minimum-reach resolution, crisis precedence), detailed in
  [Keep, Share, Shareability, Takedown, and Lineage](./scene-sharing-distribution.md).
- **Takedown cascade**: voice retirement, source retraction, workflow-class
  freeze, persona termination, and account deletion all reach kept and shared
  artifacts within SLA; nightly sweeps verify compliance. The seven structured
  dispositions (`DISPOSITION_KINDS`) and their `DispositionImpact` records live
  in `takedown/dispositions.ts`; the revocation SLA is the concrete
  `LIVING_SCENE_REVOCATION_SLA_SECONDS = 60`.
- **Tenant share policy**: per-tenant share toggle, scope toggle, retention
  policy, and audit are enforced; ambiguous policy fails closed. In the BFF this
  is `resolveTenantSharePolicy` returning `denied_tenant_internal_scope` when a
  tenant-internal scope is exceeded; the deny-header path is gated behind
  `OSHUN_LIVING_SCENES_HEADER_POLICY_OVERRIDES` (honored only in non-production
  runtimes), and the `living-scene-card` e2e spec exercises it.
- **Public viewer**: `noindex` respected by default
  (`LIVING_SCENE_DEFAULT_ROBOTS_META = 'noindex, nofollow, noarchive'`,
  flippable to `LIVING_SCENE_INDEXABLE_ROBOTS_META = 'index, follow'` on
  opt-in), rate-limit and abuse pipelines functional, and a per-page abuse
  report routing to Lilith. Retired offerings show the tombstone copy
  `PUBLIC_VIEWER_TOMBSTONE_COPY = 'This offering has been retired.'`. The
  `public-scene-abuse-report` and `scene-viewer-deepening` e2e specs cover these
  paths (per `WALKTHROUGH/results/v1-triage-2026-06-23.md`).
- **Reduced-motion parity**: every full-motion render has a reduced-motion
  render with equivalent narration and AD; reduced-motion is independently
  shareable.
- **Accessibility**: caption accuracy, AD coverage, screen-reader navigability,
  keyboard cue access, and OS-accessibility-setting respect.
- **Backend swap**: AAA-tier sessions can flip between `chained-clip` and
  `phase-176-nous` without breaking the customer surface; contemplative-tier
  sessions never resolve to `phase-176-nous` (the `resolveBackend` hard guard).
- **Lineage and remix**: replay-with-tweak from public artifacts produces
  correctly attributed children; takedown of a parent does not delete unrelated
  children; cross-user remix attribution is honored per consent. See
  [Output Gallery, Lineage, Branch, and Replay](./output-gallery-lineage.md).
- **Latency budgets**: P50/P95 frame latency, segment latency, and cue latency
  are met under nominal load _and_ under backpressure (the degradation ladder
  above).

### Content-addressed idempotency in the share path

A subtle but real correctness property the e2e suite pins is that the keep/share
endpoints in `apps/oshun/bff/src/routes/living-scenes.ts` are
_content-addressed_: two identical shares of the same artifact at the same time
mint the _same_ `shortCode` / `shareId`, so a retried share is idempotent rather
than creating a duplicate public link. This is why the `scene-viewer-deepening`
and `living-scene-card` specs run with
`OSHUN_LIVING_SCENES_HEADER_POLICY_OVERRIDES=1` and an injected `x-oshun-now`
clock: without a deterministic clock, two same-time shares land on different
seconds, produce different short-codes, and break the idempotency assertion. The
override is a test-only seam — it is ignored in production runtimes and falls
back to the wall clock.

## Related

- [Scene Safety, Determinism, Provenance, and Cue Privacy](./scene-governance.md)
  — the governance spine these gates verify: the PSE killer, crisis frame,
  determinism harness, and the four provenance channels.
- [Keep, Share, Shareability, Takedown, and Lineage](./scene-sharing-distribution.md)
  — the shareability matrix, takedown dispositions, and personal-artifact
  lifecycle the share and takedown tests exercise.
- [Live Direction, Conductor Runtime, and Blend Kernel](./direction-conductor-blend.md)
  — the cue and blend pipeline whose latency budgets this page sets.
- [Isis Generation Control](./isis-generation-control.md) — the release-gate
  dashboard that surfaces eval failures and blocks promotion.
- [Generation Audience Tiers and Surface Boundaries](./generation-tiers-and-surfaces.md)
  — the entitlement tiers the backend resolver keys on.
- [Living Scenes — Concept and Customer Promise](./living-scenes-overview.md) —
  the customer promise these quality and accessibility bars protect.
- The features hub: [../features.md](../features.md).