Lilith Metaverse · Architecture

Tara Live Classes, Aja Coaching & Lilith Commons

A focused page within the Lilith Metaverse Architecture documentation. The full map and every sibling page live in the Architecture hub.

8sections14 minread2diagrams1table
← Back to the Architecture hubMarkdown source
On this page

If the avatar/audio stack is V3's embodiment substrate, the three subsystems on this page are what people actually come to do inside Lilith: take a contemplative live class from a verified instructor (Tara), get their real body coached from a webcam without the footage ever leaving the device (Aja), and free-roam a shared social world of libraries, observatories, debate halls, and gardens (Lilith Commons). Where combat in V2 had to be provably deterministic, these tenants have a different burden of proof: they have to be provably safe and consent-shaped. A yoga instructor's avatar must not close its eyes or touch a student without explicit permission; an AI persona must not teach an asana its human sponsor never approved; a student's pose data must not leave their laptop; a stranger in the commons must be reportable with 60 seconds of replay evidence that stays inside the tenant's data-residency region. Every one of those is encoded as a state machine or a fail-loud gate in real TypeScript, not left as a comment.

These tenants live as a tier of domain libraries under libs/v3/ — @oshun/aja-pose (libs/v3/aja-pose), @oshun/tenant-tara-studio (libs/v3/tara-studio), and @oshun/tenant-lilith-commons (libs/v3/lilith-commons) — that own the pose math, the asana catalog, the consent/governance policies, and the venue/programming/safety/localization model. They sit on top of the embodiment runtime documented in the sibling page and feed cue events and room state through the world server and gateway. This page is the architecture-side companion for the "Avatars, Audio, and Tenant Experiences" set; the section hub is ../V3_ARCHITECTURE.md.

What ships, honestly#

The three libraries are substantive, domain-specific TypeScript with real algorithms and broad test coverage, not CRUD shells. aja-pose ships a full on-device pose pipeline (MediaPipe Tasks-Vision + MoveNet fallback), an 18-feature biomechanical extractor, a nearest-centroid asana classifier, six known-risk modification ladders, a recursive raw-pose egress blocker, and a Psyche-TTS cue-delivery budget model — 21 automation specs. tara-studio ships a 300-entry asana library, guided practice-plan generation, and a stack of consent/governance policies (eyes-open, physical-adjustment, language linter, lineage grounding, AI-persona sponsorship/region-cap/script restriction, TTS voice contracts) across ~16 spec suites plus a Prisma layer. lilith-commons is the largest single library in V3 — a 13,458-line index.ts with a 4,536-line, 143-case test suite — covering eight launch venues, programming calendars and festivals, a seven-surface accessibility suite, an 18-locale localization pipeline, and an in-world safety model.

Five honest qualifications. First, Aja's "≥ 92 % per-asana accuracy" and "≥ 90 % human-reviewed cue relevance" gates are computed against a deterministic synthetic corpus, not real labeled webcam footage. The feature extractor (extractAsanaFeatureVector) and the centroid classifier are genuine and run on real MediaPipe landmarks at runtime, but buildCanonicalAsanaTrainingCorpus synthesizes its samples from sinusoidal prototypes (canonicalAsanaPrototype) plus small perturbations, and buildAjaCueHumanReviewSet builds its "tara-senior-teachers" review cases programmatically from centroids. Treat those numbers as self-consistency proofs of the algorithm, not field-measured accuracy. Second, the pose models and the Psyche-TTS voice are runtime/provider-gated: MediaPipe and MoveNet load via dynamic import (the MediaPipe wasm bundle is vendored in node_modules, MoveNet/TF is not), and the TTS path runs through an injected AjaPsycheTtsAdapter with a deterministic test double; the 250 ms latency gate validates the model, not a live network. Third, Tara's "300 asanas" is 30 base poses × 10 lineage framings generated combinatorially, each entry carrying real per-family contraindications and modifications — substantial content, but not 300 distinct shapes. Fourth, Sophia "grounding" in the lineage editor is a deterministic check over a source registry, not a live LLM call. Fifth, the in-world rendering of all this — world-space UMG widgets, Sequencer ritual rooms, the planetarium dome — is UE content referenced by string (/Game/Venues/Commons/…); these libraries own the data, policy, and orchestration tier. The sections below say where each claim is backed.

Aja: body-aware coaching that never sees your camera#

Aja is the only V3 loop that grades a real human body. A student opts into camera-based pose estimation; frames are processed entirely on-device, and only derived features and aggregate metrics ever leave the client. The pipeline in libs/v3/aja-pose/src is six honest stages.

flowchart LR cam["Webcam frame<br/>(opt-in)"] --> rt["Pose runtime<br/>MediaPipe → MoveNet"] rt --> feat["18-feature vector<br/>extractAsanaFeatureVector"] feat --> cls["Nearest-centroid<br/>asana classifier"] target["Active asana target<br/>(class sequence)"] --> cue cls --> cue["Alignment cue gen<br/>(10 rules + ML assist)"] cls --> risk["Risk flagger<br/>→ modification ladder"] cue --> voice["Psyche-TTS voice cue<br/>(≤ 250 ms, via V3Voice)"] cue --> text["Text cue overlay"] rt -. aggregate only .-> iris["Iris memory<br/>(raw egress blocked)"]

The runtime and its fallback (runtime.ts). The primary estimator is the MediaPipe Tasks-Vision pose-landmarker (33 named landmarks), with a TensorFlow MoveNet lightning/thunder detector (17 keypoints) as fallback. AjaPosePipeline (:399) is a real fallback cascade: it initializes adapters in order, and on any init-failed/estimate-failed advances to the next adapter and fires onFallback — so a GPU-delegate failure degrades to CPU MoveNet rather than dropping the student. Both device profiles (web-m1-mac, mobile-iphone-15-pro) target 15 Hz (AJA_POSE_MINIMUM_HZ, 66.7 ms budget), and assertAjaPosePerformanceGate throws if observed Hz or inference blows it; the estimators load through a dynamicImport seam so the geometry unit-tests without wasm.

Features and classification (classifier.ts). extractAsanaFeatureVector (:391) turns landmarks into 18 rotation/scale-robust biomechanical features (ASANA_FEATURE_NAMES): shoulder/hip slope, spine angle, four limb joint angles computed as acos(dot / |a||b|), wrist/ankle spans, shoulder–hip twist, and side lengths. trainAsanaCentroidClassifier (:239) averages per-asana vectors into centroids over the 30 canonical asanas (AJA_CANONICAL_30_ASANAS, Tadasana through Savasana); classifyAsanaFeatureVector returns the nearest centroid by Euclidean distance with a first-to-second-distance confidence; and evaluateAsanaClassifier builds a confusion matrix and enforces the ≥ 92 % per-asana gate. As flagged above, the corpus is synthetic; the classifier is real.

Alignment cues (cue-generator.ts). Ten rules (AJA_ALIGNMENT_CUE_RULES, :93) compare features against thresholds — absolute-greater-than for a tilted shoulder line, centroid-deviation for a spine that has drifted from the target asana's centroid, less-than for a collapsed stance. Candidates are sorted by priority + deviation, sliced to the top N, and, when a classifier model is supplied, decorated with AjaCueMlAssistance (predicted vs. target asana, distance, max feature deviation). If nothing triggers but the classifier says the student is in the wrong shape entirely, it emits a single return-to-target cue. The relevance of these cues is gated at ≥ 90 % (AJA_CUE_RELEVANCE_MIN_HUMAN_APPROVAL) against the (synthetic) review set.

The risk ladder (risk-flagger.ts). Six known-risk rules (AJA_KNOWN_RISK_MODIFICATION_RULES, :102) cover shoulderstand, plow, lotus, chaturanga, upward-dog, and tree, each keyed to anatomical risk factors (neck, lumbar, knee, wrist, shoulder, balance, hypertension, pregnancy) and carrying a three-step modification ladder with real cue text and props ("place a block under your sacrum for supported bridge" → "legs up the wall" → "rest in savasana"). A new practitioner trips every rule for their target asana; an experienced one only trips rules matching their declared risk factors. routeAjaRiskModification routes within a 250 ms budget (AJA_RISK_MODIFICATION_BUDGET_MS) and the gate requires every validation case to route correctly and on time.

The privacy gate (privacy.ts). This is the load-bearing safety property. containsAjaRawPoseData (:161) recursively walks any egress payload and returns true if it contains any raw-pose key (image, frame, poses, landmarks, worldLandmarks, world, keypoints, segmentationMasks). buildAjaIrisAggregateMetricsEnvelope produces an Iris-bound envelope of counts/averages only and runs it through assertAjaNoRawPoseDataLeavesDevice, which throws if raw data is present. So the only thing that can reach Iris is aggregate metrics, and the gate fails loud, not silent.

Voice delivery (voice-cue.ts). Cues are spoken in the instructor's own voice through deliverAjaVoiceCue, which builds an AjaPsycheTtsCueRequest (Opus, low-latency, routed psyche-tts → v3voice on a student-private-cue channel) and measures pose-to-first-audio against the 250 ms budget. The tone profile is validated (warmth/energy ∈ [0,1], speech rate ∈ [0.75,1.25]). The provider is an injected adapter; the gate proves the budget, not the wire.

Tara: the live-class pipeline, governed end to end#

Tara instructors author entirely from the web — the V3_ARCHITECTURE authoring split routes asana sequences, schedules, profiles, and recording opt-in through the V1 Lilith Studio, and the UE5 client renders the published manifest at runtime. tara-studio is the contract/policy tier behind that, persisted through its own Prisma models (V3LiveClassSession, V3AsanaSequence, V3Asana, V3InstructorProfile, V3InstructorCredential, V3AjaCueEvent, V3PracticePlan, V3OnDemandClassRecording).

The asana library (asana-library.ts). TARA_CANONICAL_ASANA_LIBRARY (:234) is 30 seeds × 10 lineage variants = 300 entries (TARA_ASANA_LIBRARY_GA_MINIMUM). Each carries lineage teaching notes (Iyengar-supported, Ashtanga vinyasa, trauma-informed, chair-accessible, …), ≥ 4 modifications, family-keyed contraindications (an inversion warns on neck injury and hypertension; a backbend warns on lumbar compression and pregnancy), an Aja cue bundle, an editorial signature, and a design review. validateTaraCanonicalAsanaLibrary (:240) is an all-or-nothing GA gate: every entry must be signed, design-approved, and complete.

Onboarding to a plan (practice-plan.ts). A 5-minute intake (TARA_GUIDED_ONBOARDING_DURATION_SECONDS) maps a goal/experience/intensity to a starter template and emits a 4-week schedule (TARA_PRACTICE_PLAN_DURATION_WEEKS) of on-demand-class and self-practice sessions; practice-plan-adjustment.ts re-tunes it from attendance, Aja signals, and Arete logs.

The consent and safety stack is the heart of the tenant — five independent gates, each a state machine with an audit event:

Policy File Rule
Eyes-open default eyes-open-policy.ts Avatar eyes stay open; an eyes-closed cue is suppressed unless the invitation text matches an explicit permissive pattern (EXPLICIT_EYES_CLOSED_INVITATION_PATTERNS: "if it feels", "you may", "optional", …)
No-surprise contact physical-adjustment-consent.ts Default hands-off; in-world contact is blocked until the student grants an explicit, dialog-matched consent record — otherwise dialog-required/blocked
Invitational language invitational-language-linter.ts 13 directive phrases (must, should, force, push, hold, …) are flagged blocking, forcing needs-editorial-revision before a sequence script ships
Class style mode class-style-mode.ts Directive cueing is only allowed in an explicitly-declared advanced-directive-opt-in class; the listing surfaces the declaration
Lineage grounding lineage-grounding.ts A lineage claim needs a citation; groundLineageClaim marks an uncited claim needs-citation and blocks editorial review

The eyes-open policy is notable because it exists twice and must agree: this TS Studio-side policy (evaluateTaraSequenceEyeState, :43) and the C++ runtime-side V3TaraEyesOpenPolicy documented in ./avatar-animation-and-audio.md share the same default and the same "explicit invitation required" rule, so authoring and the avatar runtime can't disagree about whether an instructor's eyes may close. The lineage grounder is a genuine little reasoning engine: buildTaraLineageEditorialReview (:204) walks each claim against a source registry with authority scores, and the fixture's uncited "direct Mysore family line from 1932" claim (TARA_LINEAGE_UNSOURCED_CLAIM_ID) is exactly what blocks the review.

AI-persona governance. Tara permits AI instructor personas, fenced by four real gates. registerTaraAiPersonaSponsor (ai-persona-sponsor.ts:107) requires a verified human instructor with credentials to sponsor a persona and bind it to a closed scope set (sequence-script-only, student-q-and-a, booking-prep, safety-handoff, …); a sponsor can disable it with an audit trail. evaluateTaraAiPersonaRegionalSchedulingCap (ai-persona-region-cap.ts:50) enforces one AI persona per four verified human instructors per region by floor division — a region with three humans gets zero personas. The sequence-script restriction routes any out-of-script asana to handoff-to-sponsor or a graceful-noop, and the nameplate lock keeps a non-removable "sponsored by a human" label on the persona. TTS voice signatures get their own five-file sub-stack: opt-in consent with 30-day withdrawal, a contract template that forbids cross-instructor/cross-tenant reuse and grants a 5 % added royalty share, a recording-session pipeline, a runtime scope-lock that auto-withdraws on misuse and alerts an operator, and a royalty-distribution settlement.

A class itself runs the V3_ARCHITECTURE pipeline: schedule (web Studio) → booking (V1 BFF) → room spawn (lilith-world-server) → UE clients load V3Mode_TaraLiveClass while fallback clients load the three.js scene → students opt into the camera feeding the Aja loop above → per-student cues render as world-space widgets (UE) or HUD text (fallback) → the session logs to Arete and, with opt-in, records to the on-demand library.

Lilith Commons: the shared social space#

Commons is the free-roam world — eight launch venues that re-render V1 domain experiences inside UE5 with Tier-2 fallbacks. lilith-commons is where the bulk of the safety, accessibility, and localization burden actually lives.

Venues and capacity. V3_LILITH_COMMONS_LAUNCH_VENUE_IDS (:150) enumerates the eight — atrium, garden-of-cycles, the-stacks (Nisaba), observatory (Nyx), debate-hall (Veritas), lecture-hall (Metis), atrium-of-practice (Arete/Tara ritual rooms), and lantern-hall — across eight venue kinds and three capacity tiers (quiet=16, standard=64, assembly=256). Each carries an editorial signoff and a design review, and the launch-catalog report gates on all eight being live, signed, and approved. Per-venue pipelines are modeled in detail: Veritas runs six radial podiums with 120-second turns and six Sophia inline citations rendered under 120 ms; Metis seats 256 across eight tiers with eight breakout cells; Nisaba's open stack exposes 216 editions across 12 shelves; and every venue gets one voice-free, avatar-hidden solitary cell (/Game/Venues/Commons/SolitaryCell/LVL_CommonsSolitaryCell_Template) so the commons always has a quiet exit.

Programming. A daily ambient-loop calendar plus scheduled programming covers every venue, with cross-tenant festivals as first-class report types: the Equinox Festival (4–7 days), Lineage Week, Crossover Weekend, and a Remembrance Hall memorial program with tone review and a two-reviewer moderation floor.

Accessibility is a seven-surface suite, each with a measured QA gate that fails loud on regression: avatar-anchored live captions (web/mobile/VR, anchored avatar-head-top, p95 ≤ 1000 ms); reduced-motion (camera ≤ 6°, avatar ≤ 1 cm, teleport locomotion); three color-vision palettes (asana-cue contrast ≥ 4.5); one-handed mobile HUD (controls in the top ≥ 56 %, hit targets ≥ 44 px); photosensitive-safe mode (strobe ≤ 3 Hz, attenuation ≥ 90 %); cognitive-load reduction (≤ 5 primary controls, dwell ≥ 1200 ms); and full keyboard/single-switch navigation (scan ≤ 1200 ms).

Localization ships real translations, not metadata. launch-locale-hud-translations.ts is the source of truth for six HUD strings across the launch locales, and the suite asserts no English string is copied through to a non-English locale (English regional variants legitimately share wording). buildV3LaunchLocalizationReport (:6680) gates 18 enumerated locales with ≥ 16 carrying HUD + dubbing + RTL coverage, atop a voice-dubbing pipeline and a documented cultural-adaptation review.

Safety is the in-world reporting → review → sanction chain:

flowchart LR point["Point at avatar<br/>createV3InWorldReportDraft"] --> cat["Pick category<br/>(harassment … minor-safety)"] cat --> replay["Attach 60 s replay<br/>(voice + motion, tenant-resident)"] replay --> submit["submitV3InWorldReport"] submit --> sla["Operator review SLA<br/>(7-day, ≥ 0.95)"] sla --> sanction["Progressive sanction<br/>warning → mute → realm-ban → account-ban"] sanction --> appeal["Appeals routing"]

A reporter points at an avatar; the report attaches a 60-second replay buffer (V3_IN_WORLD_REPORT_REPLAY_WINDOW_MS = 60_000) of voice and avatar-motion frames that stays in the tenant's residency region. The operator SLA dashboard requires ≥ 0.95 compliance over seven consecutive days, and advanceV3ProgressiveSanctionState (:12470) walks the audit-logged ladder warning → mute → realm-ban → account-ban. Minor protection is a hard default: evaluateV3MinorProtectionDefaults (:12575) locks four surfaces for a 13-year-old (voice-chat, presence-non-cohort, signed-edition-purchase, remix-licensing) with no bypass, fires operator alerts within 30 s on flagged interactions, and blocks an adult instructor from scheduling a minor cohort class without a cleared background check. All of this is persisted through the V3Report3D, V3SpatialTranscript, V3EmbodiedConsent, V3Room, and V3Venue Prisma models.

isis-motion: a labeled placeholder#

libs/v3/isis-motion is descriptor-only — src/index.ts exports a capability descriptor (motion-brief, retarget-pass, cue-export) and a readiness scorer, with no implementation behind it. It is the planned home of the motion-authoring service that would sit between Tara's asana sequences / Saraswati's motion presets and the retarget runtime. Reading it expecting a motion engine will mislead; the real motion logic lives in V3Avatar (IK, retarget, posture) and in the asana catalogs here.

Edge cases and failure modes#

  • Raw pose can't leak. assertAjaNoRawPoseDataLeavesDevice throws on any egress payload containing a raw-pose key; Iris only ever receives counts and averages.
  • The pose runtime degrades, never dies. A failed MediaPipe init or estimate cascades to MoveNet and fires onFallback; only when all adapters fail does a frame throw.
  • Eyes-closed needs words, in two places. Both the TS Studio policy and the C++ avatar policy suppress an auto eye-close cue absent an explicitly permissive invitation, and record the suppression reason.
  • Contact is blocked by default. A physical adjustment is dialog-required until a student grants a consent record whose dialogId and studentUserId match — a stale or mismatched record stays blocked.
  • Directive language fails the build. A single must/force/push in a sequence script flips the linter to needs-editorial-revision.
  • AI personas are rate-limited by humans. Region cap is floor division: fewer than four verified humans means zero personas, full stop.
  • Reports keep their evidence local. The 60-second replay is tagged with the tenant residency region and persisted only as attached-report data, never as a durable rolling transcript.
  • Synthetic-gate caveat. Aja's accuracy/relevance gates prove the algorithm against generated data; they are not field accuracy.

How it connects#

These tenants consume V3's embodiment substrate. The instructor and student avatars, the eyes-open/posture policies, and blendshape-driven expression all come from avatar, animation & spatial audio; Aja and psyche-3d are the two embodiment loops that meet in a Tara class — Aja grades the human, psyche-3d animates the avatar. The premium-persona / concert side of the same substrate is the Saraswati stage pipeline, which shares Tara's TTS-voice and provenance machinery. Booking, room spawn, cue-event fan-out, and presence ride the world server and gateway, and the Tier-2 rendering of cues and venues is the fallback web client. Where these @oshun/* packages sit in the domain map — and how Iris, Arete, Nisaba, Nyx, Veritas, and Metis reach into V1 — is catalogued in Oshun Domain Libraries; the module topology is mapped in the subsystem glossary and layout.