Lilith Metaverse · Features

Product Surfaces

A focused page within the Lilith Metaverse Features documentation. The full map and every sibling page live in the Features hub.

9sections11 minread1diagram
← Back to the Features hubMarkdown source
On this page
flowchart LR Person[Player, creator, or operator] --> Route{Capability and device} Route -->|Premium play or authoring| UE[UE5 canonical client] Route -->|Thin device| Pixel[Pixel Streaming] Route -->|Broadest reach| Web[Local web fallback] Route -->|Creator work| Studio[Creator Studio] Route -->|Safety and operations| Ops[Operator console] UE --> World[Shared world and tenant services] Pixel --> World Web --> World Studio --> World Ops --> World

The route is capability-aware rather than brand-aware. A single account may enter through several surfaces, but all durable world state and tenant policy converge on the shared service plane; fidelity may degrade, authority may not.

V3 — the Lilith Metaverse — is not a single screen you open; it is a persistent set of named worlds (yoga studios, concert halls, a Commons observatory) that the same user has to be able to reach from a gaming PC, a VR headset, a phone, or a Chromebook on hotel Wi‑Fi, and that a different set of users has to be able to author and police. That spread is the whole product problem. The premium experience — Lumen global illumination, Nanite geometry, MetaHuman and VRM avatars, Sequencer-driven concerts — can only run inside Unreal Engine 5, so the strategy is to build one UE5 project and drive every premium surface from it: native binaries, a server-rendered Pixel Streaming cook, and the editor authoring teams use. Around that canonical client sit a locally-rendered web fallback so nobody is locked out, a creator studio, and an operator console. This page is the orientation door to the V3 feature catalogue: it walks each product surface in turn, names the real code behind it, and labels honestly what is a substantial implementation, what is a thin-but-real scaffold, and what is an external dependency the repo only models. It is the features-side companion to the architecture's routing deep-dive, and it hands the mechanics off to its siblings rather than re-deriving them. The feature map this page belongs to is ../V3_features.md.

What ships, honestly#

The UE5 client is the canonical surface, and its project scaffolding is real and concrete — though it is lighter than a shipping AAA title and this page will not pretend otherwise. V3/ue/V3.uproject declares 17 modules (15 runtime — V3Core, V3Gameplay, V3World, V3Avatar, V3Animation, V3Input, V3Net, V3Voice, V3UI, V3Audio, V3VFX, V3Cinematics, V3Persistence, V3OnlineServices, V3Telemetry — plus V3Editor and V3Tests) and enables the exact engine envelope the promise rests on: PixelStreaming + PixelStreamingPlayer, CommonUI, GameplayAbilities, Niagara, Metasound, ResonanceAudio + SteamAudio, OpenXR + OpenXRHandTracking, MetaHumanRuntime/MetaHumanSDK, VRM4U, GameFeatures

  • ModularGameplay, Iris, OnlineServicesEOS, MovieRenderPipeline, and Gauntlet. There are three build targets, including a dedicated headless V3PixelStreamingWorker.Target.cs. The honest qualification: the C++ behind those modules is a scaffold, not a 200‑file gameplay codebase — the whole Source/ tree is ~73 .cpp and ~40 .h files (V3Gameplay is a single translation unit), and Content/ holds 108 .uasset files, real authored content but a modest set by AAA standards. The structural commitments (modules, targets, plugins, Game Feature tenants) are genuine; the dense gameplay flesh on top of them is not all there yet.

The two web tiers are real Next.js applications, not marketing stand-ins. The Pixel Streaming shell apps/v3/lilith-web/ centres on a 1,440‑line LilithLanding.tsx that drives the real Epic Pixel Streaming client; the Tier‑2 fallback apps/v3/lilith-web-fallback/ centres on a 4,115‑line FallbackExperience.tsx that renders locally with three.js + Rapier. Both compose substantial domain libraries (libs/v3/lilith-web-pxstream, libs/v3/lilith-engine-web-fallback). The authoring and operator surfaces are real too: libs/v3/tara-studio is a 21‑capability authoring package, the Studio shell lives under apps/oshun/web/src/app/lilith-studio/, and the operator console composes eleven panel components.

Three honest qualifications keep this page from over-claiming. First, there is no V3/web directory — the web surfaces ship under apps/v3/ and apps/lilith/, and V3/api/ is a single documentation pointer (v3-bff-openapi.md) to the Lilith BFF's /api/v3/lilith-* routes, not code. Second, the Pixel Streaming signaller and the headless UE worker fleet are external infrastructure: the browser lib binds the real Epic frontend library and the relay constructs Epic-protocol SDP, but the per-session UE process and Epic's signalling server live in the GPU fleet, not the repo. Third, the web first-load budget is a projected target, not a measured result — V3/WEB_FIRST_LOAD_BUDGET.md states the "observed" p95 figures are hand-authored projections and "no real-device run exists." Where a number is a target rather than a measurement, this page says so. The enforcement story behind the routing — the typed launch contract, the two-layer router, the relay matchmaker — is the architecture companion ../architecture/tier-routing-and-pixel-streaming.md.

The canonical client — Lilith UE5 (V3/ue/)#

The UE5 project is the surface every premium experience is actually rendered in, and the one all the other surfaces orbit. A single V3.uproject is cooked into every native target — Windows, macOS, Linux, the mobile builds, and the VR/AR builds — and into the headless server cook that the Pixel Streaming workers run. The player-facing module V3UI carries concrete CommonUI classes that ground the product copy rather than restating it: V3VrCalibration.h and V3VrComfortSettings.h back the VR comfort promise, V3AvatarNameplateLod.h the crowd-scaling nameplate behaviour, and V3TaraPhysicalAdjustmentConsentDialog.h the no-surprise-contact consent rule that the Tara authoring library also enforces. V3.uproject sets DisableEnginePluginsByDefault: true, so the enabled-plugins list is an explicit, auditable envelope rather than the engine default.

Tenants and modes as Game Feature Plugins#

The granularity claim — that V3 can load exactly the right realm, region, and session type without dragging in the rest — is backed by real .uplugin files under V3/ue/Plugins/GameFeatures/. Three tenant plugins (V3Tenant_TaraStudio, V3Tenant_SaraswatiStage, V3Tenant_LilithCommons) sit above eighteen mode plugins (V3Mode_TaraLiveClass, V3Mode_TaraPrivate, V3Mode_TaraCohort, V3Mode_SaraswatiConcert, V3Mode_SaraswatiFestival, V3Mode_SaraswatiClub, V3Mode_CommonsObservatory, V3Mode_CommonsLectureHall, V3Mode_CommonsDebateHall, V3Mode_CommonsSolitaryCell, and more). Because each mode is its own Game Feature Plugin, per-region, per-platform, and per-session loading is a real dependency boundary in the cook, not a runtime if.

The headless Pixel Streaming worker cook#

V3/ue/Source/V3PixelStreamingWorker.Target.cs is the dedicated server-side build that makes the browser-but-premium tier possible. It is a Monolithic Game target compiled with bBuildWithEditorOnlyData = false, the fifteen runtime modules listed by hand, and project defines V3_PIXEL_STREAMING_WORKER=1 / V3_HEADLESS_PIXEL_STREAMING=1. It also pins strict floating-point (-fno-fast-math -ffp-contract=off, or /fp:strict on Win64) so a server-rendered worker simulates identically to a native client on the shared wire protocol. This target is the process the relay leases per browser session — see ./client-tiers-fidelity-stadium.md.

Mobile, VR/AR, and the native shells#

Lilith Mobile and Lilith VR/AR are configurations of the same project, not separate codebases: the OpenXR and OpenXRHandTracking plugins back the Quest 3 / Vision Pro / PSVR 2 / SteamVR targets and the hand-tracking-driven yoga IK, while the mobile cook targets iOS / iPadOS / Android. Two thin native shells wrap the experience for continuity: apps/lilith/desktop/ is a Tauri 2 desktop shell (@lilith/desktop, with src-tauri/), and apps/lilith/mobile/ is a React Native companion (@lilith/mobile, with an android/ project) that provides V1 continuity surfaces and deep-links into the native client. The deep-link contract is real and documented: V3/docs/deep-links.md defines the canonical HTTPS origin https://app.oshun.com, the custom scheme authority oshun://v3, the /v3/* universal/app-link family, and a ?source=<channel> attribution scheme so a link survives Telegram/WhatsApp/push/email handoff.

The browser-but-premium tier — Pixel Streaming#

apps/v3/lilith-web/ is the thin browser shell that opens at one URL and routes a visitor into a server-rendered UE5 worker. The Next.js app (dev port 3041) renders LilithLanding.tsx, which constructs a Pixel Streaming client from libs/v3/lilith-web-pxstream and POSTs the device's launch request to /api/v3/pxstream/match, mounting the returned worker's video into a player frame. The library @oshun/lilith-web-pxstream is the real binding: player.ts imports @epicgames-ps/lib-pixelstreamingfrontend-ue5.5 — Epic's own UE5.5 Pixel Streaming frontend — and exposes a LilithPixelStreamingAdapter interface plus a createEpicPixelStreamingAdapter() factory, with a window.__OSHUN_V3_PXSTREAM_CONTRACT_ADAPTER__ seam so contract tests can inject a deterministic adapter without a live signaller. The LilithPixelStreamingMode options (adaptive 12 Mbps auto-codec, low-latency 6 Mbps H.264, quality 20 Mbps AV1) are the codec/bitrate envelope the relay's POP selection honours. LilithLanding.tsx is not a stub: alongside the player it carries a live session state band (status, mic permission, first-frame ms, input-probe), a cross-tenant weekly programming calendar, a Tara live-class booking flow with lineage-fund payout disclosure, and a full bank of accessibility controls (reduced motion, colour-vision palettes, one-handed reach, photosensitive-safe strobe attenuation, flat-stereo, cognitive-load simplification, keyboard/single-switch, launch localization) sourced from @oshun/tenant-lilith-commons. The honest boundary: the worker process and Epic signaller it ultimately negotiates with are external fleet, so what ships here is the client shell and its match call, not the GPU backend.

The no-one-locked-out tier — Tier-2 fallback#

apps/v3/lilith-web-fallback/ is the path for hardware too weak for Pixel Streaming, bandwidth below the floor, Pixel-Streaming-restricted regions, or an explicit "Lite mode". It is a Next.js app (dev port 3042) whose FallbackExperience.tsx renders the world locally rather than streaming it: the package depends on three, @react-three/fiber, @react-three/drei, and @dimforge/rapier3d-compat, and the engine wrapper libs/v3/lilith-engine-web-fallback (@oshun/lilith-engine-web-fallback) provides the real runtime. renderer.ts builds an actual three.js scene (WebGLRenderer, Scene, Mesh, lights, geometry) and selects a webgpu / webgl2 / webgl2-webgpu-fallback backend; physics.ts runs Rapier prop-pickup prediction; gateway.ts selects a WebTransport/WebSocket transport plan; steadyFps.ts, parity.ts, ajaCueValidation.ts, taraInstructorHud.ts, and the taraRecording* modules carry the sustained-FPS budgets, tenant fidelity profiles, Aja-cue accessibility, and instructor controls. The fallback speaks the same multiplayer wire protocol and uses the same VRM avatars as Tier 1, rendered at reduced fidelity (no Lumen, no Nanite, lower-poly avatars, Aja cues as HUD overlays). FallbackExperience.tsx is the single largest component in the V3 web tree (4,115 lines): it routes per [tenant], registers a service-worker cache, runs the gateway handshake, drives the Saraswati in-world catalog, and renders the full Lilith Commons program (Nisaba open-stack bookshelves, Metis lecture hall / breakouts / assessment review, Veritas debate hall, Arete journal walls and review circles). Stadium-tier concerts on this path degrade to an audio-only or pre-rendered LL‑HLS preview rather than a full crowd render — the fidelity envelope is catalogued in ./client-tiers-fidelity-stadium.md. The <8 s first-load target is a budget with projected, not measured evidence (V3/WEB_FIRST_LOAD_BUDGET.md).

The authoring surface — Lilith Studio#

V3 deliberately reuses V1's Studio shell rather than building a parallel CMS: apps/oshun/web/src/app/lilith-studio/ lives inside the V1 customer web app and ships LilithStudioShell.tsx, a LilithProvenanceInspector.tsx (the Isis provenance-bundle surface), and authoring subroutes for asana/, avatar-costume/, concerts/, music/, personas/, provenance/, scene/, and tara/. The domain logic behind the Tara authoring tools is the package libs/v3/tara-studio (@oshun/tenant-tara-studio), a real 21‑capability library — not CRUD. It carries a canonical asana library, guided practice-plan generation and adjustment, lineage grounding (Sophia-checked, unsourced claims blocked to editorial review), invitational-language linting, eyes-open sequence defaults, explicit physical-adjustment consent, and a complete instructor TTS voice-signature governance stack: opt-in consent with 30‑day withdrawal (tts-voice-consent), a contract template forbidding cross-instructor reuse with a 5% added royalty (tts-voice-contract), recording-session pipeline, royalty distribution, and a runtime scope-lock that auto-withdraws on misuse and alerts an operator. The AI-persona rules — verified-human sponsorship, sequence-script restriction, non-removable "sponsored by a human" nameplate, and a regional cap of one AI persona per four verified human instructors — are likewise real modules, not policy prose. Authored scenes are signed by Isis and promoted through V1's environment-promotion gates; the pipeline is ../architecture/authoring-and-content-pipeline.md.

The operator surface — Lilith Operator Console#

apps/oshun/admin/src/app/lilith/page.tsx extends V1's Admin Web with a session-gated operator console (it redirects to /unauthorized without an admin session). The single route is thin, but it composes eleven real panel components, each mapping to a documented operator concern: a live-room dashboard and room-takeover panel (moderation), Saraswati concert scheduling and concert-calendar approval (programming), an instructor-verification queue and Tara class-scheduling panel (onboarding), an AI-persona behaviour review panel (persona-policy audit), a generation-queue inspector, a takedown executor, and display-name-override and reputation panels (safety and trust). The console is a read-and-act surface over the same services the client uses; the heavy logic lives in those services and the panels, not the page.

The backend surfaces, in one breath#

The premium tiers are authoritative-backed: apps/v3/lilith-world-server/ (Rust) owns room state, presence, physics authority, music sync, and ticket gates; apps/v3/lilith-realtime-gateway/ (Rust) shards clients to the world-server node holding their instance; and apps/v3/lilith-pxstream-relay/ (Rust, service port 43103) is the matchmaker/signaller fleet for browser sessions. The launch decision itself is resolved in the Lilith BFF (apps/lilith/bff/src/routes/v3-lilith-launch-route.ts) — the surface picker documented at V3/api/v3-bff-openapi.md. These are the subject of the architecture companion, not this page.

How the surfaces share one spine#

The reason these are facets of one product and not five bolted-on apps is that they all read the same backbone. Every tier — native UE, the Pixel Streaming worker, and the Tier‑2 fallback — speaks the same multiplayer wire protocol to the same authoritative Rust world server, uses the same VRM avatars, and authenticates through the same V1 BFF. A Pixel Streaming worker is itself just another world-server client that happens to render server-side and forward input over a WebRTC data channel, which is why the fallback and the streamed worker can share a room. The web and operator surfaces consume produced results — decisions, schedules, provenance bundles — rather than perturbing the simulation, the same layering discipline that lets a claim on this page be traced to a module on disk. The product stays coherent because the surfaces stay layered.