Maya Domain — Features

Maya (माया) is the Infinite Virtual Universe Creation domain of the Oshun monorepo. Named after the Hindu goddess of illusion and divine creative power who weaves the fabric of perceived reality — her name deriving from Sanskrit roots meaning both "illusion" (what virtual reality creates) and "to measure/build" (what this platform enables) — Maya provides the technological stack for constructing interlinked metaverses, VR social platforms, liveable virtual cities, and games. Its long-term ambition spans Lumen-class global illumination, Nanite-class virtualized geometry, multi-physics simulation, HRTF spatial audio, planetary-scale procedural world generation, LLM-powered NPC characters, large-scale multiplayer, VR/XR across major headsets, creator tooling, modding and governance, and a curated inspiration library.

Maya is an early-stage domain with a real spine. What exists on disk today is 17 libraries under libs/maya/ (there is no apps/maya/ or services/maya/): one Rust Cargo workspace, engine-core, holding 28 crates that implement the engine kernel, ECS, job system, renderer, physics, audio, atmosphere, avatars, NPCs, multiplayer, VR/XR, and procedural-generation runtimes; nine substantial TypeScript libraries (client, database, server, games, inspirations, testing, tooling, documentation, and the engine-core TS facade); and seven single-module TypeScript readiness-evaluation facades (genesis-terrain, genesis-urban, genesis-flora, physics, renderer, scene, world) that score engine configuration rather than run the engine. The sections below describe the feature areas Maya covers. Where a feature describes a design goal rather than shipped code, it is labelled (planned). Several headline product narratives — the modding/governance framework, the voice-interactive modding companion, and game-platform integrations — are roadmap items, marked accordingly.

Universe Engine Core (@maya/engine-core)#

The foundation of the Maya stack is a Rust Cargo workspace of 28 crates that implement every layer of the real-time simulation: memory management, entity data, CPU scheduling, rendering, physics, audio, atmosphere, avatar animation, NPC AI, multiplayer networking, VR hardware abstraction, and procedural world generation. All other Maya features — gameplay systems, the TypeScript server, the client SDK — are built on top of this foundation.

The kernel (maya-kernel), ECS (maya-ecs), job system (maya-jobs), fiber executor (maya-fibers), allocators (maya-alloc), math (maya-math), spatial structures (maya-spatial), world partitioning (maya-world), and time/simulation (maya-time) crates provide memory management, scheduling, scene partitioning, and the plugin extension points that the renderer, physics, audio, and gameplay crates build on. A thin TypeScript facade (engine-core/src/index.ts) exposes a virtual-environment readiness check.

Entity-Component-System (ECS) Architecture#

Game objects, NPCs, terrain chunks, lights, and every other simulation entity are represented in an Entity-Component-System. This design separates identity (entity IDs), state (components — pure data structs), and behaviour (systems that iterate components). The cache-friendly layout is what allows modern CPUs to process thousands of entities per millisecond.

• ECS Data Model: All game objects — players, terrain chunks, NPCs, vehicles, lights — are represented as lightweight entity IDs with attached components (pure data structs with no logic). Systems iterate over entities that match a component signature and apply behaviour. This cache-friendly design allows modern CPUs to process thousands of entities per millisecond with excellent spatial locality. Maya's ECS is the backbone every other engine layer is built on.
• Job System and Fibers: A work-stealing thread pool executes engine jobs across all available CPU cores. Long-running jobs (physics simulation, animation, AI) are expressed as fiber-based coroutines that can be suspended and resumed, ensuring the rendering thread is never stalled. This enables deterministic frame budgeting across 16+ cores.
• Memory Allocators: Three allocation strategies cover the engine's needs — arena allocators for per-frame temporary data that zero on frame boundary (zero GC cost), pool allocators for fixed-size component arrays with O(1) alloc/free, and stack allocators for hierarchical sub-systems. Combined, they eliminate heap fragmentation in long-running sessions.
• SIMD Math Library: Vectors, matrices, and quaternions are implemented with explicit SIMD intrinsics (SSE4.2/AVX2/NEON) for physics, rendering, and AI workloads that perform thousands of transform computations per frame.
• Spatial Data Structures: Octrees for scene partitioning, Bounding Volume Hierarchies (BVH) for ray intersection, and k-d trees for nearest-neighbour queries provide the acceleration structures that enable visibility culling, physics broadphase, and AI perception at planetary scales.

World Management#

A live virtual world is too large to fit in memory at once. Maya's world management layer divides the world into a streaming grid so that only the player's local neighbourhood is loaded and simulated at any time.

• World Partitioning and Streaming: The world is divided into a hierarchical grid of streaming cells. As the player moves, cells load in the background from disk, NVMe SSDs, or network — the player never observes a loading screen. Cell streaming is predictive: the system anticipates the camera trajectory and prefetches cells before they're needed.
• Origin Rebasing: Standard floating-point coordinates lose sub-centimetre precision beyond approximately 20 km from the world origin. Maya solves this by rebasing the coordinate origin to stay near the player, keeping all positions in a high-precision local neighbourhood. This makes planetary-scale worlds practical without floating-point jitter in physics or rendering.
• Persistent World State and Versioning: World state (player-built structures, environmental changes, NPC deaths) is serialised to a binary format and can be migrated across game version updates using a migration toolchain. Multiple parallel world instances can be diffed and merged — essential for live-service content updates.
• World Instancing: The same world definition can run as multiple simultaneous simulation instances (e.g., different server shards of a dungeon), each with independent state.

Time and Simulation Control#

Decoupling simulation time from rendering frame rate is essential for both determinism and player experience. Maya's time system provides a fixed-timestep accumulator as the simulation heartbeat, with higher-level tools for replay, debugging, and in-world calendar simulation layered on top.

• Fixed-Timestep Simulation: Physics, AI, and game logic run at a fixed 60 Hz or configurable rate, decoupled from the variable rendering frame rate using interpolation. This ensures deterministic gameplay regardless of GPU performance.
• Deterministic Replay and Rollback: The simulation can be run deterministically from any saved state, enabling recording, replay, cheating detection, and client-side prediction rollback for multiplayer netcode.
• Time Dilation and Calendar Systems: Game time can be dilated (slow motion, fast-forward), paused, or stepped one frame at a time for debugging. World calendars with seasons, holidays, and day/night cycles are first-class primitives — not just a sky shader parameter.

Real-Time Rendering Pipeline (maya-renderer crate)#

The maya-renderer crate (~95 source files) is a frame-graph render pipeline. Render passes are declared as nodes in a directed acyclic graph, and the pipeline automatically infers resource lifetimes, barriers, and pass ordering from those declarations. This frees shader authors and feature writers from manually managing GPU synchronisation.

The render_graph module builds a directed acyclic graph of render passes with automatic resource-lifetime and ordering inference, and ~94 feature modules cover the rendering techniques below. The renderer is currently backend-agnostic — it models the pipeline in vendor-neutral Rust and does not yet bind a concrete Vulkan / Metal / DirectX 12 / WebGPU backend, so the multi-backend and hardware-ray-tracing capabilities described here are design targets layered on the existing module structure.

Render Architecture#

• Render Graph: Passes (shadow maps, GI probes, forward opaque, transparent, post-process) are declared as nodes in a directed acyclic graph. The renderer automatically infers resource lifetimes, inserts barriers, and reorders passes for maximum async compute overlap — no manual synchronisation code required.
• Multi-Backend Abstraction: A single high-level API compiles to Vulkan (PC/Linux), Metal (macOS/iOS), DirectX 12 (Windows), and WebGPU (browser) without source changes. This enables the same renderer to power native desktop games, mobile, and in-browser experiences.
• GPU-Driven Rendering: Draw calls are issued from the GPU itself using indirect dispatch. The CPU submits a single command buffer regardless of scene complexity; the GPU culls and dispatches its own work. This enables scenes with millions of draw calls that would stall a CPU-driven renderer.
• Adaptive Resolution and Temporal Upscaling: The render target can be dynamically reduced to maintain frame rate, then upscaled back to native resolution using DLSS (NVIDIA), FSR 3 (AMD), or XeSS (Intel). This delivers near-native quality at a fraction of the pixel cost, essential for VR and high-refresh-rate modes.

Global Illumination (Lumen-Class)#

Global illumination — computing how light bounces between surfaces — is one of the most expensive operations in real-time rendering. Maya uses a combination of software ray tracing (works on any GPU) and optional hardware acceleration (NVIDIA RTX / AMD RDNA 2+) to deliver Lumen-class multi-bounce indirect lighting.

• Software Ray Tracing: A signed-distance-field (SDF) representation of the scene allows rays to be traced without dedicated RT hardware. This enables Lumen-class indirect lighting on any DirectX 12 / Vulkan GPU, not just NVIDIA RTX cards.
• Hardware Ray Tracing Acceleration: On NVIDIA RTX and AMD RDNA 2+ hardware, DXR/VK_KHR_ray_tracing accelerates both the SDF scene representation and direct specular reflections, providing significantly higher quality reflections and shadows.
• Infinite Diffuse Bounces with Temporal Stability: Indirect lighting is computed from a surface cache and radiance cache that accumulates samples across frames. This produces naturally soft, multi-bounce illumination with no banding or fireflies, stable under camera motion.
• Dynamic Time-of-Day and Emissive GI: The GI system updates in real time as the sun moves, allowing seamless transitions from noon to dusk to night, and correctly accumulates light from emissive surfaces like neon signs and fire.

Virtualized Geometry (Nanite-Class)#

Traditional engines require artists to manually author several levels of detail for each mesh. Nanite-class virtualized geometry eliminates this by letting the GPU select the appropriate geometric detail per-pixel at runtime from a precomputed hierarchy.

• Hierarchical Cluster Representation: Every mesh is compiled into a hierarchical tree of micro-triangular clusters. At runtime, the GPU selects the appropriate LOD clusters per-pixel based on screen-space error, producing film-quality meshes without manual LOD authoring.
• Software Rasterization for Micro-Triangles: Triangles smaller than a pixel are rasterised by a custom compute shader rather than the fixed-function hardware rasteriser, avoiding the overhead of submitting billions of tiny triangles through the traditional pipeline.
• Disk Streaming with LRU Cache: Cluster data is paged from disk on demand into a fixed GPU memory budget, with a least-recently-used eviction policy. Film-quality photogrammetry assets (billions of polygons) are viable in real-time because only the visible micro-clusters are ever in VRAM.
• Displacement Mapping and World-Blend Materials: Virtualized geometry supports per-cluster displacement mapping and materials that blend based on world-position (e.g., mud accumulates in crevices, snow settles on flat surfaces).

Shadows and Lighting#

• Virtual Shadow Maps: Shadows are stored at extreme resolution in a virtual texture. Only the shadow texels that are actually sampled during shading are computed, allowing per-object shadow distances and sharp contact shadows on every light source without the traditional soft-shadow/performance trade-off.
• Ray-Traced Area Light Shadows: Lights with non-zero area (windows, fluorescent tubes, sky panels) cast soft shadows whose penumbra width correctly grows with distance, computed via multi-sample ray tracing.
• Light Functions and IES Profiles: Every light can carry a material-based function (animated gobo pattern, dynamic projection) and an IES photometric profile describing the real-world intensity distribution of architectural luminaires.

Materials and Shading#

Physically-based rendering ensures that every material interaction conserves energy — meaning bright specular highlights come at the cost of diffuse brightness, exactly as in the real world.

• Physically-Based Rendering (PBR): All surface interactions are modelled with energy-conserving BRDFs — GGX specular, Lambertian diffuse, clear coat, anisotropic brushed metal, subsurface scattering for skin and organic tissue, cloth shading, eye refraction with caustics, and hair/fur rendering. Each model is grounded in measured real-world material data.
• Node-Based Material Graph: Artists compose materials visually by connecting nodes — texture samples, mathematical operations, procedural generators, data lookups. The graph compiles to optimised shader bytecode. Material functions encapsulate reusable logic (e.g., a "wet surface" node that darkens albedo and raises specularity).
• Material LOD: Complex materials with many texture samples and heavy calculations simplify at distance, reducing ALU and texture bandwidth on far objects where the quality difference is imperceptible.

Post-Processing#

• ACES Tone Mapping and Color Grading: The Academy Color Encoding System (ACES) transform converts the high-dynamic-range render buffer to a displayable image in a perceptually uniform, film-industry-standard way. Color grading via LUTs allows cinematic looks to be applied globally.
• Physically-Based Depth of Field: Bokeh depth of field uses a gather-based approach with circular aperture shapes, producing the characteristic out-of-focus disc patterns of real camera lenses rather than the uniform blurs of cheaper techniques.
• Post-Process Volumes: Areas of the world can have per-region post-processing settings — the player experiences different tone mapping in a neon-lit bar vs. a foggy forest, without any manual trigger code.

Advanced Neural Rendering — (planned)#

The maya-renderer crate does not yet contain NeRF or Gaussian-splatting modules. The @maya/renderer readiness facade does model a MayaRendererNeuralRepresentation (none / nerf / gaussian-splatting / hybrid) so that engine configuration can be scored, but the runtime is (planned).

• Neural Radiance Fields (NeRF): Photorealistic scene capture from multiple photographs produces a continuous volumetric representation that can be rendered from any viewpoint. Static environments (heritage sites, real-world locations) can be imported into Maya with photographic fidelity.
• 3D Gaussian Splatting: A collection of Gaussian primitives — far less expensive to render than NeRF — provides real-time novel view synthesis at playable frame rates. Creator-captured real-world locations are the primary use case.
• Generative Texture Synthesis: AI pipelines (integrated with Isis domain) synthesise seamlessly tileable PBR texture sets from text descriptions, enabling surface materials that would be impossible to photograph.
• Neural Upscaling Integration: DLSS, FSR 3, and XeSS all use learned neural networks to reconstruct a high-resolution frame from a lower-resolution input with temporal accumulation, providing near-native quality at a fraction of the shading cost.

Multi-Physics Simulation Engine (maya-physics crate)#

The maya-physics crate (~59 source files) is a backend-agnostic multi-physics layer. A single PhysicsBackendRegistry selects a solver from requested capabilities, so the simulation code is decoupled from any specific physics SDK. The crate covers the full range of physical phenomena a game world needs: rigid-body collision, constraints, ragdolls, vehicles, buoyancy, cloth, fluids, destruction, and secondary motion.

No concrete Rapier / Havok / PhysX binding is present in the crate today — the solvers are native Rust. The @maya/physics readiness facade scores cloth, fluid, destruction, and particle simulation configuration against a cpu / gpu / hybrid backend choice.

Rigid Body Dynamics#

• Large-World Physics with Origin Rebasing: Physics bodies maintain floating-point precision even when the world coordinate origin is rebased, ensuring stable stacking and collision at any distance from the starting point.
• Hierarchical Simulation Zones: Physics LOD tiers — full simulation for near-player objects, simplified kinematic for mid-range, dormant for distant objects. Transitions are smooth and imperceptible, enabling planetary-scale worlds without blowing the physics budget.
• Deterministic Multiplayer Physics: A deterministic simulation mode allows all clients to run the same physics steps from a shared initial state, producing bit-identical results across platforms. This is the foundation for rollback netcode and replay systems.

Soft Body, Cloth, and Fluid#

• Soft Body and Destruction: Deformable meshes, fracture systems, and procedural destruction are handled via position-based dynamics (PBD) and Voronoi/pre-fracture workflows. Buildings can collapse realistically under impact loads, terrain can crater, and vehicles deform on collision.
• Fluid Dynamics (SPH and FLIP): Two fluid solvers are provided — Smoothed Particle Hydrodynamics (SPH) for small-scale interactive fluids (splashing water, blood, liquid metals) and FLIP for large-scale ocean-quality simulations. Both integrate with the rendering pipeline for physically-correct refraction and subsurface scattering.
• Cloth Simulation: Verlet-integrated cloth with self-collision, wind response, and material properties (stiffness, damping, thickness) drives character clothing, flags, banners, and terrain fabric overlays.

Advanced Character Physics (maya-physics — ragdoll + muscle modules)#

Characters need physics that blends with animation rather than replacing it — a punch landing should knock the character's upper body back physically while the legs continue their keyframed walk cycle. Maya's ragdoll and muscle modules provide this continuous blend:

• Active Ragdoll and Muscle Simulation: maya-physics includes a RagdollArticulation that maps a skeleton to collision bodies and constraints and round-trips animation poses through simulated body poses, plus a muscle_simulation module modelling MuscleFiber / MuscleGroup attachments (origin/insertion, contraction ratio, stiffness, damping) over the crate's SoftBody. The PhysicsAnimationBlender merges keyframe and ragdoll poses and emits motor targets for weighted physical follow-through, so the blend between animation and active physics is continuous rather than a binary switch. A full 200+-muscle proprioceptive/pain/fatigue "Euphoria-class" motor nervous system on top of this foundation is (planned).

Spatial Audio Engine (maya-audio crate)#

The maya-audio crate (~60 source files) is a three-dimensional audio engine providing binaural HRTF rendering, ambisonics, occlusion analysis, room/portal propagation, and HRTF personalisation from ear/head scans. Unlike stereo panning, HRTF convolution encodes the directional acoustic signature of the listener's own ear geometry, producing convincing height and distance cues without surround speakers.

• Head-Related Transfer Function (HRTF) Spatialization: Sound sources are convolved with the player's head-related transfer function — a frequency-dependent filter that encodes the acoustic signature of how sound reaches the eardrum from any direction — creating perceptual cues for elevation and distance that make audio convincingly three-dimensional without surround speakers. HRTFs can be personalised using photogrammetry of the player's ear geometry.
• Room Acoustic Simulation: The engine traces sound rays through geometry to compute early reflections, late reverberation, occlusion, and diffraction. A marble cathedral sounds fundamentally different from a carpeted bedroom, and the difference updates in real time as doors open and close.
• Ambisonics Encoding/Decoding: Scene audio is encoded in third-order ambisonics — a spherical harmonic representation that can be decoded to any speaker layout (stereo, 5.1, 7.1, binaural) at playback time, enabling future-proof asset creation.
• Procedural Audio: Physics events (collision between materials, fluid splashes, wind through foliage) drive granular synthesis and physical modelling engines, producing sounds unique to each event rather than repeating canned samples. Contextual ambient layers respond to player location, weather, and time of day.
• Music Force System: An adaptive music engine that transitions seamlessly between emotional states (exploration, tension, combat, victory) using horizontal re-sequencing and vertical layering without audible seams or looping artifacts.

Procedural Universe Generation#

Maya's genesis subsystem procedurally generates terrain, cities, buildings, and vegetation from compact data specifications, rather than requiring hand-authored assets for every square kilometre of a virtual world. This is implemented as three Rust crates inside engine-core — maya-genesis-terrain (~61 files), maya-genesis-urban (~139 files), and maya-genesis-flora (~61 files) — each paired with a single-module TypeScript readiness facade (@maya/genesis-terrain / -urban / -flora) that scores generation configuration. There is no top-level libs/maya/genesis directory and no genesis-architecture or genesis-cosmos library; building/interior generation lives inside maya-genesis-urban.

Beyond the three Rust crates, Phase 40 (§40.22–40.28) adds six further SOTA procedural-generation packages, present as TypeScript libraries under libs/maya/ (depth varies — some are thin early implementations):

• @maya/genesis-nca — Neural Cellular Automata growth models for self-organizing structures, damage regeneration, and organic pattern synthesis.
• @maya/genesis-tiling — Aperiodic tiling generators (Wang tiles, the einstein/hat monotile) for non-repeating ground cover, texture layout, and level tiling without visible repetition.
• @maya/genesis-physarum — Physarum (slime-mold) transport-network simulation for organic road, river, trail, and cave-network growth.
• @maya/genesis-sketch — Sketch-to-terrain: user or agent strokes interpreted into heightfield edits and biome placement.
• @maya/genesis-neural-flora — Neural L-systems for learned plant morphology beyond the rule-based generators in maya-genesis-flora.
• @maya/genesis-scene-agent — LLM-agent scene orchestration that turns natural-language scene direction into placement, layout, and dressing operations over the other genesis crates.

These mirror the engine-level Neith procgen crates (Phase 53); Maya's packages own world-scale integration with the genesis pipeline and world state.

Terrain Generation (maya-genesis-terrain crate)#

Terrain generation proceeds from geological foundations up through climate and ecology, producing worlds that feel geologically plausible rather than arbitrary.

• Planetary-Scale Heightmaps: Fractal noise stacked across octaves produces geologically plausible continents and ocean basins. A tectonic plate simulation drives mountain range formation, rift valleys, and volcanic arcs. Erosion simulation — hydraulic (flowing water cuts channels), thermal (frost-shatter scree slopes), and coastal (wave undercutting) — then weathers the initial heightmap into naturally rounded terrain.
• Cave and Cavern Networks: Three-dimensional Worley noise combined with geological layering generates cave systems that plausibly extend beneath the terrain surface. Caverns form at soluble rock boundaries; lava tubes follow ancient volcanic flow paths.
• Climate-Driven Biomes and Ecology: A global climate model (temperature, precipitation, prevailing winds) is applied to the terrain to derive biome boundaries using the Whittaker biome classification. Each biome has rules for vegetation cover, ground material, and wildlife population. Seasonal variation and fire spread dynamics are simulated over in-game years.
• Geological Authenticity: Rock strata layering, mineral deposit placement, volcanic and geothermal feature generation (calderas, fumaroles, hot springs), and glacier/ice sheet simulation bring geological realism to world generation.

City Generation (maya-genesis-urban crate)#

City generation starts at the scale of road networks and land-use planning — the large-scale decisions that determine whether a city feels like Tokyo or Paris — and then works down to individual facade details.

• L-System Road Networks: Road networks grow from seed nodes using rewriting rules that model urban planning conventions — grid patterns in flat terrain, organic curves in hilly terrain, ring roads and radial arterials around commercial centres. Land-use zoning (residential, commercial, industrial, parkland) is simulated alongside the road network.
• District Architectural Identity: Each district carries an architectural style flag — cyberpunk, neoclassical, brutalist, solarpunk, etc. — that governs the facade grammar applied when generating buildings. Style flags blend at district boundaries, producing natural transitions rather than hard edges.
• Urban Simulation: Traffic density, pedestrian flows, economic gradients (rent by proximity to amenities), and infrastructure network layouts (water, power, transit) are all simulated at the city scale, producing a world that feels driven by plausible human geography rather than arbitrary decoration.
• City Archetypes: Pre-configured biome-to-city pipelines produce modern megacities, cyberpunk dystopias (Blade Runner/Ghost in the Shell aesthetic), futuristic utopias with vertical gardens and clean energy, alien metropolises (non-human architecture and scale), historical recreations (ancient Rome, medieval European, Imperial Chinese), and fantasy cities (floating islands, underwater domes, tree-grown spires).

Building Generation (maya-genesis-urban — building modules)#

Individual buildings are generated using parametric shape grammars — compact rule sets that produce unlimited unique facades while respecting the architectural style of their district.

• Shape Grammar Facades: Buildings are generated using a parametric shape grammar — the footprint is extruded, floors subdivided into bays, bays articulated with windows, balconies, and cornice detailing using rules specific to the district's architectural style. The system produces unlimited unique facades from a compact rule set.
• Functional Interiors: Interior layouts are generated with room types matching the building's declared function (office, apartment, shop, factory, palace). Furniture and prop placement respects ergonomic clearances and supports NPC pathfinding.
• Working Building Systems: Elevators and escalators move occupants between floors on procedurally generated time schedules. Doors can be locked, opened, and broken by physics. HVAC ducts are navigable by small AI agents. Dynamic damage states (broken windows, fire damage, structural collapse) are represented at the mesh level.
• Style Libraries: Brutalist, Art Deco, Neo-Gothic, Cyberpunk, Alien Organic, Crystalline, Biomechanical — each with period-accurate ornament grammars, material palettes, and proportion systems.

Vegetation and Nature (maya-genesis-flora crate)#

Vegetation is generated using two complementary algorithms: L-systems for individual tree topology (capturing species-specific branching patterns) and space-colonization growth for placing trees and undergrowth across a biome according to light and soil rules.

• L-System Tree Growth: Trees and shrubs are grown using parametric L-system rewriting rules parameterised by species, age, terrain slope, and light availability. The resulting geometry is Nanite-ready (no polygon budget concerns) and includes physically-correct wind response using a skeletal animation derived from the growth structure.
• Ecosystem Dynamics: Plant species compete for light and soil nutrients using simplified growth simulation. Fire spreads stochastically through dry vegetation based on wind and moisture. Alien biomes use exaggerated parameters — bioluminescent spores, gravity-defying floating root structures, crystalline leaf surfaces.

AI World Generation — (planned)#

No world-ai library or AI-world-generation crate exists on disk. The text-to-world, style-guided, and semantic-editing features below are a roadmap layered on the procedural-generation crates.

• Text-to-World: Describe a world concept in natural language (a paragraph or even a few keywords) and receive a complete biome configuration, city layout, and population starter as an editable starting point. The AI proposes; the creator curates.
• Style-Guided Generation: Feed reference images or fictional universe tags (Blade Runner, Miyazaki, Solarpunk) to steer the procedural parameters of terrain, urban morphology, and architectural grammar in the direction of that aesthetic reference.
• Semantic Layer Editing: Edit the world at the conceptual level — "make this district poorer and older" or "this river should have flooded this valley" — and watch the procedural parameters update to reflect the change.

Avatar and Character Systems (maya-embodiment crate)#

The maya-embodiment crate (~153 source files — the largest crate in the workspace) is the avatar creation, morphing, and animation stack. Creating a convincing avatar requires solving several hard problems simultaneously: the body must look right across a continuous range of shapes, animate convincingly with sparse VR tracking data, and convey facial expression in real time. Maya's embodiment system addresses all three.

The ParametricBodyMorphingSystem drives descriptor-based body morphing with rig bindings, body measurements, and clothing-fit metrics; a deep animation module set covers IK, motion matching, retargeting, compression, and procedural secondary motion.

Avatar Creation and Customisation#

• Parametric Body Morphing: A blendshape-based morphable body model supports continuous variation across body shape dimensions (height, weight distribution, limb proportions, muscle mass) with physically-correct cloth deformation following body shape changes. Face scanning via device camera produces a 3D mesh reconstruction for high-fidelity self-representation.
• AI-Assisted Character Generation: Text prompts or reference images drive an AI pipeline that configures body morphology, chooses compatible skin tones and textures, and styles hair, clothing, and accessories. Users can iterate from the AI proposal with manual refinements.
• Species and Creature Creation: Non-human avatar species — fantasy races, sci-fi aliens, anthropomorphic animals, abstract geometric forms — are supported through species-specific morphable base meshes with appropriate rig topologies. The avatar system is not human-centric; any bipedal or quadrupedal form can be tracked and animated.
• Digital Couture: High-fidelity fashion and clothing simulation with physically-based cloth materials, style inheritance from real-world fashion trends, and a creator marketplace for wearable assets.

Animation Systems#

• Full-Body IK from Sparse Tracking: With only an HMD and two controllers (3-point tracking), the system infers a plausible full-body pose using a learned inverse-kinematics model trained on mocap data. Additional trackers (waist, feet, elbows) progressively improve fidelity. The result is convincing embodiment without a full mocap suit.
• Motion Matching: Instead of hand-authored state machines, an animation database of mocap clips is searched in real time for the pose that best matches the character's current trajectory and velocity. Transitions are blended automatically, eliminating the jarring snapping of traditional animation graphs.
• AI Motion Generation (Motion Diffusion) (planned): For situations without matching mocap data, a diffusion model would synthesise plausible motion from a description (e.g., "character limps while carrying a heavy box up stairs"), feeding the same motion-matching database as recorded clips. No motion-diffusion module exists in maya-embodiment today.
• FACS Facial Animation and Neural Lip Sync: 46 Action Units from the Facial Action Coding System (FACS) drive 250+ blend shapes, enabling a full range of human expressions. Lip sync is driven from phoneme streams (from ASR or TTS pipelines) in real time with <15 ms audio-visual sync accuracy. Micro-expressions (50–500 ms involuntary emotional leaks), naturalistic blinks (15–20 per minute), and saccadic eye movements are all modelled.
• ARKit Blend Shapes: All 52 ARKit facial blend shapes (iPhone's TrueDepth camera output) are fully supported, enabling real-time facial tracking that drives the avatar face directly from the player's expression.
• Procedural Secondary Motion: Hair, clothing, and accessory secondary motion is computed procedurally from the primary skeleton's acceleration, providing natural overlap and follow-through without per-asset simulation setup.
• Animation Retargeting: Motion captured on one skeleton topology automatically retargets to a different skeleton topology (e.g., human mocap → creature rig) using proportional bone mapping and foot IK correction.
• Animation Compression: Skeleton streams use entropy coding and delta compression across frames. A 64-bone avatar at 60 Hz can be compressed to under 1 kbps, enabling thousands of simultaneous avatars in a shared world.
• VR Body Tracking Tiers: Three-point (HMD + controllers), six-point (+ waist tracker + feet), and full-body eleven-point tracking — each tier handled by a dedicated calibration and inference pipeline, with smooth upgrade/downgrade transitions when tracker hardware is added or removed.
• Learned Motion Matching (Phase 79 — planned): A lightweight neural network compresses the mocap database by 10–100× for memory-constrained platforms (mobile, Quest), while a decompressor reconstructs the best-matching pose in real time.

AI Characters and NPCs (maya-souls crate)#

The maya-souls crate (~76 source files) is the NPC AI stack: LLM-powered characters with consistent personality, long-term memory, and autonomous behaviours. The central architectural decision is to place a provider-neutral chat-completion layer (MayaLlmIntegrationLayer) between the NPC logic and any specific AI API — so the same NPC personality system works with Claude, GPT-4, or a locally hosted Llama model without code changes.

The llm_integration module provides a provider-neutral chat-completion layer (MayaLlmIntegrationLayer) that normalizes Claude / GPT / local Llama-style APIs; behaviour, memory, personality, and social modules build on it.

LLM-Powered Dialogue#

• Character-Consistent Dialogue: Each NPC carries a persona card (backstory, personality traits, current goals, knowledge scope, speech patterns) that is injected into the LLM context alongside world state. The NPC will refuse to discuss events it wasn't present for, remember past conversations with the player, and update its opinions based on narrative outcomes.
• Long-Term Memory and Relationships: Episodic memories (significant conversations, witnessed events, received favours or slights) are stored in a vector database and retrieved via semantic similarity. Relationships between NPCs and players are quantified (trust, affection, enmity) and influence dialogue tone and willingness to assist.
• Multi-Modal Interaction: NPCs perceive voice (speech recognition), gesture (IK pose analysis), and environmental context (nearby objects, weather, time of day) as inputs alongside text. An NPC blacksmith will comment on the weather and adjust their hammering rate based on nearby ambient sound.

Personality and Emotion Systems#

• Big Five Personality Modelling: Each NPC's behaviour is governed by a Big Five trait vector (openness, conscientiousness, extraversion, agreeableness, neuroticism). An extraverted NPC initiates conversations with strangers; a neurotic one expresses worry about events the player reports; a conscientious guard actually checks IDs rather than waving people through.
• Emotional State Machines: NPCs have continuous emotional states (valence-arousal-dominance model) that shift in response to events and decay back to a personality-derived baseline over time. Emotion drives facial expression blending, gesture frequency, and the tone of generated dialogue.

Autonomous Behaviours#

• NVIDIA ACE Integration: The perception-planning-action loop from NVIDIA's Avatar Cloud Engine provides the scaffolding for NPCs that autonomously navigate the world, pursue goals, react to events, and respond to player actions in sub-100 ms latency.
• NPC-to-NPC Social Simulation: NPCs track relationships with each other, propagate rumours through social networks, form factions, and participate in simulated economies. A visiting merchant NPC will sell their wares, patronise local taverns, and gossip about regional events with other NPCs — all without any scripted interaction.
• Physicalised NPC Life Simulation (Phase 79 — planned): NPCs have simulated sleep/hunger/social needs and daily schedules. They physically react to weather (take shelter during rain, sweat in heat). Economic simulation drives job-seeking behaviour, commerce, and trade routes at the city scale.
• Procedural Narrative Emergence (Phase 79 — planned): A quest and consequence system tracks every significant player action and generates reactive NPC dialogue and emergent storylines. Multiple parallel narratives run simultaneously; player choices in one narrative arc can become the backstory of another.
• NPC Archetypes: Quest givers that generate procedural objectives with tracked consequences; shopkeepers with dynamic inventory and haggling logic; companion characters with relationship arcs that evolve over dozens of hours; ambient population with daily schedules (sleep, work, eat, socialise).

Multiplayer and Social Systems (maya-nexus crate)#

The maya-nexus crate (~77 source files) is the multiplayer infrastructure: a deterministic, transport-agnostic client-server topology of gateways, simulation nodes, and clients. Being transport-agnostic means the session lifecycle, interest management, and delta compression are defined in terms of typed envelopes — not raw sockets — so the underlying transport can be swapped or upgraded without rewriting the multiplayer logic.

The crate includes modules for interest management, client prediction, interpolation, lag compensation, delta compression, mesh topology, social systems (friends, guilds, parties), and cross-world identity and asset transfer.

Network Architecture#

• Distributed Server Mesh: Worlds are partitioned into spatial zones, each hosted by server nodes in the mesh. Zones are dynamically resized and rebalanced as player density shifts. Edge computing nodes reduce latency for geographically concentrated player populations.
• Client-Side Prediction and Server Reconciliation: Players experience responsive local simulation; the server authoritatively reconciles divergent states and applies rollback where needed. Delta compression transmits only the differences between successive world snapshots, minimising bandwidth.
• Interest Management at Scale: The interest management layer ensures each client only receives updates for entities within its perceptual range. Configurable per-entity update rates (physics-driven objects at 60 Hz; distant NPCs at 2 Hz) further reduce bandwidth without sacrificing responsiveness.
• Relay and NAT Traversal: A global relay network enables direct P2P connections where available and seamless relay fallback where NAT traversal fails, ensuring players in corporate networks and behind strict firewalls can connect.

Social Features#

• Cross-Platform Identity: Players maintain a persistent avatar, social graph, and inventory across all worlds in the Maya metaverse, regardless of the game title or platform. A player's reputation earned in one world is portable to another.
• Spatial Voice and Text: Voice chat is spatially attenuated and directionally localised using HRTF. Text chat supports emoji, rich media, and machine translation (40+ languages). Custom gestures and emotes are animated on the avatar in real time.
• Groups, Clans, and Events: Hierarchical social structures — friend groups, clans, guilds, alliances — with role-based permissions, shared inventories, and scheduled events. A reputation and trust system surfaces players who consistently enrich social interactions and flags those who do not.

World Linking (Metaverse Portals)#

• Seamless World-to-World Travel: Portal objects in one world trigger a server-side handoff that transports the player's avatar, inventory, and identity to a different world without a loading screen visible to the player (pre-loading happens in the background while the portal animation plays).
• Asset and Identity Transfer: Standards-based asset formats (glTF, USD) ensure avatar appearance and purchased items render correctly across different worlds with different renderer configurations. A common identity layer (OpenID Connect + blockchain attestation for ownership) ensures portability without fragmentation.

Next-Generation Transport — (planned)#

maya-nexus includes server_reconciliation and rts_lockstep modules, but no QUIC/WebTransport transport module. The transport upgrades below are a roadmap on top of the crate's existing transport-agnostic envelopes.

• QUIC/WebTransport Protocol: The next-generation QUIC transport protocol (the foundation of HTTP/3) eliminates head-of-line blocking that plagues TCP-based games. Multiple independent data streams per connection — positional updates, chat, physics events — never block each other even under packet loss. WebTransport enables this in browser contexts.
• Spatial Partitioning for 100k Concurrent Users: A hierarchical interest management tree partitions the world into zones. Zone servers handle physics authority for their region. The mesh auto-scales in real-time to handle player density spikes without manual intervention.
• Sub-20ms Server Reconciliation via Deterministic Lockstep+: A hybrid of client-side prediction (for immediate responsiveness) and deterministic lockstep (for physics-consistent reconciliation) produces rollback-free server authority with typical reconciliation latencies under 20 ms at 60 Hz on 100 Mbps links.

VR/XR Systems (maya-immersion crate)#

The maya-immersion crate (~76 source files) is the VR/XR device abstraction layer. Its design goal is to make the same game code run on Meta Quest, SteamVR, Apple Vision Pro, Pico, PSVR2, and HTC Vive without platform-specific branches — the game interacts with platform-agnostic types and drivers, and the device-specific modules handle the translation.

External SDK dependencies are hidden behind abstract structs/traits — only glam, serde, and bitflags are compile-time dependencies, so the headset SDK integrations are scaffolded interfaces rather than linked vendor SDKs.

Device Support#

• Headset Compatibility: Meta Quest 2/3/Pro, Apple Vision Pro, HTC Vive (all variants), Pico 4, PlayStation VR2, and all PC VR headsets (SteamVR, OpenXR). A single OpenXR-compliant runtime handles all devices; platform-specific optimisations (ASW, ATW, reprojection) are applied automatically.
• Input Modalities: Six-DOF controller tracking, optical hand tracking (gesture recognition without controllers), eye tracking (gaze input and foveated rendering), full-body tracking (optional tracker hardware), and haptic glove support.
• Future Input (@maya/future-input, Phase 40.28): An emerging-input abstraction (libs/maya/future-input) covering consumer brain-computer interfaces (EEG-based intent/attention signals with BCI calibration and accessibility features), EMG wristband neural input, and other post-controller modalities, normalized into the same input-action layer as controllers and hands so experiences adopt new devices without rewrites.
• AR/MR Pass-Through: Mixed reality pass-through support for Quest 3 and Apple Vision Pro, enabling AR content anchored to physical room geometry reconstructed from the headset's spatial mapping sensors.

Haptic Integration#

The haptic system federates across multiple body-suit and glove vendors through a single haptic bus. A physics event anywhere on the avatar drives the correct actuator set on whatever haptic hardware the player happens to be wearing.

• Multi-Device Haptic Feedback: bHaptics TactSuit (full-body tactile feedback), bHaptics TactGlove, Teslasuit, HaptX Gloves, and standard controller rumble are all supported through a unified haptic bus. A physics event (taking damage in the left shoulder) propagates to the correct actuator set on whichever device combination the player is using.
• Audio-to-Haptic Conversion: Music and ambient audio drive haptic patterns in real time, turning sound into physical sensation. This is used to create immersive environments (bass from nearby explosions felt in the chest) and rhythm game mechanics.
• Social Touch: Avatar-to-avatar physical contact (handshake, high five, hug) drives haptic feedback on both participants using the physical interaction model, making social contact feel meaningful.

Comfort and Accessibility#

• Comfort Locomotion Options: Teleportation with arc preview, smooth locomotion with vignette comfort filter, room-scale physical movement, snap rotation (configurable angle increments), and vehicle-based travel (seated in a vehicle that moves through the world).
• Accessibility Modes: One-handed controller schemes, voice command control, high-contrast and colourblind visual modes, motion-sensitivity scaling (reduce camera sway), seated mode with height calibration, and visual impairment support (magnification, auditory description).

Spatial AR and AR Cloud — (planned)#

No spatial-ar library exists, and maya-immersion has no AR-cloud, passthrough, plane-detection, or world-meshing modules. The capabilities below are a roadmap.

• AR Cloud Anchor Persistence: Digital objects placed in the real world persist between sessions via cloud-anchored spatial maps. Multiple users sharing the same physical space see the same AR content at the same physical position — enabling shared AR experiences.
• World-Scale Meshing: Continuous room-scale and building-scale 3D mesh reconstruction from device sensors, enabling AR content that correctly occludes behind real furniture, walls, and floors.
• Plane Detection and Semantic Segmentation: Automatic detection of flat surfaces (floor, table, wall) and semantic labelling of real-world object types, enabling physics-aware AR content that sits on tables and avoids walls.

User-Generated Content and Creator Tools — (planned)#

There is no forge library on disk. The in-world building, visual scripting, AI-assisted creation, and asset-pipeline features below are a roadmap. The closest shipped code is @maya/tooling (a desktop world-editor and asset-tool library — see Development Tooling) and the @maya/client mod-and-plugin-loading module; an in-VR builder and a UGC scripting sandbox are (planned).

In-World Building Tools#

• VR Building Interface: Intuitive grab-and-place building in VR using block/voxel primitives, CSG operations (union, subtract, intersect), and procedural assist (snap to grid, align to surface, proportional scaling). Non-VR users access an equivalent desktop interface with gizmo-based manipulation.
• 3D Modelling and Sculpting: Polygon modelling, freeform sculpting, and UV layout tools available inside the world editor. Assets can be authored directly in Maya without external DCC software, lowering the skill floor for creators.
• Visual Logic Programming: A node-based visual scripting system exposes game events (player enters zone, NPC is killed, time reaches sunset) and actions (play animation, spawn object, change material, send network message) as connectable nodes. Non-programmers can build complex interactive experiences without writing code.
• TypeScript Scripting API: Advanced creators write TypeScript/JavaScript against a documented API surface. Scripts run in a sandboxed WebAssembly environment with capability-gated access to engine APIs.

AI-Assisted Creation#

• Text-to-3D Mesh: Natural language descriptions ("a mossy stone archway overgrown with vines") produce 3D geometry within seconds using generative AI integrated with the Isis domain. The result is placed in the scene and can be further refined by the creator.
• Text-to-Texture: Describe a surface look in words and receive a seamlessly tileable PBR texture set (albedo, normal, roughness, metallic) generated by AI. Works for materials not easily photographed — alien alloys, magical inks, impossible architectures.
• AI World Generation: Describe a world concept in a paragraph and receive a complete biome, city, and population configuration as a starting point. The creator then curates and extends the AI's proposal.
• AI NPC Personality Creation: Define an NPC with a few paragraphs of backstory and receive a fully configured personality card, dialogue examples, and behavioural trait parameters ready for the souls system.

Asset Pipeline#

• Import/Export Standards: FBX, glTF, USD, and Alembic assets from external DCC tools (Blender, Maya DCC, Houdini, Cinema 4D) import cleanly with materials mapped to Maya's PBR model. Exports carry attribution metadata.
• Photogrammetry Integration: RealityScan and Polycam outputs (3D Gaussian Splats, mesh + photogrammetry texture) import directly. The pipeline auto-optimises for Nanite streaming and generates runtime LODs.
• Marketplace and Attribution: A built-in asset marketplace enables creators to sell, license, and share assets with configurable royalty terms. Attribution is tracked on every remixed or derivative asset.

Modding and Governance Framework — (planned)#

Status update (2026-06-12): four skeleton crates now exist on disk — libs/maya/forge-resolver, forge-conflict, forge-sandbox, and forge-compositor landed 2026-06-01 (commit 62045b89b3) as V7 Ixchel work (see V7/V7_features.md § Ixchel). They are small, honest API skeletons (the resolver ships a deterministic exact-manifest resolver and lock-file API; full PubGrub is still future work). There is still no agora, variants, bazaar, loom, or crucible library anywhere under libs/ — those remain design-vision only. The original note below is retained for context; read it as "design vision, now partially skeleton-implemented under V7":

No modding or governance code exists on disk (superseded 2026-06-01 for the four forge-* crates, accurate for the rest). There is no forge, agora, variants, bazaar, loom, or crucible library under libs/maya/. This entire section is a design vision for a future unified modding, governance, game-forking, and world-expression platform. The package names in parentheses below are proposed, not shipped.

Universal Modding Framework (Maya Forge)#

• 12 Moddable Domains: Every aspect of a game is independently moddable — assets, code (WASM-sandboxed), rules (algebraic composition), worlds (spatial partitioning), UI layout, narrative graph, physics zones, AI behaviour trees, audio mix buses, economy parameters, social features, and total conversions. Mods are composable layers, not load-order-dependent file overrides.
• Hot-Reload Without Restart: Mods can be injected into a running game session in both single-player and multiplayer without restarting the client or server. The mod lifecycle manager handles resource cleanup and state preservation across hot-reloads.
• SAT-Based Dependency Resolution (@maya/forge-resolver): A PubGrub-variant SAT solver resolves mod dependency graphs with SemVer constraints, optional dependencies, feature gates, and capability-based alternatives. Resolution produces a lock file for reproducible mod setups. When resolution fails, human-readable explanations describe exactly which requirement conflicts caused the failure.
• Semantic Conflict Resolution (@maya/forge-conflict): Instead of the fragile "load order" model used by every competitor, Maya's conflict engine analyses the intent of each mod's changes. Rules mods compose algebraically (additive, multiplicative, min/max); world mods use spatial partitioning; code mods use hook ordering with disjoint-state detection; audio mods get independent mix buses. Unresolvable true contradictions are surfaced to the community for democratic resolution.
• WASM Security Sandbox (@maya/forge-sandbox): Six sandbox tiers (DataOnly, Scripted, Extended, System, Native, Trusted) grant progressively wider engine API access. Each tier enforces per-mod CPU, memory, disk, and GPU time budgets per frame with graceful throttling on overrun. Audit logs capture all API calls for debugging and abuse detection.
• Layer Composition Engine (@maya/forge-compositor): Builds the final runtime game state by applying all active mod layers in priority order (Engine < BaseGame < ContentPack < ServerRealm < CommunityVariant < PersonalOverride), tracking per-field provenance (which mod contributed each value) for debugging.

Democratic Game Governance (Maya Agora)#

• On-Chain Proposal and Voting: Community members can propose rule changes, balance adjustments, or content additions via an on-chain governance system (Aje domain integration). Proposals go through a structured lifecycle: drafting, deliberation period, voting, execution, and appeal.
• 10+ Voting Models: Token-weighted, reputation-based, quadratic (square root of tokens, favouring broader participation), conviction voting (lock tokens for extended commitment to build signal), delegated (assign your vote to a trusted community member), time-weighted (rewards long-term holders), and more.
• Multi-Tier Governance: Governance operates at five levels — platform-wide, per-game, per-realm (server shard), per-server, and per-guild. Each tier has its own sovereign rule space that cannot be overridden by parent governance without community consent.

Git-Like Game Forking (Maya Variants)#

• Fork Any Game: Any game in the Maya ecosystem can be forked — the creator receives a full copy of the game's base layer and can diverge it into a new variant. The variant publishes as a standalone experience that retains attribution and revenue-sharing links to the original.
• Community Canonical Votes: When multiple variants of a game exist, the community votes on which variant represents the canonical "main branch." Popular variants can overtake the original in canonical status.
• Cross-Variant Asset Sharing: A variant can declare that it consumes assets from the parent game's marketplace, so creators of the original benefit economically from popular forks.

Mod Economy (Maya Bazaar)#

• Creator Monetisation: Mod creators set pricing (free, one-time purchase, subscription, usage-based) with platform revenue sharing. Dependency revenue chains automatically flow a portion of a mod's earnings to the creators of its dependencies.
• AI Balance Verification (Maya Crucible): Every mod combination is automatically stress-tested by fleets of AI agents that play out corner cases, discover exploits, and verify economic balance before and after deployment. Balance is a continuously monitored emergent property, not a static analysis hope.

Infinite World Expression (Maya Loom)#

• World Genomes: Users define "world genomes" — parameterised procedural generation configurations that specify physics constants, biome algorithms, dimension rules, and narrative constraints — as shareable mods. Applying a genome to a base game produces an entirely different experience without forking the code.
• Runtime Expression Layering: World expression layers apply on top of the running game without modifying the base game's world data. A player can apply an "eternal winter" expression layer and experience the same world with pervasive snow without affecting other players on the same server (unless the server owner enables it).

AI Modding Companion — (planned)#

No forge-companion library exists on disk. This section describes a planned voice-interactive, AI-powered modding companion with an expressive animated avatar and a transparent "Glass Workshop" spatial interface — a design vision, not shipped code.

Voice Conversation Engine#

• Streaming ASR with Game-Audio Noise Filtering: Real-time speech recognition using streaming providers (OpenAI Whisper, AssemblyAI, Deepgram) with a Web Audio API worklet that isolates voice from game sound effects, music, and ambient audio using spectral gating and voice activity detection. First partial transcript within 150 ms of speech onset.
• Spatial Voice Commands: Deictic references ("that tree", "this room", "over there") are resolved to game world coordinates using camera raycasts, gaze tracking (Tobii, Quest Pro, webcam-based estimation), and hand-pointing gesture detection. Ambiguous references trigger a clarification flow with highlighted candidates.
• Intent Classification Across 12 Mod Domains: A multi-class intent classifier maps natural language to the 12 Maya Forge mod domains at four granularity levels — novice ("make it cooler"), intermediate ("increase emission rate"), expert ("modify the Rayleigh scattering coefficient"). Compound commands are split into ordered sub-commands with dependency detection.
• Streaming TTS with Emotion Modulation: Text-to-speech with emotion annotation support — the companion's voice changes prosody (pitch, rate, emphasis) based on the emotional tag of the response. First audio byte within 200 ms.

Expressive Animated Avatar#

• FACS Facial Animation: 46 Action Units drive 250+ blend shapes with continuous emotion state (valence-arousal-dominance model). The avatar produces micro-expressions, naturalistic blinks, and saccadic eye movements that glance at the workspace when referencing content.
• Big Five Personality System: Eight personality archetypes (The Enthusiast, The Professor, The Guardian, The Trickster, The Artisan, The Commander, The Sage, The Companion) influence voice characteristics, gesture frequency and amplitude, word choice, and explanation depth. Personality subtly evolves based on player interaction patterns over time.
• Procedural Body Language: Beat gestures on stressed syllables, deictic gestures when referencing objects, 120+ motion-captured gesture clips, full-body posture states, and idle behaviours (weight shifting, breathing, subtle swaying).

Glass Workshop Transparent Spatial Interface#

• Spatial Panel Manager: Floating panels orbit, dock, and arrange in 3D space around the avatar. Panel types include live diff viewer, system dependency graph, reasoning stream, timeline scrubber, preview viewport, and console output. In VR, panels are physical 3D objects; on flat screens they render as overlay windows.
• Live Diff Viewer: Every change the AI makes appears as a real-time syntax-highlighted diff with inline natural-language annotations and per-line approve/reject controls.
• Reasoning Stream: A beautiful visualisation of the AI's decision-making steps — analysis, decision, action, and verification — rendered with characteristic animations at three verbosity levels.
• System Dependency Graph: An interactive node graph shows every game system affected by the current mod — pulsing nodes indicate active modifications, ripple animations show propagation, and dependency warnings highlight dangerous chains.

Atmospheric and Environmental Systems (maya-atmosphere crate)#

The maya-atmosphere crate (~75 source files) implements physically based sky, volumetric clouds, weather, and water surfaces. The goal is an atmosphere that responds correctly to every variable — time of day, altitude, atmospheric composition, weather state, and observer position — using the same equations that govern real atmospheric optics.

The crate's PhysicallyBasedAtmosphereScatteringSystem computes sky radiance from observer position, view direction, and sun direction.

• Physically-Based Sky: The Hillaire sky model computes sun and sky colour at any time of day and altitude as a function of atmospheric composition using real-world Rayleigh (wavelength-dependent scattering by air molecules) and Mie (larger particle scattering) equations. The same equations that make real sunsets orange at the horizon produce the same effect in Maya.
• Volumetric Clouds: A 3D noise-based cloud model with full volumetric self-shadowing, multiple scattering (the soft silver lining around clouds), and time-evolution simulation. Clouds cast volumetric god rays (light crepuscular rays) and respond to wind direction.
• Ocean Simulation (FFT Waves): Statistical ocean surface simulation using fast Fourier transform synthesis of a Philips wave spectrum. Wave height, choppiness, and swell direction are wind-driven. Deep underwater rendering includes caustic light patterns, particle turbidity, and bioluminescence.
• GPU Particle Simulation: Millions of particles simulated on the GPU with collision against terrain geometry. Used for dust storms, volcanic ash, magical effects, river spray, crowd confetti, and combat debris.
• Weather System: Precipitation (rain, snow, sleet, hail), lightning, dynamic weather fronts that move across the map, tornado formation, and blizzard ground-level visibility effects. Weather is networked: all clients in a region observe the same weather state.

Game Framework (@maya/games)#

The @maya/games TypeScript library (~100 source modules) provides reusable gameplay systems that game developers assemble into full game genres. Rather than targeting a single game type, the library provides modular building blocks organised into 19 sub-modules that can be combined: stats, progression, skills, abilities, inventory, items, equipment, crafting, gathering, survival, loot, mmo, classes, party, companions, quests, combat, and hathor.

The genre "templates" below describe how these modules combine; the implemented spine is the combat, mmo, survival, quest, party, companion, and progression module sets.

Implemented module sets:

• Open World RPG systems: the quests, inventory, items, equipment, crafting, skills, abilities, progression, and stats modules — quest definition/chain/objective/tracking with consequence and faction-reputation tracking, inventory, crafting, and skill/progression systems.
• MMO systems (games/src/mmo/): guild system with ranks/permissions, bank, and housing; dungeon instancing; raid encounters; boss mechanics; world-boss spawning; PvP and battleground/arena; ranking matchmaking; auction house; mail; achievements; seasonal content.
• Survival systems (games/src/survival/): hunger/thirst, temperature exposure, sleep/rest, disease, food/drink items, tool durability, base building and upgrades, structure-placement validation, base-defense raids, shelter mechanics, environmental hazards, resource nodes, wildlife hunting, pet taming.
• Combat systems (games/src/combat/): weapon, melee, ranged, projectile, combo-chain, blocking/parrying, dodge-roll, cover, status-effect, AI-combat-behaviour, group-tactics, difficulty-scaling, and analytics systems, on top of a health-damage and damage-calculation core.
• Party, companions, gathering, loot, classes — additional module sets for grouping, companion characters, resource gathering, loot tables, and character classes.

Racing/vehicle, dedicated FPS/TPS, fighting-game, horror, platformer, and puzzle genre frameworks, and a @maya/game-accessibility library, do not exist as @maya/games module sub-directories — those genre templates are (planned). (maya-physics does provide a VehicleConfig, and maya-renderer/maya-embodiment carry accessibility-testing tooling, but a games-layer template for those genres is not present.)

Three roadmap phases extend the games layer with planned envelopes beyond the implemented spine:

• Combat Systems SOTA (Phase 84 — planned) — a multi-genre combat framework on top of the games/src/combat/ core: a frame-data engine (startup/active/recovery frames, hit/hurt/throw boxes, cancel windows), grappling and ground-game systems, a body-damage and physiology model (per-region damage, accumulating injuries affecting capability), structure and posture combat, environmental/contextual combat, freeflow crowd combat, a dedicated 2D/3D fighting-game engine, pro-wrestling and MMA simulation rule sets, directional melee, combat animation and visual-feedback systems, and combat AI trained per genre. Aja supplies combat motion data and animation retargeting; Bellona's gameplay combat runtime and Hathor's NPC behavior layers consume the framework.

• Voice-First Gameplay Input (Phase 82) — voice as a primary game input, carried by the libs/maya/voice-* packages (voice-tactics, voice-dialogue, voice-negotiation, voice-narrative, voice-tts): an on-device ASR fast path with tactical intent classification, SLM-backed natural-language command decomposition, spatial reasoning from voice ("flank left behind the red truck"), squad tactical command execution with multi-agent formation control, combat AI that treats voice orders as override-priority inputs, personality-matched TTS acknowledgments, and wake word / push-to-talk / open-mic activation modes. Iris and Psyche supply the speech stack; Hathor supplies NPC dialogue semantics.

• Living World Evolution (Phase 83) — runtime evolution of the generated world, carried by libs/maya/urban-evolution-manager, libs/maya/world-history, and the libs/maya/npc-* packages (npc-occupations, npc-agency-orchestrator, npc-cooperation-negotiation, npc-distillation, npc-cloud-fallback): urban evolution (city growth, decay, and rebuilding over simulated time), living infrastructure systems (power, water, road networks that degrade and get repaired), procedural building interiors on demand, and NPC lifecycle and skill progression so the same characters age, learn, and change occupation across sessions. Hathor owns the NPC agency semantics (see DOMAINS/hathor/features.md).

• Narrative Systems (games/src/hathor/): the Hathor integration imports worldbuilding data into Maya's game systems via hathor-world-data-import, hathor-cultural-rules-integration, hathor-religion-belief-integration, hathor-language-naming-integration, and hathor-economic-model-integration.

Roadmap Package Surface (Phases 73–83)#

The libs/maya/ tree carries the package families for the Maya roadmap phases. Several doc sections above describe these phases as wholly planned; the packages below are present on disk (implementation depth varies from thin scaffolds to multi-module libraries), and the earlier prose should be read together with this inventory:

• VR Production Studio (Phase 73) — vr-studio, vr-studio-web, and spatial-ar: Maya's side of the immersive director's workspace whose studio envelope is described in DOMAINS/yemaya/features.md.
• Forge Studio creator suite (Phase 76.60–76.67) — forge-studio-core, -mocap, -voice, -sculpt, -vfx, -world, -cinema, -music: the integrated in-world creator studio over the Forge modding framework.
• Sentinel mod safety (Phase 76.42–76.45) — sentinel-core, sentinel-scanner, sentinel-ip, sentinel-provenance: mod security scanning, IP/copyright checks, and provenance for the mod pipeline.
• Loom world expression (Phase 76.68–76.72) — loom-core, loom-biomes, loom-dimensions, loom-evolution, loom-multiverse, plus the ML set loom-inverse (inverse procedural), loom-diffgen (differentiable generation), loom-semantic (semantic brush), loom-terrain-ml (terrain diffusion), and loom-agent (LLM-agent city/world generation).
• Crucible balance verification (Phase 76.52–76.59) — crucible-core, -agents, -adversarial, -balance, -scenarios, -regression, -live, -governance: AI playtesting agents, adversarial exploit search, balance regression, and live monitoring.
• Agora / Bazaar / Variants (Phase 76) — agora-core, -constitution, -councils, -delegation, -proposals, -sybil, -voting (democratic governance); bazaar-core, -licensing, -revenue, -treasury (mod economy); variants-core, -compat, -diff, -merge, -registry (git-like game forking).
• Nexus multiplayer SOTA (Phase 77) — instancing, lobbies, matchmaking, live-events, spectator, tournaments, ugc-runtime, anticheat, nexus-live-voting, nexus-mod-sync, server, client: session instancing, lobby/session flow, live events, spectator and replay, UGC multiplayer runtime, tournament/esports infrastructure.
• Social fabric (Phase 80) — guild-operations, guild-recruitment, lfg-community, mentorship, social-graph, social-discovery, social-events, social-facilitation, community-companion, community-federation, community-governance, community-health, community-recognition, community-reputation, trust-safety, moderation.
• NPC cognition and agency (Phases 81/83) — npc-agency-orchestrator, npc-cooperation-negotiation, npc-distillation, npc-cloud-fallback, npc-occupations, urban-evolution-manager, world-history, world-social-runtime (see Tiered NPC Cognition in DOMAINS/hathor/features.md).
• Voice gameplay (Phase 82) — voice-tactics, voice-dialogue, voice-negotiation, voice-narrative, voice-tts.
• Open-world depth (Phase 79) — ecology, animal-database, env-storytelling, ornament, jewelry, organic-materials, economy, economy-tools, politics, geospatial, neural-capture, renderer-advanced.
• Interoperability — standards (the metaverse-standards library the earlier "(planned)" note predates).

MMO Social Fabric (Phase 80)#

Community formation, belonging, and health systems that help players meet people, find their place, and remain meaningfully connected — especially for shy, isolated, or returning players. Carried by the social-fabric package family listed under Roadmap Package Surface above.

• Social Identity and Intent Graph: Each player carries a PlayerSocialProfile (relationship edges, group comfort level, activity interests, availability windows, communication preferences) and a PlayerIntentState (what they want right now: questing, crafting, chatting, teaching, exploring). Real-time presence aggregates across friends, guilds, and platform connections.
• Discovery and Serendipity Engine: A recommendation engine surfaces compatible players, guilds, events, and quest groups based on shared intent, schedule overlap, reliability signals, and complementary roles. Shy-player and low-pressure lanes (no-mic, first-timer, mentor-led, small-group) ensure discovery is non-intimidating.
• Community Group Finder (LFG): Structured listing schema (activity, objective, tone, language, voice preference, teaching/learning posture) replaces freeform LFG text. Community tags (first-timers welcome, chill run, speed clear, lore run) set expectations.
• Guild Recruitment Platform: Searchable guild profiles with culture, schedule, size, language, and newcomer friendliness. Provisional membership with trial role and onboarding checklist. In-world recruitment boards connected to live recruitment profiles.
• Mentorship Network: Explicit mentor/mentee relationship tier with skill-based matching, session scheduling, and progress documentation.
• Trust and Safety Architecture: Community health monitoring tracking guild activity patterns, social isolation signals, and harassment patterns. AI-assisted social facilitation for awkward or inexperienced players — helping compose inviting listings, offering low-pressure introductions, and providing contextual social coaching.

Inspiration Library (@maya/inspirations)#

The @maya/inspirations TypeScript library (~75 source modules) is a curated reference-data library that feeds style-consistent parameters into the procedural generation pipeline. When generating a city, a biome, or a building, the genesis crates can be seeded from these reference sets to target a specific real-world or fictional aesthetic rather than a purely random result.

A shared reference-types.ts vocabulary (CityReferenceDensity, CityReferenceGridPattern, CityReferenceTransitMode, CityReferenceLandUse, CityReferenceClimateProfile, …) types the data.

• Real-World City References: parameter sets for Manhattan, Hong Kong, Tokyo, Dubai, Singapore, European historic cities, Paris/London/Rome, and Middle Eastern, African, South American, and Southeast Asian urban patterns, with reference-photography measurements.
• Fictional Universe References: Blade Runner LA 2049, Night City, Neo Tokyo, Mega-City One, Matrix mega-city, Ghost in the Shell New Port City, Coruscant, Mass Effect Citadel, Halo New Mombasa, Final Fantasy Midgar, Game of Thrones / Lord of the Rings / Elder Scrolls cities, Mirror's Edge.
• Biome References: temperate forest, tropical rainforest, desert/dune, tundra/arctic, mountain/alpine, ocean/coastal, cave/underground, plus exotic variants — bioluminescent, crystal/mineral, magical/enchanted forest, corrupted/tainted, organic-biomechanical.
• Style Guides: original cyberpunk and fantasy style guides, plus character, vehicle/prop, audio, and UI/UX style references; tooling for color-palette extraction, style transfer, mood boards, and narrative-tone mapping.
• Yemaya Integration: a Yemaya integration test-fixture module (yemaya-integration-test-fixtures.ts) defines the cross-reference surface toward Yemaya's creative databases. This fixture establishes the data-handoff shape for when full Yemaya integration is built out, so Maya can consume creative-asset databases from Yemaya's domain.

Metaverse Interoperability Standards#

A libs/maya/standards library now exists on disk carrying the domain-wide standards surface (maya-immersion also carries an openxr_runtime_integration module). Depth of each commitment below varies; treat the bullets as the capability envelope the library implements.

• OpenUSD Integration: Pixar's Universal Scene Description (USD) is the emerging metaverse interchange standard. Maya supports USD as a first-class asset format — export worlds as USD stages, import USD assets from DCC tools and other platforms, and participate in the emerging USD-based metaverse pipeline.
• Khronos glTF 2.0 and Extensions: glTF is the web-native 3D format. Maya's asset pipeline exports fully compliant glTF 2.0 with custom extensions for Maya-specific data (physics properties, AI personality, interaction scripts). Avatar appearance data in glTF is readable by any standards-compliant viewer.
• OpenXR Input Abstraction: OpenXR standardises VR input across all headsets and runtimes. Maya's input system is built on OpenXR action bindings — the same game works on Quest, SteamVR, and Apple Vision Pro without controller-specific code.
• W3C WebXR Device API: Web-based Maya experiences use the WebXR Device API for VR/AR access in the browser, ensuring compatibility with any WebXR-capable browser and reducing deployment friction.

Platform Infrastructure#

Client SDK (@maya/client)#

The @maya/client TypeScript library (~30 source modules) is the entry point for all client-side platform concerns. It abstracts away platform differences so that game code written against this layer runs on desktop, mobile, browser, and VR headsets without modification. The library covers:

• A platform abstraction layer (MayaClientPlatformCapability, MayaClientWindowState, MayaClientXrSessionState).
• Per-platform build modules for Windows, macOS, Linux, Android, iOS, WebGPU/WebXR browser, Meta Quest standalone, SteamVR, and Apple Vision Pro.
• Store distribution and CI/CD modules for all major storefronts and automatic updates.
• Runtime modules for asset loading/caching, input handling, window/display management, configuration, settings persistence, offline support, mod/plugin loading, client-side validation, performance profiling, debugging, and crash reporting.

Server Infrastructure (@maya/server)#

The @maya/server TypeScript Node.js library (~27 service modules) drives the game server. Its central piece is WorldSimulationLoop — a fixed-timestep loop with spiral-of-death prevention — surrounded by the full range of live-service modules: world simulation loop, player connection management, sessions, anti-cheat, server-side physics, AI-NPC simulation, economy transactions, event broadcasting, server clustering, zone handoff, metrics, administration, hot-reload, social graph, matchmaking, achievements, leaderboards, content moderation, asset storage, analytics, user management, inventory, payment processing, and email integration. Kubernetes/Helm packaging and an OpenTelemetry/Prometheus/Grafana observability stack are deployment concerns, not present in source, and remain (planned).

Data Persistence (@maya/database)#

The @maya/database TypeScript library (~30 source modules) provides all domain schemas, validators, SQL DDL generators, and data-access infrastructure. It covers schemas for: users, avatars, inventory items, worlds/zones, buildings/objects, NPC AI state, quests/progression, social relationships, economy transactions, analytics, moderation, asset metadata, permissions, and audit logs. The data-access layer includes MayaRepository, query builders, a transaction manager, a Redis-style cache layer, a read-replica router, and a shard router. Operations and compliance modules handle archival, backup/restore, point-in-time recovery, export, GDPR, anonymization, retention, monitoring, and query optimization.

The library has no Drizzle ORM dependency; schemas are plain TypeScript with generate*TablesDDL() functions.

Development Tooling (@maya/tooling)#

The @maya/tooling TypeScript library (~48 source modules) is a desktop tooling suite in three groups. The world editor covers viewport navigation, selection/transform tools, property inspector, hierarchy outliner, asset browser, terrain/foliage/lighting/material/particle/animation/audio editors, play-in-editor mode, and collaborative editing. Profiling and debugging covers frame/GPU/memory/network profilers, physics/audio/AI/render debuggers, console logging, watch/breakpoint systems, replay time-travel debug, crash-dump analysis, telemetry, and performance-regression testing. Asset tools cover converter CLI, texture compression, mesh optimization, LOD/collision/navmesh generation, lightmap baking, audio processing, validation, comparison, dependency analysis, size reporting, migration, batch processing, and localization.

Testing Infrastructure (@maya/testing)#

The @maya/testing TypeScript library (~48 source modules) is a test-harness suite in three groups. Unit testing includes per-subsystem test modules for engine-core, renderer, physics, audio, networking, AI, economy, procedural generation, serialization, math, data structures, API contracts, coverage, and mutation testing. Integration testing covers client-server, database, auth-flow, multiplayer, economy, UGC upload/download, world streaming, save/load, cross-platform, VR-device, performance-benchmark, e2e, chaos engineering, and load testing. Visual testing covers screenshot comparison, render-regression, perceptual diff, cross-platform visual parity, HDR tone mapping, post-processing, particle, material, shader, lighting, animation, VR-stereo, UI screenshot tests, automated visual QA, and visual-test reporting.

Documentation (@maya/documentation)#

The @maya/documentation TypeScript library (~30 source modules) — its own libs/maya/documentation library, not a tooling subsystem — covers API documentation (TypeDoc generation, API-reference website, code-example embedding, interactive API explorer, versioned docs, changelog generation, deprecation tracking, migration guides, search, doc testing, localization, PDF export, offline docs, API-diff tools, documentation CI/CD) and tutorials and guides (getting-started, first-world, avatar-customization, building-basics, scripting-introduction, multiplayer, VR-development, AI-NPC, procedural- generation, asset-pipeline, economy-setup, optimization, deployment, best-practices, and video-tutorial-series modules).

Cross-Domain Platform Integration — (planned)#

No imports of Neith, Nous, or other platform-domain packages were found in Maya source. The integrations below are roadmap items; nothing in this section describes shipped Maya code.

Bellona owns engine export bridges, Hathor owns narrative simulation, Isis owns asset generation, and Themis owns governance/IP primitives; Maya documents the runtime and world-creation features that integrate those services.

• Neith game-platform substrate (planned) — the roadmap positions Maya as a consumer of a Neith platform layer for platform HAL, console/mobile/ Steam targets, cross-platform identity, cloud-gaming encode/transport, fleet orchestration, online subsystem (matchmaking, lobbies, parties, dedicated fleets, replication, anti-cheat, liveops), a creator marketplace, a spatial runtime, and an engine audio/scripting parity layer.
• Nous neural world model (planned) — the roadmap positions Maya as a consumer of Nous world-model packages for text-to-interactive-world sessions, persistent video worlds, action-conditioned simulators, shared memory banks, multiplayer joins, and world-model evaluation.