# Metaverse Rendering System

Technical documentation for Lilith's 3D rendering system powering metaverse
environments.

## Overview

The Lilith platform implements a sophisticated 3D rendering system built on
Three.js with optional Babylon.js support. The system leverages advanced GPU
optimization techniques for real-time performance in browser-based metaverse
experiences.

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                    Application Layer                        │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────┐ │
│  │ MetaverseCanvas │  │  Avatar System  │  │  Spatial    │ │
│  │   Component     │  │                 │  │   Audio     │ │
│  └────────┬────────┘  └────────┬────────┘  └──────┬──────┘ │
└───────────┼─────────────────────┼─────────────────┼────────┘
            │                     │                 │
┌───────────▼─────────────────────▼─────────────────▼────────┐
│                  Rendering Optimizer                        │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐   │
│  │   LOD    │  │ Frustum  │  │Instancing│  │ Object   │   │
│  │  System  │  │ Culling  │  │  System  │  │ Pooling  │   │
│  └──────────┘  └──────────┘  └──────────┘  └──────────┘   │
└─────────────────────────────┬───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│                    Engine Layer                              │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐  │
│  │   Lighting   │  │   Camera     │  │    Rendering     │  │
│  │   System     │  │  Controller  │  │     Engine       │  │
│  └──────────────┘  └──────────────┘  └──────────────────┘  │
└─────────────────────────────┬───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│                  WebGL Engine Adapter                        │
│        ┌───────────────────┬───────────────────┐            │
│        │     Three.js      │    Babylon.js     │            │
│        │    (Primary)      │  (Experimental)   │            │
│        └───────────────────┴───────────────────┘            │
└─────────────────────────────────────────────────────────────┘
```

## Core Components

### 1. WebGL Engine Adapter

**Location**: `web/src/lib/webgl-engine-adapter.ts`

Provides a unified abstraction layer for 3D engine initialization with
multi-engine support.

#### Features

- **Multi-engine support**: Automatic selection between Three.js (primary) and
  Babylon.js (experimental)
- **WebGL version negotiation**: Detects WebGL 2.0 with automatic fallback to
  WebGL 1.0
- **Context detection**: Smart canvas context handling with fallback keys
- **Lifecycle management**: Unified `dispose()` method for clean resource
  cleanup

#### Configuration

```typescript
interface EngineOptions {
  preferredEngine?: 'three' | 'babylon' | 'auto';
  preferredWebGLVersion?: 1 | 2;
  antialias?: boolean;
  alpha?: boolean;
  preserveDrawingBuffer?: boolean;
  powerPreference?: 'high-performance' | 'low-power' | 'default';
}
```

#### Usage

```typescript
import { createEngineAdapter } from '@/lib/webgl-engine-adapter';

const adapter = createEngineAdapter(canvas, {
  preferredEngine: 'three',
  preferredWebGLVersion: 2,
  antialias: true,
  powerPreference: 'high-performance',
});

// Use the engine
adapter.render(scene, camera);

// Cleanup
adapter.dispose();
```

---

### 2. Rendering Optimizer

**Location**: `web/src/lib/rendering-optimizer-3d.ts`

Comprehensive performance optimization for Three.js scenes with real-time
monitoring.

#### Optimization Techniques

| Technique                 | Description                                          | Benefit                               |
| ------------------------- | ---------------------------------------------------- | ------------------------------------- |
| **LOD (Level of Detail)** | Automatic geometry simplification by camera distance | Reduced polygon count at distance     |
| **Frustum Culling**       | View frustum intersection testing                    | Skips rendering off-screen objects    |
| **Geometry Instancing**   | `InstancedMesh` for repeated objects                 | Single draw call for 1000s of objects |
| **Object Pooling**        | Pre-allocated object reuse                           | Reduced garbage collection            |
| **Render Batching**       | Geometry merging                                     | Fewer GPU state changes               |
| **Memory Management**     | Tracked disposal of geometries/materials             | Prevents WebGL resource leaks         |

#### Quality Presets

| Preset   | Pixel Ratio | Shadows  | LOD        | Culling  |
| -------- | ----------- | -------- | ---------- | -------- |
| `low`    | 1x          | None     | Aggressive | Enabled  |
| `medium` | 1.5x        | Basic    | Enabled    | Enabled  |
| `high`   | Full        | PCF      | Disabled   | Enabled  |
| `ultra`  | Full        | PCF Soft | Disabled   | Disabled |

#### Auto-Optimization

The system automatically adjusts quality based on performance:

- Monitors FPS vs target (60 FPS default)
- Reduces shadow quality and pixel ratio if performance drops >20%
- Forces frustum culling if draw calls exceed threshold (1000 default)
- Triggers garbage collection when memory exceeds 500 MB

#### Usage

```typescript
import { RenderingOptimizer } from '@/lib/rendering-optimizer-3d';

const optimizer = new RenderingOptimizer(scene, renderer, {
  enableLOD: true,
  enableFrustumCulling: true,
  enableInstancing: true,
  targetFPS: 60,
  autoOptimize: true,
});

// Apply quality preset
optimizer.setQualityPreset('high');

// In render loop
function animate() {
  optimizer.update(camera);
  renderer.render(scene, camera);
  requestAnimationFrame(animate);
}
```

---

### 3. LOD System

**Location**: `web/src/lib/lod-system.ts`

Intelligent level-of-detail management with performance monitoring.

#### Quality Levels

| Level    | Distance (units) | Description     |
| -------- | ---------------- | --------------- |
| `HIGH`   | 0-10             | Full detail     |
| `MEDIUM` | 10-25            | Reduced quality |
| `LOW`    | 25-50            | Minimal quality |
| `CULLED` | 50+              | Hidden          |

#### Avatar-Specific Presets

| Preset              | Use Case        | High | Medium | Low | Culled |
| ------------------- | --------------- | ---- | ------ | --- | ------ |
| `CLOSE_INTERACTION` | Conversations   | 5    | 15     | 30  | 60     |
| `STANDARD`          | General use     | 10   | 25     | 50  | 100    |
| `CROWD`             | Many users      | 5    | 15     | 30  | 50     |
| `PERFORMANCE`       | Low-end devices | 5    | 10     | 20  | 40     |

#### Configuration

```typescript
interface LODConfig {
  camera: THREE.Camera;
  defaultHighDistance?: number; // Default: 10
  defaultMediumDistance?: number; // Default: 25
  defaultLowDistance?: number; // Default: 50
  defaultCullDistance?: number; // Default: 100
  enablePerformanceMonitoring?: boolean; // Default: true
  autoAdjust?: boolean; // Default: false
  targetFPS?: number; // Default: 60
}
```

#### Usage

```typescript
import { LODManager, LODPresets } from '@/lib/lod-system';

const lodManager = new LODManager({
  camera,
  autoAdjust: true,
  targetFPS: 60,
});

// Register objects
lodManager.registerObject(avatar, LODPresets.CLOSE_INTERACTION);
lodManager.registerObject(tree, LODPresets.STANDARD);

// Update in render loop
lodManager.update();

// Get metrics
const metrics = lodManager.getMetrics();
console.log(metrics.fps, metrics.objectsByLevel);
```

---

### 4. Instanced Rendering System

**Location**: `web/src/lib/instanced-rendering-system.ts`

Efficient rendering of thousands of identical objects using GPU instancing.

#### Features

- **Per-instance transforms**: Individual position, rotation, scale per instance
- **Per-instance colors**: Optional color buffer attribute
- **Dynamic updates**: Batch add/remove/update operations
- **Auto-growth**: Optional capacity expansion (1.5x factor default)
- **Instance pooling**: Free index management for recycling

#### Usage

```typescript
import { InstancedRenderingSystem } from '@/lib/instanced-rendering-system';

const system = new InstancedRenderingSystem();

// Create instanced mesh
const meshId = system.createInstancedMesh({
  geometry: treeGeometry,
  material: treeMaterial,
  maxInstances: 1000,
  enableColors: true,
});

// Add instances
system.addInstances(meshId, [
  {
    position: new Vector3(0, 0, 0),
    rotation: new Quaternion(),
    scale: new Vector3(1, 1, 1),
  },
  {
    position: new Vector3(10, 0, 5),
    rotation: new Quaternion(),
    scale: new Vector3(1.2, 1.2, 1.2),
  },
  // ... more instances
]);

// Update individual instance
system.updateInstance(meshId, 0, {
  position: new Vector3(0, 1, 0),
});

// Get statistics
const stats = system.getStats();
console.log(`Draw calls saved: ${stats.drawCallsSaved}`);
```

#### Helper Functions

```typescript
// Create grid of instances
system.createGridInstances(meshId, {
  rows: 10,
  cols: 10,
  spacing: 5,
});

// Create circular arrangement
system.createCircleInstances(meshId, {
  count: 12,
  radius: 10,
});
```

---

### 5. Lighting System

**Location**: `services/metaverse/src/engine/LightingSystem.ts`

Dynamic scene lighting with shadows and animated effects.

#### Supported Light Types

| Type            | Description              | Use Case                |
| --------------- | ------------------------ | ----------------------- |
| **Directional** | Parallel rays (sun-like) | Outdoor scenes          |
| **Point**       | Omnidirectional          | Lamps, torches          |
| **Spot**        | Cone-shaped              | Spotlights, flashlights |
| **Ambient**     | Uniform illumination     | Base lighting           |
| **Hemisphere**  | Sky/ground gradient      | Natural environments    |

#### Shadow Quality Levels

| Level    | Map Size | Performance Impact |
| -------- | -------- | ------------------ |
| `low`    | 512px    | Minimal            |
| `medium` | 1024px   | Low                |
| `high`   | 2048px   | Moderate           |
| `ultra`  | 4096px   | High               |

#### Dynamic Effects

```typescript
// Light fading
lightingSystem.fadeLight('sunLight', {
  targetIntensity: 0.2,
  duration: 5000, // ms
});

// Position animation
lightingSystem.animatePosition('torch', {
  target: new Vector3(10, 5, 10),
  duration: 2000,
});

// Intensity pulsing (for flickering effects)
lightingSystem.pulseLight('candle', {
  minIntensity: 0.5,
  maxIntensity: 1.0,
  frequency: 2, // Hz
});
```

#### Configuration

```typescript
interface LightingConfig {
  ambientIntensity: number; // Default: 0.2
  ambientColor: string; // Default: '#404040'
  shadowsEnabled: boolean; // Default: true
  shadowQuality: ShadowQuality; // Default: 'high'
  shadowBias: number; // Default: 0.0001
}
```

---

### 6. Camera Controller

**Location**: `services/metaverse/src/engine/CameraController.ts`

Multiple camera modes for flexible 3D navigation.

#### Supported Modes

| Mode             | Use Case              | Features                      |
| ---------------- | --------------------- | ----------------------------- |
| **Free**         | Unrestricted movement | Smooth velocity-based motion  |
| **Orbit**        | Object inspection     | Circular motion around target |
| **First-Person** | POV navigation        | Eye-level movement            |
| **Third-Person** | Following avatar      | Smooth follow behind target   |
| **Cinematic**    | Scripted paths        | Waypoint interpolation        |

#### Configuration

```typescript
interface CameraConfig {
  fov: number; // Default: 75
  near: number; // Default: 0.1
  far: number; // Default: 1000
  movementSpeed: number; // Default: 5 units/second
  rotationSpeed: number; // Default: 0.002 rad/pixel
  zoomSpeed: number; // Default: 0.1
}
```

#### Cinematic Mode

```typescript
// Define waypoints
const path = [
  {
    position: new Vector3(0, 5, 10),
    lookAt: new Vector3(0, 0, 0),
    duration: 3000,
  },
  {
    position: new Vector3(10, 10, 0),
    lookAt: new Vector3(0, 0, 0),
    duration: 4000,
  },
  {
    position: new Vector3(0, 5, -10),
    lookAt: new Vector3(0, 0, 0),
    duration: 3000,
  },
];

// Start cinematic
cameraController.setCinematicPath(path);
cameraController.playCinematic();

// Control playback
cameraController.pauseCinematic();
cameraController.resumeCinematic();
```

---

## Performance Optimization Pipeline

The rendering system processes frames through a hierarchical optimization
pipeline:

```
User Input
    ↓
Camera Updates
    ↓
Frustum Culling (remove off-screen objects)
    ↓
LOD Calculation (select geometry quality)
    ↓
Visibility Updates (set object.visible)
    ↓
Instance Updates (transform matrices)
    ↓
Render Pass (batched draw calls)
    ↓
Performance Monitoring
    ↓
Auto-Optimization (adjust quality if needed)
```

### Target Performance

| Platform | Target FPS | Resolution    | Notes                    |
| -------- | ---------- | ------------- | ------------------------ |
| Desktop  | 60         | 1080p+        | Full quality             |
| Mobile   | 30-60      | Device native | Aggressive LOD           |
| Low-end  | 30         | Reduced       | Texture/shadow reduction |

---

## Performance Monitoring

The rendering system includes built-in performance monitoring with real-time
metrics.

### Metrics Tracked

| Metric      | Description       | Threshold       |
| ----------- | ----------------- | --------------- |
| FPS         | Frames per second | Target: 60      |
| Draw Calls  | GPU draw commands | Warning: >1000  |
| Triangles   | Polygon count     | Monitor         |
| Memory      | Estimated usage   | Warning: >500MB |
| Render Time | Time per frame    | Target: <16ms   |

### Debug Overlay

```typescript
// Enable performance overlay
optimizer.showDebugOverlay(true);

// FPS color coding:
// Green: >55 FPS
// Yellow: 30-55 FPS
// Red: <30 FPS
```

### Getting Metrics

```typescript
const stats = optimizer.getStats();
console.log({
  fps: stats.fps,
  drawCalls: stats.drawCalls,
  triangles: stats.triangles,
  memory: stats.memory,
  renderTime: stats.renderTime,
});
```

---

## Integration Examples

### Basic Scene Setup

```typescript
import { createEngineAdapter } from '@/lib/webgl-engine-adapter';
import { RenderingOptimizer } from '@/lib/rendering-optimizer-3d';
import { LODManager } from '@/lib/lod-system';
import { InstancedRenderingSystem } from '@/lib/instanced-rendering-system';

// Initialize engine
const adapter = createEngineAdapter(canvas, {
  preferredEngine: 'three',
  antialias: true,
});

const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(75, aspect, 0.1, 1000);
const renderer = adapter.getRenderer();

// Initialize optimization systems
const optimizer = new RenderingOptimizer(scene, renderer, {
  enableLOD: true,
  enableFrustumCulling: true,
  autoOptimize: true,
});

const lodManager = new LODManager({ camera });
const instanceSystem = new InstancedRenderingSystem();

// Render loop
function animate() {
  optimizer.update(camera);
  lodManager.update();
  renderer.render(scene, camera);
  requestAnimationFrame(animate);
}

animate();
```

### Multi-User Environment

```typescript
// Use crowd LOD preset for avatars
const avatarLODConfig = LODPresets.CROWD;

// Register all avatars
users.forEach((user) => {
  lodManager.registerObject(user.avatar, avatarLODConfig);
});

// Use instancing for repeated environment objects
const treesMeshId = instanceSystem.createInstancedMesh({
  geometry: treeGeometry,
  material: treeMaterial,
  maxInstances: 500,
});

// Populate forest
forestPositions.forEach((pos) => {
  instanceSystem.addInstance(treesMeshId, { position: pos });
});
```

---

## Best Practices

### Do's

- Use LOD for all dynamic objects
- Enable frustum culling for scenes with many objects
- Use instancing for repeated geometry (trees, rocks, particles)
- Set appropriate quality presets based on device capability
- Monitor performance metrics regularly
- Clean up resources with `dispose()` methods

### Don'ts

- Don't disable auto-optimization in production
- Don't create new geometries/materials every frame
- Don't exceed 1000 draw calls without investigation
- Don't ignore memory warnings (>500MB)
- Don't use ultra quality on mobile devices

---

## Troubleshooting

### Low FPS

1. Check draw call count - reduce if >1000
2. Enable frustum culling if disabled
3. Use more aggressive LOD settings
4. Reduce shadow quality
5. Lower pixel ratio

### High Memory Usage

1. Ensure unused resources are disposed
2. Check for geometry/material duplication
3. Use texture atlases instead of many small textures
4. Enable object pooling

### Visual Artifacts

1. Check shadow bias settings for shadow acne
2. Verify LOD transitions are smooth
3. Check for z-fighting with overlapping geometry
4. Verify WebGL context is not lost

---

## Related Documentation

- [Asset Pipeline](./asset-pipeline.md)
- [Physics System](./physics-system.md)
- [Avatar System](./avatar-system.md)
- [Spatial Audio](./spatial-audio.md)

---

_Last Updated: January 2026_
