# Metaverse Asset Pipeline

Technical documentation for Lilith's 3D asset loading, caching, and optimization
system.

## Overview

The Lilith platform implements a comprehensive asset pipeline for loading,
managing, and optimizing 3D assets in metaverse environments. The system
supports intelligent prefetching, multi-level caching, and automatic
optimization for optimal performance.

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                    Asset Request                             │
└─────────────────────────────┬───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│                  Predictive Loader                           │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐  │
│  │ Priority     │  │  Concurrent  │  │   Idle Callback  │  │
│  │   Queue      │  │   Prefetch   │  │    Background    │  │
│  └──────────────┘  └──────────────┘  └──────────────────┘  │
└─────────────────────────────┬───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│                   CDN Resolution                             │
│        ┌───────────────────┬───────────────────┐            │
│        │   Primary CDN     │   Fallback CDN    │            │
│        └───────────────────┴───────────────────┘            │
└─────────────────────────────┬───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│                    Asset Loading                             │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐  │
│  │  GLTFLoader  │  │   Texture    │  │    Audio         │  │
│  │   (Models)   │  │   Loader     │  │    Loader        │  │
│  └──────────────┘  └──────────────┘  └──────────────────┘  │
└─────────────────────────────┬───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│                 Cache & Optimization                         │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐  │
│  │  Memory      │  │   Service    │  │    LOD           │  │
│  │   Cache      │  │   Worker     │  │  Assignment      │  │
│  └──────────────┘  └──────────────┘  └──────────────────┘  │
└─────────────────────────────────────────────────────────────┘
```

## Supported Asset Formats

| Format            | Purpose             | Loader        | Notes                        |
| ----------------- | ------------------- | ------------- | ---------------------------- |
| `.glb`            | 3D Models           | GLTFLoader    | Binary GLTF (recommended)    |
| `.gltf`           | 3D Models           | GLTFLoader    | JSON GLTF with external refs |
| `.json`           | Configuration       | Fetch API     | Metadata, settings           |
| `.png/.jpg/.webp` | Textures            | TextureLoader | Image textures               |
| `.ktx2`           | Compressed Textures | KTX2Loader    | GPU-compressed               |
| `.mp3/.ogg/.wav`  | Audio               | AudioLoader   | Spatial audio                |
| `.bin`            | Binary Data         | Fetch API     | Raw data files               |

## Core Components

### 1. Predictive Loader

**Location**: `web/src/lib/predictive-loader.ts`

Intelligent prefetching system for anticipating and loading assets before
they're needed.

#### Features

- Priority-based queue management
- Configurable concurrent prefetch limit (default: 3)
- Background loading using `requestIdleCallback`
- Timeout protection (2-second fallback)
- Primary/fallback CDN URL resolution

#### Configuration

```typescript
interface PredictiveLoaderConfig {
  maxConcurrentPrefetches?: number; // Default: 3
  idleTimeout?: number; // Default: 2000ms
  enableFallback?: boolean; // Default: true
}
```

#### Usage

```typescript
import { PredictiveLoader } from '@/lib/predictive-loader';

const loader = new PredictiveLoader({
  maxConcurrentPrefetches: 5,
});

// Prefetch with priority
loader.prefetch('/assets/hero-model.glb', {
  priority: 'high',
  type: 'model',
});

// Prefetch multiple assets
loader.prefetchBatch([
  { url: '/assets/texture1.png', type: 'image' },
  { url: '/assets/texture2.png', type: 'image' },
  { url: '/assets/ambient.mp3', type: 'audio' },
]);

// Cancel pending prefetch
loader.cancel('/assets/hero-model.glb');
```

---

### 2. CDN Integration

**Location**: `web/src/lib/cdn.ts`

Manages asset URL resolution with automatic fallback support.

#### Features

- Dual-URL resolution (primary + fallback CDN)
- Environment-based CDN configuration
- URL normalization and validation
- Support for relative and absolute paths

#### Configuration

```bash
# Environment variables
NEXT_PUBLIC_CDN_URL=https://cdn.lilith.app
NEXT_PUBLIC_CDN_FALLBACK_URL=https://cdn-fallback.lilith.app
```

#### Usage

```typescript
import { resolveAssetUrl, resolveFallbackUrl } from '@/lib/cdn';

// Resolve primary URL
const primaryUrl = resolveAssetUrl('/assets/model.glb');
// => https://cdn.lilith.app/assets/model.glb

// Get fallback URL
const fallbackUrl = resolveFallbackUrl('/assets/model.glb');
// => https://cdn-fallback.lilith.app/assets/model.glb
```

---

### 3. Avatar Asset Service

**Location**: `web/src/lib/ready-player-me-avatar.ts`

Specialized loader for Ready Player Me avatars with built-in caching.

#### Features

- Integration with Ready Player Me API
- In-memory avatar cache
- Cache key generation based on normalized parameters
- Metadata tracking (profileId, loadedAt, options)
- Force-reload option

#### Configuration

```typescript
interface AvatarLoadOptions {
  bodyType?: 'fullbody' | 'halfbody';
  gender?: 'masculine' | 'feminine' | 'neutral';
  textureSize?: 256 | 512 | 1024;
  meshLod?: 0 | 1 | 2;
  morphTargets?: string[];
  quality?: 'low' | 'medium' | 'high';
}
```

#### Usage

```typescript
import { ReadyPlayerMeAvatarService } from '@/lib/ready-player-me-avatar';

const avatarService = new ReadyPlayerMeAvatarService();

// Load avatar
const avatar = await avatarService.loadAvatar('avatar-id-123', {
  bodyType: 'fullbody',
  quality: 'high',
  textureSize: 1024,
  meshLod: 0,
});

// Force reload (bypass cache)
const freshAvatar = await avatarService.loadAvatar('avatar-id-123', {
  forceReload: true,
});

// Check cache
const isCached = avatarService.isAvatarCached('avatar-id-123');

// Clear cache
avatarService.clearCache();
```

---

### 4. GLTF Model Loading

Integration with Three.js GLTFLoader for 3D model assets.

#### Features

- GLTF 2.0 specification support
- Draco compression support
- KTX2 texture compression support
- Animation extraction
- Scene hierarchy parsing

#### Usage

```typescript
import { GLTFLoader } from 'three/addons/loaders/GLTFLoader.js';
import { DRACOLoader } from 'three/addons/loaders/DRACOLoader.js';

// Setup loader with Draco decoder
const loader = new GLTFLoader();
const dracoLoader = new DRACOLoader();
dracoLoader.setDecoderPath('/draco/');
loader.setDRACOLoader(dracoLoader);

// Load model
const gltf = await loader.loadAsync('/assets/model.glb');
const model = gltf.scene;
const animations = gltf.animations;

scene.add(model);
```

---

## Caching Strategy

### Cache Levels

| Level          | Type          | Persistence | Use Case                 |
| -------------- | ------------- | ----------- | ------------------------ |
| Memory         | In-memory Map | Session     | Avatars, frequent assets |
| Service Worker | Cache API     | Persistent  | Static assets            |
| HTTP Cache     | Browser       | TTL-based   | CDN assets               |

### Memory Cache Implementation

```typescript
class AssetCache {
  private cache = new Map<string, CachedAsset>();
  private maxSize = 100; // Maximum cached items
  private maxAge = 300000; // 5 minutes

  get(key: string): CachedAsset | undefined {
    const entry = this.cache.get(key);
    if (entry && Date.now() - entry.loadedAt < this.maxAge) {
      return entry;
    }
    return undefined;
  }

  set(key: string, asset: any, metadata?: any): void {
    // LRU eviction if at capacity
    if (this.cache.size >= this.maxSize) {
      const oldest = this.findOldest();
      this.cache.delete(oldest);
    }

    this.cache.set(key, {
      asset,
      loadedAt: Date.now(),
      metadata,
    });
  }

  clear(): void {
    this.cache.clear();
  }
}
```

---

## Optimization Systems

### 1. Level of Detail (LOD)

Automatically switch asset quality based on camera distance.

#### Distance Thresholds

| Level  | Distance (units) | Quality         |
| ------ | ---------------- | --------------- |
| HIGH   | 0-10             | Full detail     |
| MEDIUM | 10-25            | Reduced quality |
| LOW    | 25-50            | Minimal quality |
| CULLED | 50+              | Hidden          |

#### Avatar LOD Presets

| Preset              | Use Case        | Distances    |
| ------------------- | --------------- | ------------ |
| `CLOSE_INTERACTION` | Conversations   | 5-15-30-60   |
| `STANDARD`          | General use     | 10-25-50-100 |
| `CROWD`             | Many users      | 5-15-30-50   |
| `PERFORMANCE`       | Low-end devices | 5-10-20-40   |

```typescript
import { LODManager, LODPresets } from '@/lib/lod-system';

const lodManager = new LODManager({ camera });

// Register with preset
lodManager.registerObject(avatar, LODPresets.CROWD);

// Custom thresholds
lodManager.registerObject(tree, {
  high: 15,
  medium: 40,
  low: 80,
  culled: 150,
});
```

### 2. Frustum Culling

Automatically hide objects outside the camera view.

```typescript
import { FrustumCullingManager } from '@/lib/frustum-culling-system';

const cullingManager = new FrustumCullingManager({ camera });

// Register objects
cullingManager.registerObject(building, {
  method: 'BOX', // or 'SPHERE'
  updateFrequency: 16, // ms between checks
});

// Update in render loop
cullingManager.update();
```

### 3. Instanced Rendering

Efficiently render many identical objects.

```typescript
import { InstancedRenderingManager } from '@/lib/instanced-rendering-system';

const instanceManager = new InstancedRenderingManager();

// Create instanced mesh
const treesMeshId = instanceManager.createInstancedMesh({
  geometry: treeGeometry,
  material: treeMaterial,
  maxInstances: 1000,
  enableColors: true,
});

// Add instances efficiently
instanceManager.addInstances(
  treesMeshId,
  treePositions.map((pos) => ({
    position: pos,
    rotation: new Quaternion(),
    scale: new Vector3(1, 1, 1),
  }))
);

// Performance savings
const stats = instanceManager.getStats();
console.log(`Draw calls saved: ${stats.drawCallsSaved}`);
```

---

## Content Browser 3D

**Location**: `web/src/lib/content-browser-3d.ts`

3D spatial organization and visualization of content assets.

### Features

- Spatial organization of content items
- Category-based circular layout
- 3D representations by content type
- Interactive selection with hover/click
- Particle system for ambient effects

### Content Type Representations

| Content Type | 3D Representation |
| ------------ | ----------------- |
| Lecture      | Books             |
| Course       | Scrolls           |
| Meditation   | Crystals          |
| Practice     | Lotus flowers     |

### Usage

```typescript
import { ContentBrowser3D } from '@/lib/content-browser-3d';

const browser = new ContentBrowser3D(scene, {
  layout: 'circular',
  categorySpacing: 5,
  itemSpacing: 2,
});

// Add content items
browser.addItems([
  { id: '1', type: 'meditation', title: 'Morning Calm' },
  { id: '2', type: 'course', title: 'Zen Basics' },
  // ...
]);

// Handle selection
browser.onSelect((item) => {
  console.log('Selected:', item.title);
});

// Update hover state
browser.onHover((item) => {
  showPreview(item);
});
```

---

## Asset Pipeline Flow

```
┌─────────────────────────────────────────────────────────────┐
│ 1. Asset Request                                            │
│    - URL construction                                       │
│    - Cache check                                            │
└─────────────────────────────┬───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│ 2. Predictive Loader                                        │
│    - Priority queue placement                               │
│    - Background prefetch scheduling                         │
└─────────────────────────────┬───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│ 3. CDN Resolution                                           │
│    - Primary URL construction                               │
│    - Fallback URL preparation                               │
└─────────────────────────────┬───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│ 4. Asset Fetch                                              │
│    - Network request                                        │
│    - Format-specific loader                                 │
│    - Progress tracking                                      │
└─────────────────────────────┬───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│ 5. Cache Storage                                            │
│    - Memory cache update                                    │
│    - Metadata recording                                     │
└─────────────────────────────┬───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│ 6. Scene Addition                                           │
│    - Parent assignment                                      │
│    - Transform setup                                        │
└─────────────────────────────┬───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│ 7. Optimization Registration                                │
│    - LOD assignment                                         │
│    - Frustum culling registration                           │
│    - Instance batching (if applicable)                      │
└─────────────────────────────┬───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│ 8. Runtime Optimization                                     │
│    - Visibility management                                  │
│    - Quality level switching                                │
│    - Memory management                                      │
└─────────────────────────────────────────────────────────────┘
```

---

## Memory Management

### Automatic Disposal

```typescript
// Dispose geometry and materials
function disposeAsset(object: THREE.Object3D): void {
  object.traverse((child) => {
    if (child instanceof THREE.Mesh) {
      // Dispose geometry
      child.geometry.dispose();

      // Dispose material(s)
      if (Array.isArray(child.material)) {
        child.material.forEach(disposeMaterial);
      } else {
        disposeMaterial(child.material);
      }
    }
  });
}

function disposeMaterial(material: THREE.Material): void {
  material.dispose();

  // Dispose textures
  if ('map' in material) material.map?.dispose();
  if ('normalMap' in material) material.normalMap?.dispose();
  if ('roughnessMap' in material) material.roughnessMap?.dispose();
  if ('metalnessMap' in material) material.metalnessMap?.dispose();
  if ('aoMap' in material) material.aoMap?.dispose();
}
```

### Memory Monitoring

```typescript
const memoryMonitor = {
  getUsage(): number {
    const info = renderer.info;
    return (
      info.memory.geometries * 0.1 + // Estimated MB per geometry
      info.memory.textures * 0.5 // Estimated MB per texture
    );
  },

  checkThreshold(maxMB: number = 500): boolean {
    return this.getUsage() > maxMB;
  },

  triggerGC(): void {
    // Force garbage collection of disposed assets
    renderer.renderLists.dispose();
  },
};
```

---

## Best Practices

### Loading

- **Prefetch intelligently**: Load assets before they're visible
- **Use appropriate formats**: GLB for models, KTX2 for textures
- **Implement fallbacks**: Always have CDN fallback URLs
- **Cache aggressively**: Keep frequently-used assets in memory

### Optimization

- **Use LOD**: Always assign LOD levels to dynamic objects
- **Enable culling**: Register all objects for frustum culling
- **Batch instances**: Use instancing for repeated geometry
- **Monitor memory**: Track and dispose unused assets

### Performance

- **Limit concurrent loads**: Default to 3 concurrent prefetches
- **Use compression**: Draco for geometry, KTX2 for textures
- **Lazy load**: Only load assets when they'll be visible soon
- **Pool objects**: Reuse instances instead of creating new ones

---

## Troubleshooting

### Slow Loading

1. Check network latency to CDN
2. Verify asset compression is enabled
3. Reduce concurrent prefetch limit
4. Check for duplicate asset loads

### Memory Issues

1. Verify assets are being disposed properly
2. Check cache size limits
3. Monitor geometry/texture counts
4. Enable aggressive LOD culling

### Visual Issues

1. Verify texture formats are supported
2. Check LOD transition smoothness
3. Validate model normals and UVs
4. Test fallback loading

---

## Related Documentation

- [Rendering System](./rendering-system.md)
- [Avatar System](./avatar-system.md)
- [Spatial Audio](./spatial-audio.md)

---

_Last Updated: January 2026_
