# Civitai Model Management

This document covers model discovery, downloads, versioning, and lifecycle
management for Civitai models in the Oshun platform.

## Overview

The model management system provides:

- **Model Discovery**: Search, browse, and filter models
- **Download Management**: Queue-based downloading with progress tracking
- **Version Control**: Handle model versions and compatibility
- **Storage Optimization**: Intelligent caching and cleanup
- **Hot-Swap**: Live model switching with memory management

## CivitAI Service

The `CivitAIService` provides a production-grade API client with resilience
patterns built in.

### Initialization

```typescript
import { CivitAIService, createCivitAIService } from '@oshun/civitai-service';

const service = createCivitAIService({
  apiKey: process.env.CIVITAI_API_KEY,
  rateLimitBuffer: 60, // requests per minute
  cacheTtl: 300, // cache TTL in seconds
  maxConcurrentDownloads: 3,
  enableCache: true,
  nsfwEnabled: true,
  contentLevel: 'R', // PG, PG13, R, X, XXX
});
```

### Configuration Options

```typescript
interface CivitAIConfig {
  apiKey: string;
  baseUrl?: string; // default: 'https://civitai.com/api/v1'
  rateLimitBuffer?: number; // default: 60
  cacheTtl?: number; // default: 300
  retryAttempts?: number; // default: 3
  retryBaseDelay?: number; // default: 1000
  enableCache?: boolean; // default: true
  maxConcurrentDownloads?: number; // default: 3
  nsfwEnabled?: boolean; // default: true
  contentLevel?: ContentLevel; // default: 'R'
}
```

## Model Discovery

### Listing Models

```typescript
const response = await service.listModels({
  query: 'portrait', // search query
  types: ['Checkpoint'], // model types
  sort: 'Highest Rated', // sort order
  page: 1,
  limit: 20,
  nsfw: false,
});

for (const model of response.items) {
  console.log(`${model.name} (${model.type})`);
  console.log(`Creator: ${model.creator.username}`);
  console.log(`Downloads: ${model.stats.downloadCount}`);
  console.log(`Rating: ${model.stats.rating}/5`);
}
```

### Search Parameters

| Parameter  | Type        | Description                |
| ---------- | ----------- | -------------------------- |
| `query`    | string      | Search query string        |
| `types`    | ModelType[] | Filter by model types      |
| `sort`     | string      | Sort order                 |
| `page`     | number      | Page number (1-indexed)    |
| `limit`    | number      | Results per page (max 100) |
| `nsfw`     | boolean     | Include NSFW content       |
| `tag`      | string      | Filter by tag              |
| `username` | string      | Filter by creator          |

### Model Types

```typescript
type ModelType =
  | 'Checkpoint'
  | 'TextualInversion'
  | 'Hypernetwork'
  | 'AestheticGradient'
  | 'LORA'
  | 'Controlnet'
  | 'Poses';
```

### Sort Options

| Value             | Description            |
| ----------------- | ---------------------- |
| `Highest Rated`   | Sort by rating         |
| `Most Downloaded` | Sort by download count |
| `Newest`          | Sort by creation date  |

### Getting Model Details

```typescript
// Get model by ID
const model = await service.getModel(133005);

console.log('Model:', model.name);
console.log('Description:', model.description);
console.log('Tags:', model.tags);
console.log('NSFW:', model.nsfw);
console.log('Commercial Use:', model.allowCommercialUse);

// List versions
for (const version of model.modelVersions) {
  console.log(`Version: ${version.name}`);
  console.log(`Base Model: ${version.baseModel}`);
  console.log(`Trained Words: ${version.trainedWords.join(', ')}`);
}
```

### Getting Version by Hash

```typescript
// Useful for identifying models by file hash
const version = await service.getModelVersionByHash(
  'abc123def456...' // SHA256 hash
);

console.log('Model:', version.model.name);
console.log('Version:', version.name);
```

### Listing Creators

```typescript
const creators = await service.listCreators({
  query: 'artist',
  limit: 20,
});

for (const creator of creators.items) {
  console.log(`Username: ${creator.username}`);
}
```

### Listing Tags

```typescript
const tags = await service.listTags({
  query: 'style',
  limit: 50,
});

for (const tag of tags.items) {
  console.log(`${tag.name}: ${tag.modelCount} models`);
}
```

## Download Management

### Download Queue

The service includes a priority-based download queue with concurrent download
support.

```typescript
// Queue a download
const downloadId = service.queueDownload({
  modelVersionId: 348913,
  destinationPath: '/models/checkpoints/',
  customFilename: 'juggernaut-xl.safetensors',
  priority: 'normal',
  verifyHash: true,
  hashAlgorithm: 'SHA256',
  onProgress: (progress) => {
    console.log(`${progress.percentage}% - ${progress.speedBps / 1024} KB/s`);
  },
});

console.log('Download queued:', downloadId);
```

### Download Priorities

| Priority | Description           | Use Case                 |
| -------- | --------------------- | ------------------------ |
| `urgent` | Immediate processing  | User-requested downloads |
| `high`   | Next in queue         | Auto-triggered downloads |
| `normal` | Standard priority     | Batch downloads          |
| `low`    | Background processing | Pre-caching              |

### Monitoring Progress

```typescript
// Get single download progress
const progress = service.getDownloadProgress(downloadId);

if (progress) {
  console.log(`Status: ${progress.status}`);
  console.log(`Progress: ${progress.percentage}%`);
  console.log(
    `Downloaded: ${progress.downloadedBytes} / ${progress.totalBytes}`
  );
  console.log(`Speed: ${progress.speedBps / 1024} KB/s`);
  console.log(`ETA: ${progress.etaSeconds} seconds`);
}

// Get all downloads
const allDownloads = service.getAllDownloads();
for (const download of allDownloads) {
  console.log(
    `${download.id}: ${download.status} - ${download.progress.percentage}%`
  );
}

// Get queue statistics
const stats = service.getDownloadStats();
console.log(`Total: ${stats.totalItems}`);
console.log(`Queued: ${stats.queued}`);
console.log(`Downloading: ${stats.downloading}`);
console.log(`Completed: ${stats.completed}`);
console.log(`Failed: ${stats.failed}`);
```

### Download Control

```typescript
// Pause download
service.pauseDownload(downloadId);

// Resume download
service.resumeDownload(downloadId);

// Cancel download
service.cancelDownload(downloadId);
```

### Download Status Flow

```
queued → downloading → completed
                    → failed
         paused ↔ queued
         cancelled
```

## Model Type Manager

The Model Type Manager provides metadata and compatibility information for
different model types.

### Supported Model Types

| Type               | Subtypes                              | Description                  |
| ------------------ | ------------------------------------- | ---------------------------- |
| **Checkpoints**    | base, refiner, inpaint, merged        | Base models for generation   |
| **LoRA/LyCORIS**   | LoRA, LoHa, LoKr, LoCon, DyLoRA, DoRA | Fine-tuning adapters         |
| **VAE**            | standard, MSE, EMA, TAESD             | Image encoding/decoding      |
| **Embeddings**     | positive, negative, style, character  | Text embeddings              |
| **ControlNet**     | canny, depth, openpose, lineart       | Conditional generation       |
| **IP-Adapters**    | standard, plus, faceid                | Image-conditioned generation |
| **Text Encoders**  | CLIP-L, CLIP-G, T5, SigLIP            | Text encoding models         |
| **Upscalers**      | ESRGAN, RealESRGAN, SwinIR            | Image upscaling              |
| **Motion Modules** | AnimateDiff v1-3, LCM                 | Video generation             |
| **Inpainting**     | BrushNet, PowerPaint, MAT, LaMa       | Image inpainting             |

### Architecture Support

| Architecture     | Versions                   |
| ---------------- | -------------------------- |
| Stable Diffusion | 1.4, 1.5, 2.0, 2.1         |
| SDXL             | 0.9, 1.0, Turbo, Lightning |
| Flux             | 1.0 S, D, D Hyper          |
| Pony             | V6                         |
| Video            | LTXV, CogVideoX, SVD       |

## Rate Limiting

### Sliding Window Rate Limiter

The service implements a sliding window rate limiter to respect API limits.

```typescript
// Check rate limit state
const rateLimitState = service.getRateLimitState();

console.log(`Remaining: ${rateLimitState.remaining}/${rateLimitState.limit}`);
console.log(`Reset at: ${rateLimitState.resetAt}`);
console.log(`Is limited: ${rateLimitState.isLimited}`);

if (rateLimitState.isLimited) {
  console.log(`Retry after: ${rateLimitState.retryAfter} seconds`);
}
```

### Rate Limit Configuration

| Setting      | Default     | Description             |
| ------------ | ----------- | ----------------------- |
| Window       | 60 seconds  | Sliding window duration |
| Max Requests | 60          | Requests per window     |
| Retry Delay  | Exponential | Backoff on 429 response |

## Circuit Breaker

### Circuit States

The circuit breaker protects against cascading failures:

```
closed → (failures exceed threshold) → open
  ↑                                      ↓
  └──────── (success threshold met) ← half_open
                                         ↑
                        (timeout elapsed) ┘
```

### Checking Circuit State

```typescript
const circuitState = service.getCircuitBreakerState();

console.log(`State: ${circuitState.state}`);
console.log(`Failures: ${circuitState.failures}`);
console.log(`Successes: ${circuitState.successes}`);

if (circuitState.state === 'open') {
  console.log(`Opened at: ${circuitState.openedAt}`);
}
```

### Circuit Breaker Configuration

| Setting           | Default    | Description                |
| ----------------- | ---------- | -------------------------- |
| Failure Threshold | 5          | Failures to open circuit   |
| Success Threshold | 3          | Successes to close circuit |
| Timeout           | 30 seconds | Time before half-open      |
| Monitor Window    | 60 seconds | Failure tracking window    |

## Request Caching

### Cache Configuration

```typescript
// Cache is enabled by default
const service = createCivitAIService({
  apiKey: process.env.CIVITAI_API_KEY,
  enableCache: true,
  cacheTtl: 300, // 5 minutes
});

// Check cache statistics
const cacheStats = service.getCacheStats();
console.log(`Size: ${cacheStats.size}`);
console.log(`Hits: ${cacheStats.hits}`);
console.log(`Misses: ${cacheStats.misses}`);
console.log(
  `Hit Rate: ${(cacheStats.hits / (cacheStats.hits + cacheStats.misses)) * 100}%`
);

// Clear cache
service.clearCache();
```

### Cache Behavior

- GET requests are cached by default
- Cache key is generated from endpoint + params
- LRU eviction when cache reaches max size
- Automatic cleanup of expired entries

### Bypassing Cache

```typescript
// Force fresh request
const model = await service.request('/models/133005', {
  useCache: false,
});
```

## Content Filtering

### Content Levels

| Level  | Description       | Includes       |
| ------ | ----------------- | -------------- |
| `PG`   | Safe for all ages | PG only        |
| `PG13` | Teen appropriate  | PG, PG13       |
| `R`    | Adult themes      | PG, PG13, R    |
| `X`    | Explicit content  | PG, PG13, R, X |
| `XXX`  | All content       | Everything     |

### Configuring Content Filters

```typescript
const service = createCivitAIService({
  apiKey: process.env.CIVITAI_API_KEY,
  nsfwEnabled: true,
  contentLevel: 'R',
});

// Check if content is allowed
const isAllowed = service.isContentLevelAllowed('PG13');
console.log('PG13 allowed:', isAllowed);
```

## Event System

### Event Types

| Event              | Description       | Data                       |
| ------------------ | ----------------- | -------------------------- |
| `request_start`    | Request initiated | endpoint, method, url      |
| `request_complete` | Request succeeded | endpoint, method, duration |
| `request_error`    | Request failed    | endpoint, method, error    |
| `rate_limited`     | Rate limit hit    | endpoint, waitTime         |
| `circuit_open`     | Circuit opened    | endpoint                   |
| `cache_hit`        | Cache hit         | endpoint, cacheKey         |
| `cache_miss`       | Cache miss        | endpoint, cacheKey         |
| `download_start`   | Download queued   | downloadId, request        |

### Subscribing to Events

```typescript
// Subscribe to events
service.on('request_complete', (event) => {
  console.log(
    `${event.data.method} ${event.data.endpoint} - ${event.data.duration}ms`
  );
});

service.on('rate_limited', (event) => {
  console.warn(`Rate limited! Wait ${event.data.waitTime}ms`);
});

service.on('circuit_open', (event) => {
  console.error(`Circuit breaker opened for ${event.data.endpoint}`);
});

// Unsubscribe
const listener = (event) => console.log(event);
service.on('cache_hit', listener);
service.off('cache_hit', listener);
```

## Analytics

### Request Analytics

```typescript
const analytics = service.getAnalytics();

console.log(`Total Requests: ${analytics.totalRequests}`);
console.log(`Successful: ${analytics.successfulRequests}`);
console.log(`Failed: ${analytics.failedRequests}`);
console.log(`Rate Limited: ${analytics.rateLimitedRequests}`);
console.log(`Avg Response Time: ${analytics.averageResponseTimeMs}ms`);

// Per-endpoint counts
for (const [endpoint, count] of Object.entries(analytics.endpointCounts)) {
  console.log(`${endpoint}: ${count} requests`);
}

// Error distribution
for (const [code, count] of Object.entries(analytics.errorCounts)) {
  console.log(`${code}: ${count} errors`);
}

// Reset analytics
service.resetAnalytics();
```

## Civitai Link (Hot-Swap)

The Civitai Link provider enables real-time model synchronization via WebSocket.

### Initialization

```typescript
import { CivitaiLinkProvider } from '@oshun/civitai-link';

const link = new CivitaiLinkProvider({
  endpoint: process.env.CIVITAI_LINK_ENDPOINT,
  apiKey: process.env.CIVITAI_LINK_KEY,
  autoReconnect: true,
  reconnectDelay: 5000,
  heartbeatInterval: 30000,
});

await link.connect();
```

### Hot-Swap Models

```typescript
// Swap to a different model
await link.swapModel({
  modelVersionId: 348913,
  slot: 'checkpoint',
  priority: 'high',
});

// Check memory stats
const memory = await link.getMemoryStats();
console.log(`VRAM Used: ${memory.vramUsed / 1024 / 1024} MB`);
console.log(`VRAM Free: ${memory.vramFree / 1024 / 1024} MB`);
console.log(`RAM Used: ${memory.ramUsed / 1024 / 1024} MB`);
```

### Download Queue

```typescript
// Queue download with priority
link.queueDownload({
  modelVersionId: 348913,
  priority: 'high',
  onProgress: (progress) => {
    console.log(`${progress.filename}: ${progress.percentage}%`);
  },
});

// Get queue status
const queue = link.getDownloadQueue();
for (const item of queue) {
  console.log(`${item.id}: ${item.status}`);
}
```

### Predictive Loading

```typescript
// Enable predictive loading based on usage patterns
link.enablePredictiveLoading({
  analyzeUsagePatterns: true,
  preloadThreshold: 0.7, // 70% confidence
});

// Get preload recommendations
const recommendations = await link.getPreloadRecommendations();
for (const rec of recommendations) {
  console.log(`Recommend preloading: ${rec.modelName}`);
  console.log(`Usage frequency: ${rec.usageFrequency}`);
  console.log(`Confidence: ${rec.confidence}`);
}
```

### Storage Optimization

```typescript
// Analyze storage usage
const analysis = await link.analyzeStorage();

console.log(`Total models: ${analysis.totalModels}`);
console.log(`Total size: ${analysis.totalSize / 1024 / 1024 / 1024} GB`);
console.log(`Duplicates: ${analysis.duplicates.length}`);

// Get cleanup suggestions
const suggestions = await link.getCleanupSuggestions();
for (const suggestion of suggestions) {
  console.log(`Suggest removing: ${suggestion.modelName}`);
  console.log(`Reason: ${suggestion.reason}`);
  console.log(`Space savings: ${suggestion.sizeSaved / 1024 / 1024} MB`);
}

// Execute cleanup
await link.executeCleanup(suggestions.map((s) => s.modelVersionId));
```

## Best Practices

### Efficient API Usage

1. **Use caching**: Enable caching for repeated queries
2. **Batch requests**: Combine multiple model lookups when possible
3. **Respect rate limits**: Monitor rate limit state proactively
4. **Handle errors gracefully**: Implement retry logic with backoff

### Download Management

1. **Use priorities wisely**: Reserve `urgent` for user-initiated downloads
2. **Monitor progress**: Provide feedback to users on download status
3. **Verify hashes**: Always verify downloaded files
4. **Clean up failures**: Remove failed downloads from queue

### Model Organization

1. **Use consistent naming**: Follow a naming convention for downloaded models
2. **Track versions**: Maintain metadata about model versions
3. **Regular cleanup**: Remove unused models periodically
4. **Monitor disk space**: Set alerts for low disk space

## Related Documentation

- [API Documentation](./api.md) - Generation API reference
- [Training Guide](./training.md) - Training custom LoRAs
- [Best Practices](./best-practices.md) - Optimization and cost management