# ComfyUI Integration Documentation

This directory contains comprehensive documentation for integrating ComfyUI
image generation capabilities into the Oshun platform.

## Quick Links

| Document                                                                          | Description                           |
| --------------------------------------------------------------------------------- | ------------------------------------- |
| [Workflow Documentation](./workflows.md)                                          | Creating and executing workflows      |
| [Node Reference](./node-reference.md)                                             | Common nodes and their configurations |
| [Performance Tuning](./performance-tuning.md)                                     | Optimization and scaling strategies   |
| [Custom Node Development](./custom-nodes.md)                                      | Building custom nodes                 |
| [RunComfy API Client](../../domains/lilith/extras/comfyui/runcomfy-api-client.md) | API client reference                  |

## Overview

ComfyUI is a node-based interface for Stable Diffusion and other image
generation models. Oshun integrates with ComfyUI through multiple providers for
scalable, production-grade image generation.

### Key Features

- **Multi-Provider Support**: RunComfy, RunPod, Modal, Replicate, Vast.ai
- **Intelligent Routing**: Cost-optimized, latency-optimized, or reliability
  modes
- **Automatic Failover**: Seamless switching between providers
- **Workflow Flexibility**: Built-in types + custom workflow support
- **Production Ready**: Rate limiting, circuit breakers, comprehensive
  monitoring

## Architecture

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                        Oshun ComfyUI Integration                             │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  Application Layer                                                           │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │  Lilith Consciousness  │  Yemaya Creative    │    Isis Factory      │    │
│  │  - Avatar generation   │  - Asset creation   │    - Batch gen       │    │
│  │  - Real-time preview   │  - Style transfer   │    - Upscaling       │    │
│  └─────────────────────────────────────────────────────────────────────┘    │
│                                      │                                       │
│  Service Layer                       ▼                                       │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │                     ComfyUI Service                                  │    │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐              │    │
│  │  │ RunComfy     │  │ RunPod       │  │ Comfy Cloud  │              │    │
│  │  │ Provider     │  │ Adapter      │  │ Provider     │              │    │
│  │  └──────────────┘  └──────────────┘  └──────────────┘              │    │
│  │                                                                      │    │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐              │    │
│  │  │ Rate Limiter │  │ Circuit      │  │ Retry        │              │    │
│  │  │              │  │ Breaker      │  │ Handler      │              │    │
│  │  └──────────────┘  └──────────────┘  └──────────────┘              │    │
│  └─────────────────────────────────────────────────────────────────────┘    │
│                                      │                                       │
│  Provider Layer                      ▼                                       │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │  RunComfy    │  RunPod    │  Modal    │  Replicate  │  Vast.ai     │    │
│  │  (Primary)   │  (Primary) │  (Alt)    │  (Alt)      │  (Budget)    │    │
│  └─────────────────────────────────────────────────────────────────────┘    │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘
```

## Getting Started

### Prerequisites

1. Provider API key (RunComfy, RunPod, or other)
2. Node.js 18+ environment
3. Network access to provider APIs

### Quick Start

```typescript
import { ComfyUIService } from '@oshun/comfyui-service';

// Initialize service
const service = new ComfyUIService({
  enableFailover: true,
  runcomfyConfig: {
    apiKey: process.env.RUNCOMFY_API_KEY,
    maxConcurrentJobs: 5,
  },
  runpodConfig: {
    apiKey: process.env.RUNPOD_API_KEY,
    endpointId: process.env.RUNPOD_COMFYUI_ENDPOINT,
  },
});

await service.initialize();

// Text-to-image generation
const result = await service.generateTxt2Img({
  prompt: 'A majestic dragon flying over a castle at sunset',
  negativePrompt: 'blurry, low quality',
  model: 'sd_xl_base_1.0.safetensors',
  width: 1024,
  height: 1024,
  steps: 30,
  cfg: 7.5,
});

console.log('Generated image:', result.outputs[0].url);
```

### Custom Workflow

```typescript
import { RunComfyProvider } from '@oshun/comfy-provider';

const provider = new RunComfyProvider({
  apiKey: process.env.RUNCOMFY_API_KEY,
});

// Submit custom workflow
const result = await provider.submitJob({
  workflow: myCustomWorkflow,
  inputs: [
    { name: 'prompt', value: 'A beautiful landscape' },
    { name: 'seed', value: 12345 },
  ],
  webhookUrl: 'https://api.myapp.com/webhooks/comfyui',
});
```

## Supported Generation Types

| Type         | Description          | Typical Latency |
| ------------ | -------------------- | --------------- |
| **txt2img**  | Text to image        | 20-60s          |
| **img2img**  | Image transformation | 15-45s          |
| **inpaint**  | Fill masked areas    | 20-60s          |
| **upscale**  | AI upscaling         | 10-30s          |
| **workflow** | Custom workflows     | Variable        |

## Supported Models

### Stable Diffusion

| Model          | VRAM | Resolution | Notes               |
| -------------- | ---- | ---------- | ------------------- |
| SD 1.5         | 4GB  | 512x512    | Fast, lower quality |
| SD 2.1         | 5GB  | 768x768    | Good quality        |
| SDXL Base      | 8GB  | 1024x1024  | High quality        |
| SDXL + Refiner | 12GB | 1024x1024  | Best quality        |

### Flux

| Model          | VRAM | Resolution | Notes             |
| -------------- | ---- | ---------- | ----------------- |
| Flux.1 Dev     | 24GB | Variable   | Excellent quality |
| Flux.1 Schnell | 20GB | Variable   | Fast variant      |

## Providers

### RunComfy (Recommended)

- Pre-configured ComfyUI environments
- Managed custom nodes
- Pay-per-use pricing
- Best for: Production workloads

### RunPod

- GPU-based serverless
- Full control over environment
- Scale to zero capability
- Best for: Custom setups

### Multi-Provider Setup

```typescript
import { ComfyCloudProvider } from '@oshun/comfy-cloud';

const provider = new ComfyCloudProvider({
  providers: ['runcomfy', 'runpod', 'modal'],
  routingStrategy: 'cost_optimized',
  enableFailover: true,
  dailyBudget: 50,
});
```

## Monitoring

### Key Metrics

- **Job Success Rate**: Target >99%
- **Average Latency**: Target <30s for standard generation
- **Queue Depth**: Monitor for scaling needs
- **Error Distribution**: Track by error code

### Health Checks

```typescript
const health = await service.healthCheck();

console.log('Providers:', health.providers);
console.log('Queue depth:', health.queue.depth);
console.log('Overall health:', health.healthy);
```

## Best Practices

1. **Use appropriate models**: Match model to task complexity
2. **Optimize steps**: 20-30 steps sufficient for most cases
3. **Batch similar jobs**: Group by model to reduce loading
4. **Cache workflows**: Template workflows for common tasks
5. **Monitor costs**: Track GPU usage and optimize

## Error Handling

```typescript
try {
  const result = await service.generateTxt2Img(params);
} catch (error) {
  if (error.code === 'RATE_LIMITED') {
    // Retry with exponential backoff
  } else if (error.code === 'TIMEOUT') {
    // Consider reducing complexity
  } else if (error.code === 'MODEL_NOT_FOUND') {
    // Check model availability
  }
}
```

## Support

- Internal: #image-gen Slack channel
- RunComfy: support@runcomfy.com
- RunPod: [RunPod Discord](https://discord.gg/runpod)

## Related Documentation

- [ElevenLabs Integration](../elevenlabs/)
- [Civitai Integration](../civitai/)
- [RunPod Infrastructure](../../infrastructure/runpod-integration.md)