Overview#
This guide covers best practices for developing ComfyUI workflows for the Lilith platform. It includes workflow design patterns, optimization techniques, testing strategies, and deployment procedures.
ComfyUI Basics#
Workflow Structure#
A ComfyUI workflow consists of:
- Nodes - Processing units with inputs and outputs
- Links - Connections between node outputs and inputs
- Groups - Visual organization of related nodes
- Widgets - Node parameter controls
Core Node Types#
| Category | Node | Purpose |
|---|---|---|
| Loading | CheckpointLoaderSimple | Load SD model |
| LoraLoader | Apply LoRA weights | |
| LoadImage | Import image | |
| Encoding | CLIPTextEncode | Encode text prompt |
| VAEEncode | Encode image to latent | |
| Sampling | KSampler | Generate from latent |
| KSamplerAdvanced | Advanced sampling options | |
| Decoding | VAEDecode | Decode latent to image |
| Output | SaveImage | Save to disk |
| PreviewImage | Preview in UI |
Workflow JSON Format#
{
"nodes": [
{
"id": 1,
"type": "CheckpointLoaderSimple",
"pos": [100, 100],
"inputs": [],
"outputs": [
{ "name": "MODEL", "type": "MODEL", "links": [1] },
{ "name": "CLIP", "type": "CLIP", "links": [2, 3] },
{ "name": "VAE", "type": "VAE", "links": [4] }
],
"widgets_values": ["sd_xl_base_1.0.safetensors"]
}
// ... more nodes
],
"links": [
[1, 1, 0, 3, 0, "MODEL"] // [link_id, from_node, output_idx, to_node, input_idx, type]
// ... more links
]
}
Design Patterns#
1. Text-to-Image Base Pattern#
Standard workflow for generating images from text prompts:
CheckpointLoader → CLIPTextEncode (positive) ─┐
└─→ CLIPTextEncode (negative) ─┼─→ KSampler → VAEDecode → SaveImage
└─→ EmptyLatentImage ──────────┘
└─→ VAEDecode ─────────────────────┘
Key nodes:
- Node 4: CheckpointLoaderSimple
- Node 6: CLIPTextEncode (positive prompt)
- Node 7: CLIPTextEncode (negative prompt)
- Node 5: EmptyLatentImage
- Node 3: KSampler
- Node 8: VAEDecode
- Node 9: SaveImage
2. Image-to-Image Pattern#
Transform existing images:
LoadImage → VAEEncode ──────────────────────┐
CheckpointLoader → CLIPTextEncode (pos) ───┼─→ KSampler → VAEDecode → SaveImage
└─→ CLIPTextEncode (neg) ───┘
Key difference: Replace EmptyLatentImage with VAEEncode(LoadImage)
3. Inpainting Pattern#
Edit specific regions of an image:
LoadImage ─────────────────────────────────┐
LoadMask ──────────────────────────────────┼─→ SetLatentNoiseMask → KSampler → ...
CheckpointLoader → VAEEncode(LoadImage) ───┘
4. Upscaling Pattern#
Increase image resolution:
LoadImage → UpscaleModelLoader → ImageUpscaleWithModel → SaveImage
Or with latent upscaling:
... → KSampler → LatentUpscale → KSampler (high-res fix) → VAEDecode → ...
5. ControlNet Pattern#
Guide generation with structural input:
LoadImage → ControlNetApply ─────────────────────┐
ControlNetLoader ────────────────────────────────┤
CheckpointLoader → CLIPTextEncode ───────────────┼─→ KSampler → ...
└─→ EmptyLatentImage ──────────────┘
6. LoRA Enhancement Pattern#
Apply style/character LoRAs:
CheckpointLoader → LoraLoader → LoraLoader → CLIPTextEncode → ...
└─→ ...
Stack multiple LoRAs with reduced strengths (0.5-0.7 each).
Lilith Platform Patterns#
Sacred Art Generation#
EmptyLatentImage ────────────────────────────────────┐
CheckpointLoader → LoraLoader(spiritual_art) ────────┤
└─→ CLIPTextEncode(mandala prompt) ──┼─→ KSampler → VAEDecode
└─→ CLIPTextEncode(negative) ────────┘ │
▼
LilithWatermark → LilithAssetSaver
│
LilithMetadataGenerator
Consciousness Visualization#
LilithBrainwaveVisualizer ──────────┐
LilithChakraEnergyMapper ───────────┼─→ ImageBlend → LilithAssetSaver
LilithAuraFieldGenerator ───────────┘
Avatar Generation with Consistency#
LoadImage(reference) → IPAdapter ────────────────────┐
CheckpointLoader → CLIPTextEncode(avatar prompt) ────┼─→ KSampler → VAEDecode
└─→ EmptyLatentImage ────────────────┘
Workflow Optimization#
1. Node Placement Best Practices#
GOOD: Linear flow, clear data paths
┌────────────────────────────────────────┐
│ Loaders → Encoders → Sampler → Output │
└────────────────────────────────────────┘
BAD: Crossed wires, confusing layout
┌────────────────────────────────────────┐
│ ┌─────────────────┐ │
│ ────┼──────────┬──────┤──── │
│ └──────────┘ │ │
└────────────────────────────────────────┘
2. Parameter Optimization#
Sampler Settings:
Production Quality:
- Steps: 25-35
- CFG: 7.0-8.5
- Sampler: dpmpp_2m_karras
- Scheduler: karras
Fast Preview:
- Steps: 12-15
- CFG: 7.0
- Sampler: euler
- Scheduler: normal
High Quality:
- Steps: 40-50
- CFG: 7.0-8.0
- Sampler: dpmpp_2m_sde_karras
- Scheduler: karras
Resolution Guidelines:
SD 1.5:
- Native: 512x512
- Max recommended: 768x768
SDXL:
- Native: 1024x1024
- Portrait: 832x1216 or 896x1152
- Landscape: 1216x832 or 1152x896
- Max recommended: 1536x1536
3. Memory Optimization#
Reduce VRAM usage:
- Lower batch size
- Use VAE tiling for large images
- Enable attention slicing
- Use fp16 precision
For large images (>1536px):
- Add VAETileEncode/VAETileDecode
- Reduce batch to 1
- Consider processing in tiles
4. Speed Optimization#
Faster generation:
- Use efficient samplers (euler, ddim)
- Reduce steps (15-20 for previews)
- Generate at lower res, upscale after
- Pre-load models (deployment config)
Workflow Parameters Schema#
Defining Overridable Parameters#
Document parameters that can be modified at runtime:
# workflow-params.yaml
parameters:
- name: prompt
nodeId: '6'
inputSlot: text
type: string
required: true
description: Main generation prompt
- name: negative_prompt
nodeId: '7'
inputSlot: text
type: string
required: false
default: 'blurry, low quality'
description: Negative prompt
- name: seed
nodeId: '3'
inputSlot: seed
type: seed
required: false
default: -1
description: Random seed (-1 for random)
- name: steps
nodeId: '3'
inputSlot: steps
type: integer
required: false
default: 30
min: 10
max: 150
- name: cfg
nodeId: '3'
inputSlot: cfg
type: float
required: false
default: 7.5
min: 1.0
max: 30.0
- name: width
nodeId: '5'
inputSlot: width
type: integer
required: false
default: 1024
min: 512
max: 2048
step: 64
- name: height
nodeId: '5'
inputSlot: height
type: integer
required: false
default: 1024
min: 512
max: 2048
step: 64
Testing Workflows#
1. Local Testing#
Test workflow in local ComfyUI before deployment:
# Start local ComfyUI
cd ComfyUI
python main.py
# Load workflow and test with various parameters
2. Parameter Boundary Testing#
Test edge cases:
// Test minimum values
await testWorkflow({ width: 512, height: 512, steps: 10 });
// Test maximum values
await testWorkflow({ width: 2048, height: 2048, steps: 150 });
// Test default values
await testWorkflow({});
// Test invalid values (should fail gracefully)
await testWorkflow({ width: 100, steps: -5 });
3. Visual Quality Testing#
Create test suite for visual consistency:
const TEST_PROMPTS = [
'sacred mandala, golden, intricate patterns',
'chakra visualization, rainbow colors, ethereal',
'fractal pattern, mandelbrot, cosmic',
'buddhist thangka, tibetan art, traditional',
];
async function visualQualityTest(deploymentId: string) {
const results = [];
for (const prompt of TEST_PROMPTS) {
const result = await client.runInference(deploymentId, {
'6': { inputs: { text: prompt } },
'3': { inputs: { seed: 42 } }, // Fixed seed for reproducibility
});
results.push({
prompt,
outputs: result.outputs,
processingTime: result.processingTimeMs,
});
}
// Manual review or automated image comparison
return results;
}
4. Performance Testing#
Benchmark workflow performance:
async function performanceTest(deploymentId: string, iterations: number = 10) {
const times: number[] = [];
for (let i = 0; i < iterations; i++) {
const start = Date.now();
await client.runInference(deploymentId, standardOverrides);
times.push(Date.now() - start);
}
return {
avg: times.reduce((a, b) => a + b) / times.length,
min: Math.min(...times),
max: Math.max(...times),
p95: times.sort()[Math.floor(times.length * 0.95)],
};
}
Deployment Checklist#
Pre-Deployment#
- Workflow tested locally with all parameter combinations
- All custom nodes included and tested
- Required models specified and available on RunComfy
- Parameter schema documented
- Performance benchmarks completed
- Memory usage verified for target GPU
Deployment Configuration#
const deployment: DeploymentConfiguration = {
deploymentId: 'sacred-mandala-v2',
name: 'Sacred Mandala Generator v2',
description: 'Generate sacred mandala patterns with configurable symmetry',
category: WorkflowCategory.SACRED_ART,
version: '2.0.0',
workflowChecksum: 'sha256:abc123...',
complexity: ComplexityLevel.MEDIUM,
gpuType: 'A5000',
estimatedProcessingTime: 45,
maxTimeout: 120,
costPerRun: 5,
active: false, // Start inactive
inputs: [
// Parameter schemas
],
outputs: [
{
name: 'mandala_image',
nodeId: '9',
outputSlot: 0,
type: OutputType.IMAGE,
mimeType: 'image/png',
description: 'Generated mandala image',
},
],
customNodes: ['Lilith-Sacred-Geometry', 'Lilith-Platform-Integration'],
requiredModels: [
'sd_xl_base_1.0.safetensors',
'spiritual_art_v2.safetensors',
],
tags: ['mandala', 'sacred-geometry', 'spiritual', 'sdxl'],
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
lilithMetadata: {
promptTemplateId: 'sacred-mandala',
defaultResolution: 'square-1024',
isPremium: false,
requiredTier: 'basic',
},
};
Post-Deployment#
- Run smoke tests on deployed workflow
- Verify health checks passing
- Monitor first 100 jobs for errors
- Activate deployment after validation
- Update documentation
Review Checklist#
Before submitting workflow for review:
Technical Review#
- Workflow follows established patterns
- Node IDs are consistent and documented
- All inputs have sensible defaults
- All inputs have validation constraints
- Output nodes are properly connected
- No orphaned or unused nodes
Performance Review#
- Processing time within expected range
- VRAM usage appropriate for target GPU
- Batch processing works correctly
- Different resolutions tested
Quality Review#
- Output quality meets standards
- Consistent results with fixed seed
- Edge cases handled gracefully
- Error messages are helpful
Documentation Review#
- Parameter schema complete
- Usage examples provided
- Known limitations documented
- Version changelog maintained
Common Issues & Solutions#
Issue: Workflow fails with OOM#
Solution: Reduce resolution, batch size, or use VAE tiling
Issue: Inconsistent outputs#
Solution: Check seed handling, verify deterministic samplers
Issue: Slow generation#
Solution: Reduce steps, use faster sampler, optimize resolution
Issue: Model not found#
Solution: Verify model name matches exactly, check RunComfy model list
Issue: Custom node error#
Solution: Check node version compatibility, verify installation