Eve library

Pipeline Architecture

Guide

Revision 1

On this page

Comprehensive architecture diagrams for all Yemaya platform pipelines.

Table of Contents#

  1. System Overview
  2. Game Creation Pipeline
  3. Film Creation Pipeline
  4. Pipeline Infrastructure
  5. Quality Gates System
  6. Worker Queue System
  7. 3D Generation Pipeline
  8. Agent Orchestration System
  9. Data Flow Patterns

System Overview#

High-Level Architecture#

flowchart TB subgraph Clients["Client Applications"] Web["Web App"] Desktop["Desktop App"] CLI["CLI Tool"] SDK["TypeScript/Python SDK"] end subgraph API["API Layer"] REST["REST API<br/>(Hono)"] WS["WebSocket<br/>(Real-time)"] GRPC["gRPC Services<br/>(Protobuf)"] end subgraph Pipelines["Pipeline Orchestration"] GamePipe["Game Creation<br/>Pipeline"] FilmPipe["Film Creation<br/>Pipeline"] ThreeD["3D Generation<br/>Pipeline"] Audio["Audio Pipeline"] end subgraph Orchestration["Agent Orchestration"] Crew["Crew System"] Workflow["Workflow Engine"] Delegation["Task Delegation"] RAG["RAG Pipeline"] end subgraph Workers["Worker System (BullMQ)"] AgentWorker["Agent Worker"] MediaWorker["Media Worker"] AssetWorker["Asset Worker"] Gen3DWorker["3D Gen Worker"] NotifyWorker["Notification Worker"] end subgraph QualityGates["Quality Gates"] Narrative["Narrative QG"] Visual["Visual QG"] AudioQG["Audio QG"] Technical["Technical QG"] Gameplay["Gameplay QG"] end subgraph Infrastructure["Infrastructure"] Postgres[(PostgreSQL)] Redis[(Redis)] MinIO[(MinIO/S3)] CDN["CDN"] end subgraph External["External Services"] AI["AI Providers<br/>(Claude, OpenAI, Gemini)"] ThreeDProviders["3D Providers<br/>(Rodin, Meshy, etc.)"] GameEngines["Game Engines<br/>(Blender, Godot, Unreal)"] end Clients --> API API --> Pipelines Pipelines --> Orchestration Pipelines --> Workers Pipelines --> QualityGates Workers --> Infrastructure Workers --> External Orchestration --> AI ThreeD --> ThreeDProviders GamePipe --> GameEngines FilmPipe --> GameEngines

Pipeline Types Comparison#

graph LR subgraph Types["Pipeline Types"] direction TB Game["Game Pipeline<br/>8 Phases<br/>3,336 LOC"] Film["Film Pipeline<br/>5 Phases<br/>2,345 LOC"] Infra["Infrastructure<br/>4 Components<br/>1,716 LOC"] QG["Quality Gates<br/>5 Types<br/>3,033 LOC"] Test["Automated Testing<br/>4 Categories<br/>2,154 LOC"] end Game --> |"Concept → Release"| GameOut["Playable Game"] Film --> |"Development → Delivery"| FilmOut["Final Film"] Infra --> |"Execution Support"| InfraOut["Pipeline Ops"] QG --> |"Validation"| QGOut["Quality Assurance"] Test --> |"Verification"| TestOut["Test Reports"]

Game Creation Pipeline#

Complete Phase Flow#

flowchart TB subgraph Phase1["Phase 1: Concept Generation"] UserPrompt["User Prompt"] GameConcept["Game Concept"] ArtDirection["Art Direction"] MarketAnalysis["Market Analysis"] MoodBoards["Mood Boards"] end subgraph Phase2["Phase 2: GDD Generation"] CoreLoop["Core Loop Design"] Mechanics["Mechanic Design"] Systems["System Design"] Levels["Level Design"] Narrative["Narrative Design"] end subgraph Phase3["Phase 3: Prototype"] Prototype["Playable Prototype"] Metrics["Prototype Metrics"] Feedback["Playtester Feedback"] Iteration["Prototype Iteration"] end subgraph Phase4["Phase 4: Asset Production"] AssetPlan["Asset Production Plan"] Models["3D Models"] Animations["Animations"] Textures["Textures"] VFX["Visual Effects"] Audio["Audio Assets"] end subgraph Phase5["Phase 5: Level Assembly"] LevelAssembly["Level Construction"] Scripting["Gameplay Scripting"] Triggers["Trigger Setup"] Lighting["Lighting Setup"] UI["UI Implementation"] end subgraph Phase6["Phase 6: Polish & Optimization"] BugFix["Bug Fixing"] PerfOpt["Performance Optimization"] Balance["Balance Tuning"] Juice["Juice Pass"] Accessibility["Accessibility"] end subgraph Phase7["Phase 7: QA & Testing"] TestStrategy["Testing Strategy"] AutoTests["Automated Tests"] AIPlaytest["AI Playtesting"] ManualTest["Manual Testing"] end subgraph Phase8["Phase 8: Release"] PlatformBuild["Platform Builds"] StoreSubmit["Store Submission"] Marketing["Marketing"] Launch["Launch"] end Phase1 --> |QualityGate| Phase2 Phase2 --> |QualityGate| Phase3 Phase3 --> |QualityGate| Phase4 Phase4 --> |QualityGate| Phase5 Phase5 --> |QualityGate| Phase6 Phase6 --> |QualityGate| Phase7 Phase7 --> |QualityGate| Phase8 Phase8 --> Released["Released Game"] style Phase1 fill:#e1f5fe style Phase2 fill:#e8f5e9 style Phase3 fill:#fff3e0 style Phase4 fill:#fce4ec style Phase5 fill:#f3e5f5 style Phase6 fill:#e0f2f1 style Phase7 fill:#fff8e1 style Phase8 fill:#efebe9

Asset Production Parallel Flow#

flowchart LR AssetSpec["Asset Specification<br/>(from GDD)"] subgraph Parallel["Parallel Production"] direction TB subgraph ModelPipe["3D Model Pipeline"] Generate3D["Generate 3D"] MeshProcess["Mesh Processing"] LODGen["LOD Generation"] end subgraph TexturePipe["Texture Pipeline"] GenTexture["Generate Textures"] PBRSetup["PBR Setup"] BakeMaps["Bake Maps"] end subgraph AnimPipe["Animation Pipeline"] Rigging["Auto-Rigging"] AnimGen["Animation Gen"] Retarget["Retargeting"] end subgraph AudioPipe["Audio Pipeline"] SFXGen["SFX Generation"] MusicGen["Music Generation"] VoiceSynth["Voice Synthesis"] end end AssetSpec --> ModelPipe AssetSpec --> TexturePipe AssetSpec --> AnimPipe AssetSpec --> AudioPipe ModelPipe --> Merge["Asset Merge"] TexturePipe --> Merge AnimPipe --> Merge AudioPipe --> Merge Merge --> Validate["Validation"] Validate --> Export["Export to Engine"]

Film Creation Pipeline#

Complete Phase Flow#

flowchart TB subgraph Dev["Phase 1: Development"] Concept["Film Concept"] Treatment["Treatment"] PitchDeck["Pitch Deck"] Budget["Budget Estimate"] Casting["Casting Suggestions"] Locations["Location Suggestions"] end subgraph PreProd["Phase 2: Pre-Production"] Screenplay["Screenplay"] Storyboard["Storyboards"] ShotList["Shot Lists"] Schedule["Production Schedule"] BudgetBreak["Budget Breakdown"] CallSheets["Call Sheets"] end subgraph Prod["Phase 3: Animation Production"] Environments["Environment Assets"] Characters["Character Assets"] Props["Prop Assets"] Animation["Animation"] LightingSetup["Lighting Setup"] Rendering["Rendering"] end subgraph Post["Phase 4: Post-Production"] EditAssembly["Edit Assembly"] VFX["VFX Integration"] ColorGrade["Color Grading"] SoundDesign["Sound Design"] MusicScore["Music Scoring"] FinalMix["Final Mix"] end subgraph Delivery["Phase 5: Delivery"] DCP["DCP Creation"] Transcode["Transcoding"] QC["Quality Control"] Distribute["Distribution"] Subtitles["Subtitles"] Dubbing["Dubbing"] end Dev --> |Greenlight| PreProd PreProd --> |Approved| Prod Prod --> |Rendered| Post Post --> |Final| Delivery Delivery --> Released["Released Film"] style Dev fill:#e3f2fd style PreProd fill:#e8f5e9 style Prod fill:#fff3e0 style Post fill:#fce4ec style Delivery fill:#f3e5f5

Post-Production Parallel Workflow#

flowchart TB RenderedShots["Rendered Shots"] subgraph PostParallel["Parallel Post-Production"] direction LR subgraph EditTrack["Edit Track"] RoughCut["Rough Cut"] FineCut["Fine Cut"] FinalCut["Final Cut"] end subgraph VFXTrack["VFX Track"] VFXPlanning["VFX Planning"] VFXExec["VFX Execution"] VFXComp["Compositing"] end subgraph AudioTrack["Audio Track"] Dialogue["Dialogue Edit"] Foley["Foley"] Ambience["Ambience"] end subgraph MusicTrack["Music Track"] Temp["Temp Score"] Compose["Composition"] Record["Recording"] end end RenderedShots --> PostParallel PostParallel --> ColorGrade["Color Grading"] ColorGrade --> FinalMix["Final Mix"] FinalMix --> Master["Master Delivery"]

Pipeline Infrastructure#

Executor Architecture#

flowchart TB subgraph Executor["PipelineExecutor"] PhaseQueue["Phase Queue"] subgraph Execution["Execution Engine"] StateManager["State Manager"] CheckpointMgr["Checkpoint Manager"] RetryPolicy["Retry Policy"] RollbackMgr["Rollback Manager"] end subgraph ResourceMgmt["Resource Management"] Scheduler["Resource Scheduler"] Allocator["Resource Allocator"] CostTracker["Cost Tracker"] Utilization["Utilization Monitor"] end subgraph Monitoring["Monitoring"] Monitor["Pipeline Monitor"] Alerts["Alert System"] Dashboard["Dashboard"] Analytics["Analytics"] end end Input["Phase Definition"] --> PhaseQueue PhaseQueue --> Execution Execution --> ResourceMgmt Execution --> Monitoring Execution --> |Success| Output["Phase Output"] Execution --> |Failure| Recovery["Recovery/Rollback"] style Executor fill:#e3f2fd

Retry Policy Strategies#

flowchart LR subgraph Strategies["Retry Strategies"] direction TB Linear["Linear Backoff<br/>delay × attempt"] Exponential["Exponential Backoff<br/>delay × 2^attempt"] Fixed["Fixed Delay<br/>constant delay"] end Failure["Phase Failure"] --> Check{"Attempts<br/>Remaining?"} Check --> |Yes| Strategies Check --> |No| Final["Final Failure"] Strategies --> Wait["Wait Period"] Wait --> Retry["Retry Execution"] Retry --> |Success| Continue["Continue Pipeline"] Retry --> |Failure| Check

Template System#

flowchart TB subgraph Templates["Pipeline Templates"] direction LR GameTemplate["Game Template"] FilmTemplate["Film Template"] CustomTemplate["Custom Template"] end subgraph Variables["Template Variables"] ProjectName["Project Name"] Genre["Genre"] Platform["Target Platform"] Quality["Quality Level"] end subgraph Overrides["Phase Overrides"] SkipPhase["Skip Phases"] CustomPhase["Custom Phases"] ModifyPhase["Modified Phases"] end Templates --> Instantiate["Template Instantiation"] Variables --> Instantiate Overrides --> Instantiate Instantiate --> Pipeline["Configured Pipeline"] Pipeline --> Execute["Execute"]

Quality Gates System#

Quality Gate Flow#

flowchart TB PhaseOutput["Phase Output"] subgraph Assessment["Quality Assessment"] Collect["Collect Artifacts"] Analyze["Run Analysis"] Score["Calculate Scores"] end PhaseOutput --> Assessment Assessment --> Decision{"Quality<br/>Decision"} Decision --> |"Score >= Threshold"| Pass["PASS"] Decision --> |"Score < Threshold"| Fail["FAIL"] Decision --> |"Manual Review"| Override["OVERRIDE"] Pass --> Continue["Continue Pipeline"] Fail --> Remediate["Remediation Phase"] Remediate --> |"Fixes Applied"| Assessment Remediate --> |"Cannot Fix"| Block["Block Pipeline"] Override --> |"With Justification"| Continue Override --> |"Rejected"| Block style Pass fill:#c8e6c9 style Fail fill:#ffcdd2 style Override fill:#fff9c4

Quality Gate Types#

flowchart LR subgraph NarrativeGate["Narrative Quality Gate"] direction TB ScriptQuality["Script Quality"] DialogueCheck["Dialogue Check"] CharConsist["Character Consistency"] PlotCoherence["Plot Coherence"] end subgraph VisualGate["Visual Quality Gate"] direction TB StyleConsist["Style Consistency"] TechSpecs["Technical Specs"] ColorPalette["Color Palette"] Optimization["Optimization"] end subgraph AudioGate["Audio Quality Gate"] direction TB LevelCheck["Level Check"] FreqResponse["Frequency Response"] SyncCheck["Sync Verification"] FormatCheck["Format Compliance"] end subgraph TechnicalGate["Technical Quality Gate"] direction TB Performance["Performance"] Stability["Stability"] Compatibility["Compatibility"] Memory["Memory Usage"] end subgraph GameplayGate["Gameplay Quality Gate"] direction TB Balance["Balance"] Difficulty["Difficulty Curve"] FunFactor["Fun Factor"] Pacing["Pacing"] end Input["Phase Output"] --> NarrativeGate Input --> VisualGate Input --> AudioGate Input --> TechnicalGate Input --> GameplayGate NarrativeGate --> Aggregate["Aggregate Score"] VisualGate --> Aggregate AudioGate --> Aggregate TechnicalGate --> Aggregate GameplayGate --> Aggregate Aggregate --> Final["Quality Decision"]

Issue Classification#

flowchart TB subgraph IssueTypes["Issue Types"] direction LR subgraph Critical["Critical"] Crash["Crashes"] DataLoss["Data Loss"] Security["Security Issues"] end subgraph Major["Major"] Broken["Broken Features"] PerfIssue["Performance Issues"] Regression["Regressions"] end subgraph Minor["Minor"] Visual["Visual Glitches"] Polish["Polish Issues"] Edge["Edge Cases"] end end Critical --> |"Must Fix"| Block["Block Release"] Major --> |"Should Fix"| Remediate["Remediation Required"] Minor --> |"Can Ship"| Track["Track for Future"] style Critical fill:#ffcdd2 style Major fill:#fff9c4 style Minor fill:#c8e6c9

Worker Queue System#

BullMQ Architecture#

flowchart TB subgraph API["API Layer"] REST["REST Endpoints"] GRPC["gRPC Services"] WS["WebSocket"] end subgraph Queues["BullMQ Queues"] AgentQ["yemaya:agent"] MediaQ["yemaya:media"] AssetQ["yemaya:asset"] Gen3DQ["yemaya:generation-3d"] NotifyQ["yemaya:notification"] end subgraph Workers["Worker Processes"] AgentW["Agent Worker<br/>3 retries, exponential"] MediaW["Media Worker<br/>2 retries, fixed"] AssetW["Asset Worker<br/>3 retries"] Gen3DW["3D Gen Worker<br/>3 retries, 7-day retention"] NotifyW["Notification Worker<br/>5 retries"] end subgraph Backend["Queue Backend"] Redis[(Redis)] end API --> |"Add Job"| Queues Queues --> Redis Redis --> Workers Workers --> |"Job Complete"| Events["Event Emission"] Workers --> |"Progress"| WS Events --> |"Webhook"| External["External Systems"] style Redis fill:#ffcdd2

Job Type Details#

flowchart LR subgraph AgentJobs["Agent Jobs"] direction TB ExecuteTask["agent.execute-task"] BatchProcess["agent.batch-process"] GenScript["agent.generate-script"] AnalyzeContent["agent.analyze-content"] Collaborate["agent.collaborate"] end subgraph MediaJobs["Media Jobs"] direction TB TranscodeV["media.transcode-video"] TranscodeA["media.transcode-audio"] ExtractFrames["media.extract-frames"] GenThumb["media.generate-thumbnail"] Concatenate["media.concatenate"] end subgraph AssetJobs["Asset Jobs"] direction TB OptimizeImg["asset.optimize-image"] GenVariants["asset.generate-variants"] ExtractMeta["asset.extract-metadata"] Validate["asset.validate"] end subgraph Gen3DJobs["3D Generation Jobs"] direction TB TextTo3D["generation-3d.text-to-3d"] ImageTo3D["generation-3d.image-to-3d"] MeshProcess["generation-3d.mesh-process"] AutoRig["generation-3d.auto-rig"] TextureGen["generation-3d.texture-generate"] end

Job Lifecycle#

stateDiagram-v2 [*] --> Waiting: Job Added Waiting --> Active: Worker Claims Active --> Completed: Success Active --> Failed: Error Failed --> Waiting: Retry Failed --> [*]: Max Retries Completed --> [*]: Done Active --> Delayed: Rate Limited Delayed --> Waiting: Delay Expires

3D Generation Pipeline#

Provider Architecture#

flowchart TB subgraph Input["Input Types"] TextPrompt["Text Prompt"] Image["Reference Image"] MultiView["Multi-View Images"] end subgraph Providers["3D Generation Providers"] Rodin["Rodin"] Meshy["Meshy"] Tripo["Tripo"] Trellis["Trellis"] Hunyuan["Hunyuan"] Threedfy["Threedfy"] Local["Local Model"] end subgraph Processing["Mesh Processing"] Optimize["Optimization"] Cleanup["Mesh Cleanup"] UVUnwrap["UV Unwrapping"] LODGen["LOD Generation"] end subgraph Texturing["Texture Processing"] PBRGen["PBR Generation"] TextureApply["Texture Application"] BakeMaps["Map Baking"] end subgraph Export["Export Formats"] GLB["GLB"] GLTF["GLTF"] FBX["FBX"] OBJ["OBJ"] USD["USD/USDZ"] end Input --> |"Provider Selection"| Providers Providers --> |"Raw Mesh"| Processing Processing --> |"Clean Mesh"| Texturing Texturing --> |"Textured Model"| Export style Providers fill:#e3f2fd style Processing fill:#e8f5e9 style Texturing fill:#fff3e0

Quality Levels#

flowchart LR subgraph QualityLevels["Quality Levels"] Draft["Draft<br/>Low poly, fast"] Standard["Standard<br/>Balanced"] High["High<br/>Detailed"] Ultra["Ultra<br/>Maximum quality"] end Draft --> |"~30s"| DraftOut["Preview Model"] Standard --> |"~2min"| StdOut["Production Ready"] High --> |"~5min"| HighOut["High Detail"] Ultra --> |"~10min"| UltraOut["Hero Asset"] style Draft fill:#fff9c4 style Standard fill:#c8e6c9 style High fill:#bbdefb style Ultra fill:#e1bee7

Complete Generation Flow#

sequenceDiagram participant Client participant API participant Queue participant Worker participant Provider participant Storage Client->>API: POST /3d/generate API->>Queue: Add Job API-->>Client: { jobId, status: "queued" } Queue->>Worker: Process Job Worker->>Provider: Generate 3D Model loop Progress Updates Provider-->>Worker: Progress % Worker-->>API: WebSocket Update API-->>Client: Progress Event end Provider-->>Worker: Raw Model Worker->>Worker: Mesh Processing Worker->>Worker: Texture Processing Worker->>Storage: Upload Assets Worker->>API: Job Complete API-->>Client: Completion Event Client->>API: GET /3d/jobs/{id} API-->>Client: { status: "completed", assets: [...] }

Agent Orchestration System#

Crew System Architecture#

flowchart TB subgraph CrewDefinition["Crew Definition"] Roles["Agent Roles"] Goals["Role Goals"] Tools["Available Tools"] Backstory["Agent Backstory"] end subgraph TaskManagement["Task Management"] Tasks["Task Queue"] Dependencies["Task Dependencies"] Priority["Priority Assignment"] end subgraph ExecutionModes["Execution Modes"] Sequential["Sequential<br/>One-by-one"] Parallel["Parallel<br/>Concurrent"] Hierarchical["Hierarchical<br/>Manager delegates"] end subgraph Communication["Communication"] Bus["Crew Message Bus"] Handoff["Agent Handoffs"] Conflict["Conflict Resolution"] end CrewDefinition --> TaskManagement TaskManagement --> ExecutionModes ExecutionModes --> Communication Communication --> Results["Aggregated Results"] style CrewDefinition fill:#e3f2fd style ExecutionModes fill:#e8f5e9

Workflow Engine#

flowchart TB subgraph WorkflowDef["Workflow Definition"] StateGraph["State Graph"] Nodes["Workflow Nodes"] Edges["Conditional Edges"] Transitions["State Transitions"] end subgraph Executor["Workflow Executor"] CurrentState["Current State"] EventStream["Event Stream"] ErrorHandler["Error Handler"] end subgraph Routing["Routing Logic"] Conditions["Condition Evaluation"] NextState["Next State Selection"] Fallback["Fallback Paths"] end WorkflowDef --> Executor Executor --> Routing Routing --> |"State Change"| Executor Executor --> Output["Workflow Output"] style WorkflowDef fill:#fff3e0

RAG Pipeline#

flowchart LR subgraph Ingestion["Document Ingestion"] Documents["Source Documents"] Chunker["Document Chunker"] Embedder["Embedding Generator"] end subgraph Storage["Vector Storage"] VectorStore["In-Memory Vector Store"] Index["Similarity Index"] end subgraph Retrieval["Retrieval"] Query["User Query"] Search["Semantic Search"] TopK["Top-K Results"] end subgraph Generation["Augmented Generation"] Context["Context Assembly"] Prompt["Prompt Construction"] LLM["LLM Generation"] end Documents --> Chunker --> Embedder --> VectorStore Query --> Search --> VectorStore VectorStore --> TopK --> Context Context --> Prompt --> LLM --> Response["Generated Response"]

Delegation System#

flowchart TB IncomingTask["Incoming Task"] subgraph Routing["Task Routing"] Rules["Routing Rules"] SkillMatch["Skill Matching"] LoadBalance["Load Balancing"] Complexity["Complexity Analysis"] end subgraph Agents["Agent Pool"] Specialist1["Specialist Agent 1"] Specialist2["Specialist Agent 2"] Generalist["Generalist Agent"] Fallback["Fallback Agent"] end subgraph Fallbacks["Fallback Strategies"] Escalate["Escalate to Human"] Retry["Retry with Different Agent"] Decompose["Decompose Task"] end IncomingTask --> Routing Routing --> |"Best Match"| Agents Agents --> |"Success"| Result["Task Result"] Agents --> |"Failure"| Fallbacks Fallbacks --> |"Retry"| Routing Fallbacks --> |"Human Review"| HITL["Human-in-the-Loop"]

Data Flow Patterns#

Pattern 1: Linear Phase Flow#

flowchart LR Input["Phase Input"] subgraph Phase["Phase Execution"] Process["Processing"] Output["Phase Output"] Events["Event Emission"] end subgraph QG["Quality Gate"] Assess["Assessment"] Decision{"Pass?"} end Input --> Phase Phase --> QG Decision --> |"Yes"| NextPhase["Next Phase"] Decision --> |"No"| Remediate["Remediation"] Remediate --> Phase

Pattern 2: Parallel Production#

flowchart TB Spec["Asset Specification"] subgraph Parallel["Parallel Execution"] direction LR Track1["Track 1: Models"] Track2["Track 2: Textures"] Track3["Track 3: Animation"] Track4["Track 4: Audio"] end Spec --> Track1 Spec --> Track2 Spec --> Track3 Spec --> Track4 Track1 --> Merge["Merge Results"] Track2 --> Merge Track3 --> Merge Track4 --> Merge Merge --> Validate["Validation"] Validate --> Next["Next Phase"]

Pattern 3: Event-Driven Queue#

flowchart TB subgraph Producer["Job Producer"] API["API Endpoint"] Scheduler["Scheduler"] Trigger["Event Trigger"] end subgraph Queue["Job Queue"] Pending["Pending Jobs"] Active["Active Jobs"] Completed["Completed Jobs"] Failed["Failed Jobs"] end subgraph Consumer["Job Consumer"] Worker["Worker Process"] Handler["Job Handler"] Result["Result Processor"] end subgraph Notifications["Notifications"] WebSocket["WebSocket"] Webhook["Webhook"] Email["Email"] end Producer --> Queue Queue --> Consumer Consumer --> Notifications

Pattern 4: Agent Collaboration#

flowchart TB Task["Complex Task"] subgraph Decomposition["Task Decomposition"] Analyze["Analyze Task"] Split["Split into Subtasks"] Assign["Assign to Agents"] end subgraph Execution["Agent Execution"] Agent1["Agent 1"] Agent2["Agent 2"] Agent3["Agent 3"] subgraph Communication["Inter-Agent Comm"] Handoff["Task Handoff"] Share["Knowledge Sharing"] Resolve["Conflict Resolution"] end end subgraph Aggregation["Result Aggregation"] Collect["Collect Results"] Merge["Merge Outputs"] Validate["Validate Consistency"] end Task --> Decomposition Decomposition --> Execution Execution --> Aggregation Aggregation --> Final["Final Result"]

Pattern 5: Checkpoint/Recovery#

flowchart TB subgraph Execution["Pipeline Execution"] Phase1["Phase 1"] --> CP1["Checkpoint 1"] CP1 --> Phase2["Phase 2"] Phase2 --> CP2["Checkpoint 2"] CP2 --> Phase3["Phase 3"] Phase3 --> CP3["Checkpoint 3"] end subgraph Storage["Checkpoint Storage"] State["Execution State"] Artifacts["Generated Artifacts"] Context["Context Data"] end subgraph Recovery["Recovery Flow"] Failure["Failure Detected"] LoadCP["Load Checkpoint"] Restore["Restore State"] Resume["Resume Execution"] end CP1 --> Storage CP2 --> Storage CP3 --> Storage Failure --> LoadCP Storage --> LoadCP LoadCP --> Restore --> Resume Resume --> |"From Phase 2"| Phase2 style CP1 fill:#c8e6c9 style CP2 fill:#c8e6c9 style CP3 fill:#c8e6c9

Component Interaction Summary#

Cross-System Communication#

flowchart TB subgraph Systems["Core Systems"] direction LR Pipelines["Pipelines"] Agents["Agents"] Workers["Workers"] QualityGates["Quality Gates"] end subgraph Communication["Communication Channels"] Events["TypedEventEmitter"] Queues["BullMQ Queues"] GRPC["gRPC Streaming"] WS["WebSocket"] Webhooks["Webhooks"] end subgraph State["State Management"] DB[(PostgreSQL)] Cache[(Redis)] Storage[(MinIO)] end Systems --> Communication Communication --> State Events --> |"Internal Events"| Systems Queues --> |"Async Jobs"| Systems GRPC --> |"Service Calls"| Systems WS --> |"Real-time Updates"| Clients["Clients"] Webhooks --> |"External Notifications"| External["External Systems"]

Key Files Reference#

Component Location Lines
Game Pipeline packages/agents/src/pipelines/game-pipeline.ts 3,336
Film Pipeline packages/agents/src/pipelines/film-pipeline.ts 2,345
Infrastructure packages/agents/src/pipelines/infrastructure.ts 1,716
Quality Gates packages/agents/src/pipelines/quality-gates.ts 3,033
Automated Testing packages/agents/src/pipelines/automated-testing.ts 2,154
Crew System packages/agents/src/orchestration/crew.ts ~500
Workflow Engine packages/agents/src/orchestration/workflow.ts ~400
Workers apps/workers/src/workers/ Various
3D Generation packages/3d-generation/src/ Various

See Also#