This guide explains how to integrate between domains in the Oshun platform, covering communication patterns, event-driven architecture, and best practices.
Table of Contents#
- Overview
- Integration Principles
- Communication Patterns
- Domain Clients
- Event-Driven Integration
- Common Integration Scenarios
- Data Contracts
- Error Handling
- Testing Integrations
- Best Practices
- Anti-Patterns
Overview#
Oshun follows a domain-driven design with multiple bounded contexts. Each domain owns specific data and capabilities, communicating with others through well-defined interfaces. The 13 application domains are: Iris, Lilith, Yemaya, Isis, Sophia, Hathor, Bellona, Tara, Veritas, Psyche, Nyx, Aja, and Aphrodite. Additional library-only domains (Aje, Themis, Galatea, Shakti, Nous, Uzume, etc.) provide cross-cutting capabilities.
┌─────────────────────────────────────────────────────────────────────────┐
│ Cross-Domain Communication │
│ │
│ ┌─────────┐ REST/gRPC ┌─────────┐ REST/gRPC ┌───────┐│
│ │ Yemaya │◄──────────────────►│ Isis │◄─────────────────►│Bellona││
│ │Creative │ (Synchronous) │Generate │ (Synchronous) │ Build ││
│ │ Studio │ │ Factory │ │Bridge ││
│ └────┬────┘ └────┬────┘ └───┬───┘│
│ │ │ │ │
│ │ ┌────────────────────┼────────────────────┐ │ │
│ │ │ │ │ │ │
│ └─────────┼────────────────────┼────────────────────┼───────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌────────────────────────────────────────────────────┐ │
│ │ EVENT BUS (Redis Streams) │ │
│ │ (Asynchronous) │ │
│ └────────────────────────────────────────────────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │ Lilith │ │ Sophia │ │ Hathor │ │
│ │Conscious│ │Knowledge│ │ World │ │
│ │ Exp. │ │ RAG │ │Building │ │
│ └─────────┘ └─────────┘ └─────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
Domain Responsibilities#
| Domain | Owns | Provides To Others |
|---|---|---|
| Iris | Conversations, Memory, Tools | AI assistant, reasoning |
| Lilith | Users, Sessions, Personas | User profiles, consciousness |
| Yemaya | Projects, Assets, Teams | Project context, asset metadata |
| Isis | Generated assets, GPU Jobs | AI generation, asset creation |
| Sophia | Documents, Embeddings | RAG search, citations |
| Hathor | Worlds, NPCs, Quests | World data, narrative structures |
| Bellona | Builds, Exports | Engine artifacts, runtime systems |
| Tara | Meditations, Courses, Progress | Meditation content, analytics |
| Veritas | Articles, Claims, Sources | Fact-checking, news verification |
| Psyche | Assessments, Sessions, Models | Mental health services, analytics |
| Nyx | Star catalogs, Simulations | Astronomical data, education |
| Aja | Motion data, Animations | Motion AI, pose estimation |
| Aphrodite | Streams, Broadcasts, Chat | Live streaming, real-time comms |
Integration Principles#
1. Domain Isolation#
Each domain owns its data and exposes it only through APIs:
// ✅ CORRECT: Access via domain client
import { createIsisClient } from '@isis/client';
const isis = createIsisClient({ baseUrl: process.env.ISIS_API_URL });
const job = await isis.generation.create({ prompt: 'A sunset' });
// ❌ INCORRECT: Direct database access across domains
// NEVER DO THIS - it violates domain boundaries
// import { prisma } from '@isis/database';
// const job = await prisma.generationJob.create({ ... });
2. Single Source of Truth#
Each entity has exactly one owning domain:
// User data is owned by Auth/Lilith
const user = await authClient.users.get(userId);
// Project data is owned by Yemaya
const project = await yemayaClient.projects.get(projectId);
// Generated assets are owned by Isis
const asset = await isisClient.assets.get(assetId);
3. API-First Design#
All cross-domain communication uses documented APIs:
// Internal gRPC for service-to-service
import { SophiaGrpcClient } from '@sophia/proto';
const sophia = new SophiaGrpcClient('sophia-service:50051');
const results = await sophia.search({ query: 'meditation techniques' });
// REST for external/public access
const response = await fetch(`${SOPHIA_API}/v1/search`, {
method: 'POST',
headers: { Authorization: `Bearer ${token}` },
body: JSON.stringify({ query: 'meditation techniques' }),
});
4. Event-Driven Coordination#
Cross-domain workflows use events for loose coupling:
// Producer (Isis): Publish when asset is generated
await eventBus.publish('isis.asset.generated', {
assetId: 'asset-123',
projectId: 'project-456',
type: 'image',
url: 'https://storage.oshun.io/...',
});
// Consumer (Yemaya): React to asset generation
eventBus.subscribe('isis.asset.generated', async (event) => {
await yemayaService.attachAssetToProject(
event.payload.projectId,
event.payload.assetId
);
});
Communication Patterns#
Synchronous Communication#
Use for request-response patterns where immediate response is needed.
REST APIs#
import { createHttpClient } from '@oshun/http-client';
// Create configured client with retries and timeouts
const client = createHttpClient({
baseUrl: process.env.ISIS_API_URL,
timeout: 30000,
retries: 3,
headers: {
Authorization: `Bearer ${await getServiceToken()}`,
},
});
// Make requests
const response = await client.post('/v1/generation/jobs', {
prompt: 'A beautiful sunset over mountains',
model: 'stable-diffusion-xl',
parameters: { steps: 50, guidance: 7.5 },
});
gRPC (Internal Services)#
import { credentials } from '@grpc/grpc-js';
import { IsisClient } from '@isis/proto';
// Create gRPC client
const isis = new IsisClient(
'isis-service.isis.svc.cluster.local:50051',
credentials.createInsecure()
);
// Unary call
const job = await isis.createJob({
prompt: 'A sunset',
model: 'stable-diffusion-xl',
});
// Server streaming
const stream = isis.streamJobProgress({ jobId: job.id });
for await (const update of stream) {
console.log(`Progress: ${update.progress}%`);
}
Asynchronous Communication#
Use for fire-and-forget, long-running, or multi-consumer scenarios.
Event Bus (Redis Streams)#
import { EventBus } from '@oshun/event-bus';
const eventBus = new EventBus({
redis: { url: process.env.REDIS_URL },
consumerGroup: 'yemaya-service',
});
// Publishing events
await eventBus.publish('yemaya.project.created', {
projectId: 'proj-123',
ownerId: 'user-456',
name: 'My New Project',
});
// Subscribing to events
eventBus.subscribe('isis.asset.*', async (event) => {
switch (event.type) {
case 'isis.asset.generated':
await handleAssetGenerated(event.payload);
break;
case 'isis.asset.failed':
await handleAssetFailed(event.payload);
break;
}
});
// Start consuming
await eventBus.start();
Job Queues (BullMQ)#
import { createQueue, createWorker } from '@oshun/queue';
// Producer: Add job to queue
const exportQueue = createQueue('bellona-export');
await exportQueue.add(
'export-project',
{
projectId: 'proj-123',
format: 'godot',
version: '4.2',
},
{
priority: 1,
attempts: 3,
backoff: { type: 'exponential', delay: 1000 },
}
);
// Consumer: Process jobs
const worker = createWorker('bellona-export', async (job) => {
const { projectId, format, version } = job.data;
// Report progress
await job.updateProgress(10);
// Do the export
const result = await exportProject(projectId, format, version);
await job.updateProgress(100);
return result;
});
Domain Clients#
Each domain provides a TypeScript client for integration.
Installing Domain Clients#
// Domain clients are available as workspace packages
import { createIsisClient } from '@isis/client';
import { createSophiaClient } from '@sophia/client';
import { createHathorClient } from '@hathor/client';
import { createBellonaClient } from '@bellona/client';
import { createYemayaClient } from '@yemaya/client';
import { createLilithClient } from '@lilith/sdk';
Client Configuration#
import { createIsisClient } from '@isis/client';
// Basic configuration
const isis = createIsisClient({
baseUrl: process.env.ISIS_API_URL || 'http://localhost:4001',
});
// Full configuration
const isis = createIsisClient({
baseUrl: process.env.ISIS_API_URL,
timeout: 30000,
retries: 3,
// Authentication
auth: {
type: 'service-account',
credentials: {
clientId: process.env.SERVICE_CLIENT_ID,
clientSecret: process.env.SERVICE_CLIENT_SECRET,
},
},
// Request interceptors
interceptors: {
request: async (config) => {
config.headers['X-Correlation-ID'] = getCorrelationId();
return config;
},
response: async (response) => {
trackMetrics(response);
return response;
},
},
});
Client Usage Examples#
Isis Client (Generation)#
import { createIsisClient } from '@isis/client';
const isis = createIsisClient({ baseUrl: process.env.ISIS_API_URL });
// Create a generation job
const job = await isis.generation.create({
type: 'image',
prompt: 'A majestic dragon flying over a castle',
model: 'stable-diffusion-xl',
parameters: {
width: 1024,
height: 1024,
steps: 50,
guidance: 7.5,
},
});
// Poll for completion
const result = await isis.generation.waitForCompletion(job.id, {
pollingInterval: 2000,
timeout: 300000,
});
// Get generated asset
const asset = await isis.assets.get(result.assetId);
console.log(`Generated: ${asset.url}`);
Sophia Client (Knowledge)#
import { createSophiaClient } from '@sophia/client';
const sophia = createSophiaClient({ baseUrl: process.env.SOPHIA_API_URL });
// Ingest a document
const document = await sophia.documents.ingest({
source: 'https://example.com/research-paper.pdf',
collection: 'research',
metadata: {
author: 'Dr. Smith',
year: 2024,
},
});
// Search with RAG
const results = await sophia.search({
query: 'What are the key findings about meditation?',
collections: ['research'],
limit: 10,
includeContext: true,
});
// Cite sources
for (const result of results.hits) {
console.log(`${result.content} [${result.citation}]`);
}
Hathor Client (Worldbuilding)#
import { createHathorClient } from '@hathor/client';
const hathor = createHathorClient({ baseUrl: process.env.HATHOR_API_URL });
// Create a world
const world = await hathor.worlds.create({
name: 'Veilborn',
genre: 'dark_fantasy',
description: 'A realm where shadows hold ancient secrets',
});
// Add an NPC
const npc = await hathor.characters.create({
worldId: world.id,
name: 'Elder Morvain',
role: 'quest_giver',
personality: {
traits: ['wise', 'mysterious', 'cautious'],
voice: 'gravelly and slow',
},
});
// Generate NPC dialogue
const dialogue = await hathor.dialogue.generate({
characterId: npc.id,
context: 'Player asks about the ancient prophecy',
mood: 'ominous',
});
Bellona Client (Build)#
import { createBellonaClient } from '@bellona/client';
const bellona = createBellonaClient({ baseUrl: process.env.BELLONA_API_URL });
// Create an export job
const exportJob = await bellona.exports.create({
projectId: 'proj-123',
engine: 'godot',
version: '4.2',
format: 'gltf',
options: {
includeAnimations: true,
lodLevels: [1.0, 0.5, 0.25],
textureSize: 2048,
},
});
// Monitor progress
bellona.exports.onProgress(exportJob.id, (progress) => {
console.log(`Export: ${progress.stage} - ${progress.percent}%`);
});
// Download artifacts
const artifacts = await bellona.exports.getArtifacts(exportJob.id);
for (const artifact of artifacts) {
await downloadFile(artifact.url, artifact.filename);
}
Event-Driven Integration#
Event Schema#
All domain events follow a standard envelope:
interface DomainEvent<T = unknown> {
// Metadata
id: string; // Unique event ID (UUID)
type: string; // Event type (e.g., "isis.asset.generated")
source: string; // Source domain
timestamp: string; // ISO 8601 timestamp
correlationId: string; // Request trace ID
// Content
payload: T; // Event-specific payload
metadata: Record<string, unknown>;
}
Event Naming Convention#
Events follow the pattern: {domain}.{entity}.{action}
isis.asset.generated
isis.job.started
isis.job.completed
isis.job.failed
sophia.document.ingested
sophia.index.updated
hathor.world.created
hathor.world.published
hathor.simulation.completed
bellona.build.started
bellona.build.completed
bellona.export.ready
yemaya.project.created
yemaya.asset.attached
yemaya.collaboration.updated
lilith.meditation.started
lilith.meditation.completed
lilith.achievement.earned
Event Catalog#
| Event | Payload | When Emitted |
|---|---|---|
isis.asset.generated |
{ assetId, jobId, type, url, metadata } |
Asset generation completes |
isis.job.failed |
{ jobId, error, attempts } |
Generation fails after retries |
sophia.document.ingested |
{ documentId, collection, chunks } |
Document fully processed |
sophia.index.updated |
{ indexId, documentCount } |
Index rebuild completes |
hathor.world.published |
{ worldId, version, entities } |
World version published |
bellona.build.completed |
{ buildId, engine, artifacts } |
Build job completes |
yemaya.project.created |
{ projectId, ownerId, name } |
New project created |
lilith.meditation.completed |
{ sessionId, userId, duration } |
Meditation session ends |
Publishing Events#
import { EventBus } from '@oshun/event-bus';
import { withSpan } from '@oshun/tracing';
const eventBus = new EventBus({ redis: { url: process.env.REDIS_URL } });
// Simple publish
await eventBus.publish('isis.asset.generated', {
assetId: 'asset-123',
jobId: 'job-456',
type: 'image',
url: 'https://storage.oshun.io/assets/asset-123.png',
metadata: {
width: 1024,
height: 1024,
model: 'stable-diffusion-xl',
},
});
// Publish with correlation (for tracing)
await withSpan('publish-event', async (span) => {
await eventBus.publish('isis.asset.generated', payload, {
correlationId: span.spanContext().traceId,
metadata: {
userId: context.userId,
projectId: context.projectId,
},
});
});
// Publish multiple events atomically
await eventBus.publishBatch([
{ type: 'isis.job.completed', payload: { jobId: 'job-1' } },
{ type: 'isis.asset.generated', payload: { assetId: 'asset-1' } },
]);
Subscribing to Events#
import { EventBus } from '@oshun/event-bus';
import { createLogger } from '@oshun/logging';
const logger = createLogger('yemaya-event-handler');
const eventBus = new EventBus({
redis: { url: process.env.REDIS_URL },
consumerGroup: 'yemaya-service',
});
// Subscribe to specific event
eventBus.subscribe('isis.asset.generated', async (event) => {
logger.info('Asset generated', { assetId: event.payload.assetId });
await attachAssetToProject(event.payload);
});
// Subscribe to pattern (all Isis events)
eventBus.subscribe('isis.*', async (event) => {
logger.debug('Isis event received', { type: event.type });
});
// Subscribe with error handling
eventBus.subscribe(
'sophia.document.ingested',
async (event) => {
try {
await indexDocumentForProject(event.payload);
} catch (error) {
logger.error('Failed to index document', { error, event });
// Event will be retried or sent to DLQ
throw error;
}
},
{
maxRetries: 3,
deadLetterQueue: 'yemaya-dlq',
}
);
// Start consuming
await eventBus.start();
// Graceful shutdown
process.on('SIGTERM', async () => {
await eventBus.stop();
});
Event Handlers in Domains#
Each domain should have an event-handlers library:
// libs/yemaya/event-handlers/src/index.ts
import { EventBus } from '@oshun/event-bus';
import { createLogger } from '@oshun/logging';
import { handleIsisAssetGenerated } from './handlers/isis';
import { handleSophiaDocumentIngested } from './handlers/sophia';
import { handleHathorWorldPublished } from './handlers/hathor';
import { handleBellonaBuildCompleted } from './handlers/bellona';
const logger = createLogger('yemaya-event-handlers');
export function registerEventHandlers(eventBus: EventBus) {
// Isis events
eventBus.subscribe('isis.asset.generated', handleIsisAssetGenerated);
eventBus.subscribe('isis.job.failed', handleIsisJobFailed);
// Sophia events
eventBus.subscribe('sophia.document.ingested', handleSophiaDocumentIngested);
// Hathor events
eventBus.subscribe('hathor.world.published', handleHathorWorldPublished);
// Bellona events
eventBus.subscribe('bellona.build.completed', handleBellonaBuildCompleted);
logger.info('Registered all event handlers');
}
// libs/yemaya/event-handlers/src/handlers/isis.ts
import { DomainEvent } from '@oshun/event-bus';
import { createYemayaService } from '@yemaya/service';
interface AssetGeneratedPayload {
assetId: string;
jobId: string;
type: string;
url: string;
metadata: Record<string, unknown>;
}
export async function handleIsisAssetGenerated(
event: DomainEvent<AssetGeneratedPayload>
) {
const yemayaService = createYemayaService();
const { assetId, metadata } = event.payload;
// Find associated project from correlation
const projectId = event.metadata.projectId as string;
if (!projectId) {
return; // Not project-related, skip
}
// Attach asset to project
await yemayaService.assets.attach({
projectId,
assetId,
type: event.payload.type,
url: event.payload.url,
metadata,
});
}
Common Integration Scenarios#
Scenario 1: Generate Asset for Project (Yemaya → Isis)#
// In Yemaya service
import { createIsisClient } from '@isis/client';
import { EventBus } from '@oshun/event-bus';
export class ProjectAssetService {
private isis = createIsisClient({ baseUrl: process.env.ISIS_API_URL });
private eventBus: EventBus;
async generateAssetForProject(projectId: string, request: GenerateRequest) {
// 1. Create generation job in Isis
const job = await this.isis.generation.create({
...request,
metadata: {
projectId,
source: 'yemaya',
},
});
// 2. Track the pending generation
await this.trackPendingGeneration(projectId, job.id);
// 3. Return immediately (async completion via events)
return { jobId: job.id, status: 'processing' };
}
// Called by event handler when isis.asset.generated is received
async onAssetGenerated(event: IsisAssetGeneratedEvent) {
const { assetId, metadata } = event.payload;
const projectId = metadata.projectId as string;
if (!projectId) return;
// Create project asset reference
await this.attachAssetToProject(projectId, {
externalId: assetId,
type: event.payload.type,
url: event.payload.url,
metadata: event.payload.metadata,
});
// Notify project collaborators
await this.eventBus.publish('yemaya.asset.attached', {
projectId,
assetId,
});
}
}
Scenario 2: Search Knowledge for World (Hathor → Sophia)#
// In Hathor service
import { createSophiaClient } from '@sophia/client';
export class WorldResearchService {
private sophia = createSophiaClient({ baseUrl: process.env.SOPHIA_API_URL });
async groundWorldInResearch(worldId: string, topic: string) {
// 1. Search Sophia for relevant research
const results = await this.sophia.search({
query: topic,
collections: ['mythology', 'history', 'folklore'],
limit: 20,
filters: {
verified: true,
},
});
// 2. Extract key facts
const facts = results.hits.map((hit) => ({
content: hit.content,
citation: hit.citation,
confidence: hit.score,
}));
// 3. Attach to world as lore sources
await this.attachLoreSources(worldId, facts);
return facts;
}
async generateGroundedNpcDialogue(npcId: string, context: string) {
const npc = await this.getNpc(npcId);
const world = await this.getWorld(npc.worldId);
// 1. Search for relevant world knowledge
const knowledge = await this.sophia.search({
query: context,
collections: [`world-${world.id}`],
limit: 5,
});
// 2. Generate dialogue with grounding
const dialogue = await this.generateDialogue(npc, context, {
groundingContext: knowledge.hits.map((h) => h.content).join('\n'),
});
return dialogue;
}
}
Scenario 3: Export World to Engine (Hathor → Bellona)#
// In Hathor service
import { createBellonaClient } from '@bellona/client';
import { EventBus } from '@oshun/event-bus';
export class WorldExportService {
private bellona = createBellonaClient({
baseUrl: process.env.BELLONA_API_URL,
});
async exportWorldToEngine(
worldId: string,
engine: 'godot' | 'unreal' | 'unity'
) {
// 1. Compile world data
const worldData = await this.compileWorldForExport(worldId);
// 2. Create export job in Bellona
const exportJob = await this.bellona.exports.create({
source: 'hathor',
sourceId: worldId,
engine,
data: worldData,
options: {
includeDialogue: true,
includeQuests: true,
includeNpcs: true,
format: engine === 'godot' ? 'tres' : 'json',
},
});
return { exportJobId: exportJob.id };
}
private async compileWorldForExport(worldId: string) {
const [world, npcs, quests, locations, timeline] = await Promise.all([
this.getWorld(worldId),
this.getNpcs(worldId),
this.getQuests(worldId),
this.getLocations(worldId),
this.getTimeline(worldId),
]);
return {
world,
entities: { npcs, locations },
narrative: { quests, timeline },
};
}
}
// Event handler for export completion
async function handleBellonaExportReady(event: BellonaExportReadyEvent) {
const { exportId, sourceId, artifacts } = event.payload;
// Update world with export artifacts
await hathorService.updateWorldExports(sourceId, {
exportId,
artifacts,
exportedAt: new Date(),
});
}
Scenario 4: Full Pipeline (Yemaya → Isis → Bellona)#
// Orchestrating a full asset generation and export pipeline
import { createIsisClient } from '@isis/client';
import { createBellonaClient } from '@bellona/client';
export class AssetPipelineService {
async generateAndExport(projectId: string, request: PipelineRequest) {
// Step 1: Generate asset in Isis
const job = await this.isis.generation.create({
type: request.assetType,
prompt: request.prompt,
model: request.model,
metadata: { projectId, pipeline: true },
});
// Step 2: Wait for generation
const generated = await this.isis.generation.waitForCompletion(job.id);
// Step 3: Export for target engine
const exportJob = await this.bellona.exports.create({
source: 'isis',
sourceId: generated.assetId,
engine: request.targetEngine,
options: request.exportOptions,
});
// Step 4: Wait for export
const exported = await this.bellona.exports.waitForCompletion(exportJob.id);
return {
assetId: generated.assetId,
exportId: exported.id,
artifacts: exported.artifacts,
};
}
}
Scenario 5: Real-time Collaboration (Yemaya with Events)#
// Real-time sync using events
import { EventBus } from '@oshun/event-bus';
import { WebSocketServer } from '@oshun/websocket';
export class CollaborationService {
constructor(
private eventBus: EventBus,
private wsServer: WebSocketServer
) {
// Bridge domain events to WebSocket clients
this.eventBus.subscribe(
'yemaya.asset.*',
this.broadcastToProject.bind(this)
);
this.eventBus.subscribe(
'isis.asset.generated',
this.broadcastToProject.bind(this)
);
this.eventBus.subscribe(
'hathor.world.updated',
this.broadcastToProject.bind(this)
);
}
private async broadcastToProject(event: DomainEvent) {
const projectId = event.metadata.projectId as string;
if (!projectId) return;
// Send to all connected collaborators
await this.wsServer.broadcast(`project:${projectId}`, {
type: event.type,
payload: event.payload,
timestamp: event.timestamp,
});
}
}
Data Contracts#
Defining Contracts#
Cross-domain data contracts are defined in libs/contracts/:
// libs/contracts/types/src/asset.ts
import { z } from 'zod';
export const AssetReferenceSchema = z.object({
id: z.string().uuid(),
type: z.enum(['image', 'video', 'audio', '3d', 'text']),
url: z.string().url(),
mimeType: z.string(),
size: z.number().positive(),
metadata: z.record(z.unknown()).optional(),
createdAt: z.string().datetime(),
});
export type AssetReference = z.infer<typeof AssetReferenceSchema>;
// libs/contracts/events/src/isis.ts
import { z } from 'zod';
import { DomainEventSchema } from './base';
import { AssetReferenceSchema } from '@oshun/contracts-types';
export const IsisAssetGeneratedEventSchema = DomainEventSchema.extend({
type: z.literal('isis.asset.generated'),
payload: z.object({
assetId: z.string().uuid(),
jobId: z.string().uuid(),
asset: AssetReferenceSchema,
generation: z.object({
model: z.string(),
prompt: z.string(),
parameters: z.record(z.unknown()),
}),
}),
});
export type IsisAssetGeneratedEvent = z.infer<
typeof IsisAssetGeneratedEventSchema
>;
Validating Contracts#
import { IsisAssetGeneratedEventSchema } from '@oshun/contracts-events';
eventBus.subscribe('isis.asset.generated', async (event) => {
// Validate event against contract
const validatedEvent = IsisAssetGeneratedEventSchema.parse(event);
// Now TypeScript knows the exact shape
console.log(validatedEvent.payload.asset.url);
});
Version Compatibility#
// Support multiple event versions
import { z } from 'zod';
const AssetGeneratedV1Schema = z.object({
assetId: z.string(),
url: z.string(),
});
const AssetGeneratedV2Schema = z.object({
assetId: z.string(),
asset: AssetReferenceSchema,
});
// Union for backwards compatibility
const AssetGeneratedPayloadSchema = z.union([
AssetGeneratedV2Schema,
AssetGeneratedV1Schema,
]);
// Transform V1 to V2 format
function normalizeAssetEvent(payload: unknown) {
const parsed = AssetGeneratedPayloadSchema.parse(payload);
if ('asset' in parsed) {
return parsed; // V2 format
}
// Convert V1 to V2
return {
assetId: parsed.assetId,
asset: {
id: parsed.assetId,
url: parsed.url,
// ... fill defaults
},
};
}
Error Handling#
Cross-Domain Errors#
import { AppError, NotFoundError, ValidationError } from '@oshun/errors';
// Domain-specific errors
class IsisGenerationError extends AppError {
constructor(
message: string,
public jobId: string,
public cause?: Error
) {
super(message, 'ISIS_GENERATION_ERROR', 500, { jobId }, cause);
}
}
// Handling errors from other domains
async function generateWithFallback(request: GenerateRequest) {
try {
return await isis.generation.create(request);
} catch (error) {
if (error instanceof NotFoundError) {
// Model not found, try fallback
return await isis.generation.create({
...request,
model: 'fallback-model',
});
}
if (error instanceof ValidationError) {
// Invalid request, can't recover
throw error;
}
// Unexpected error, log and rethrow
logger.error('Generation failed', { error, request });
throw new IsisGenerationError(
'Failed to generate asset',
request.jobId,
error
);
}
}
Circuit Breaker Pattern#
import { CircuitBreaker } from '@oshun/http-client';
const isis = createIsisClient({
baseUrl: process.env.ISIS_API_URL,
circuitBreaker: {
failureThreshold: 5,
resetTimeout: 30000,
monitorInterval: 10000,
},
});
// Circuit will open after 5 failures
try {
await isis.generation.create(request);
} catch (error) {
if (error.code === 'CIRCUIT_OPEN') {
// Service is unavailable, use fallback
return await useFallbackGeneration(request);
}
throw error;
}
Retry with Backoff#
import { retry } from '@oshun/retry';
const result = await retry(() => sophia.search({ query: 'meditation' }), {
maxAttempts: 3,
delay: 1000,
backoff: 'exponential',
retryIf: (error) => error.status >= 500,
});
Dead Letter Queue#
eventBus.subscribe(
'isis.asset.generated',
async (event) => {
await processAsset(event);
},
{
maxRetries: 3,
deadLetterQueue: 'yemaya-dlq',
onMaxRetriesExceeded: async (event, error) => {
// Alert on repeated failures
await alerting.notify({
severity: 'warning',
message: `Failed to process event after 3 attempts`,
context: { eventId: event.id, error: error.message },
});
},
}
);
Testing Integrations#
Unit Testing with Mocks#
import { describe, it, expect, vi } from 'vitest';
import { createMockIsisClient } from '@isis/client/testing';
describe('ProjectAssetService', () => {
it('should generate asset and track pending', async () => {
const mockIsis = createMockIsisClient();
mockIsis.generation.create.mockResolvedValue({
id: 'job-123',
status: 'processing',
});
const service = new ProjectAssetService(mockIsis);
const result = await service.generateAssetForProject('proj-1', {
prompt: 'A sunset',
});
expect(result.jobId).toBe('job-123');
expect(mockIsis.generation.create).toHaveBeenCalledWith({
prompt: 'A sunset',
metadata: { projectId: 'proj-1', source: 'yemaya' },
});
});
});
Integration Testing#
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { TestEventBus } from '@oshun/event-bus/testing';
describe('Asset Generation Integration', () => {
let eventBus: TestEventBus;
beforeAll(async () => {
eventBus = new TestEventBus();
await eventBus.start();
});
afterAll(async () => {
await eventBus.stop();
});
it('should receive asset generated event', async () => {
const received = eventBus.waitForEvent('isis.asset.generated');
// Trigger generation (via API or test helper)
await triggerGeneration({ prompt: 'A sunset' });
const event = await received;
expect(event.payload.assetId).toBeDefined();
});
});
Contract Testing#
import { describe, it, expect } from 'vitest';
import { IsisAssetGeneratedEventSchema } from '@oshun/contracts-events';
describe('Event Contract: isis.asset.generated', () => {
it('should validate correct event', () => {
const event = {
id: '123e4567-e89b-12d3-a456-426614174000',
type: 'isis.asset.generated',
source: 'isis',
timestamp: '2024-01-15T10:30:00Z',
correlationId: 'corr-123',
payload: {
assetId: '123e4567-e89b-12d3-a456-426614174001',
jobId: '123e4567-e89b-12d3-a456-426614174002',
asset: {
id: '123e4567-e89b-12d3-a456-426614174001',
type: 'image',
url: 'https://storage.oshun.io/assets/123.png',
mimeType: 'image/png',
size: 1024000,
createdAt: '2024-01-15T10:30:00Z',
},
generation: {
model: 'stable-diffusion-xl',
prompt: 'A sunset',
parameters: { steps: 50 },
},
},
metadata: {},
};
expect(() => IsisAssetGeneratedEventSchema.parse(event)).not.toThrow();
});
it('should reject invalid event', () => {
const invalidEvent = {
type: 'isis.asset.generated',
payload: { assetId: 'not-a-uuid' },
};
expect(() => IsisAssetGeneratedEventSchema.parse(invalidEvent)).toThrow();
});
});
End-to-End Testing#
import { describe, it, expect } from 'vitest';
import { TestHarness } from '@oshun/testing';
describe('E2E: Generate Asset Pipeline', () => {
const harness = new TestHarness();
it('should generate asset and attach to project', async () => {
// 1. Create test project
const project = await harness.yemaya.createProject({
name: 'Test Project',
});
// 2. Generate asset
const result = await harness.yemaya.generateAsset(project.id, {
prompt: 'A sunset over mountains',
});
// 3. Wait for completion
await harness.waitForEvent('isis.asset.generated', {
timeout: 60000,
filter: (e) => e.metadata.projectId === project.id,
});
// 4. Verify asset attached
const projectAssets = await harness.yemaya.getProjectAssets(project.id);
expect(projectAssets).toHaveLength(1);
expect(projectAssets[0].type).toBe('image');
});
});
Best Practices#
1. Use Domain Clients#
Always use the provided domain clients instead of raw HTTP:
// ✅ Good: Type-safe, handles auth, retries, errors
const isis = createIsisClient({ baseUrl: ISIS_URL });
await isis.generation.create({ prompt: 'A sunset' });
// ❌ Bad: No type safety, manual error handling
await fetch(`${ISIS_URL}/v1/generation`, { method: 'POST', body: '...' });
2. Include Correlation IDs#
Always propagate correlation IDs for tracing:
// In HTTP middleware
app.use((req, res, next) => {
req.correlationId = req.headers['x-correlation-id'] || generateUUID();
res.setHeader('X-Correlation-ID', req.correlationId);
next();
});
// In event publishing
await eventBus.publish('domain.event', payload, {
correlationId: req.correlationId,
});
// In domain client calls
await isis.generation.create(request, {
headers: { 'X-Correlation-ID': correlationId },
});
3. Handle Partial Failures#
Design for partial failures in distributed operations:
async function processMultipleAssets(assetIds: string[]) {
const results = await Promise.allSettled(
assetIds.map((id) => isis.assets.get(id))
);
const successful = results.filter((r) => r.status === 'fulfilled');
const failed = results.filter((r) => r.status === 'rejected');
if (failed.length > 0) {
logger.warn('Some assets failed to load', {
successful: successful.length,
failed: failed.length,
});
}
return successful.map((r) => r.value);
}
4. Implement Idempotency#
Make event handlers idempotent:
eventBus.subscribe('isis.asset.generated', async (event) => {
// Check if already processed
const existing = await db.processedEvents.findUnique({
where: { eventId: event.id },
});
if (existing) {
logger.debug('Event already processed', { eventId: event.id });
return;
}
// Process event
await processAssetGenerated(event);
// Mark as processed
await db.processedEvents.create({
data: { eventId: event.id, processedAt: new Date() },
});
});
5. Use Timeouts#
Always set timeouts for cross-domain calls:
const isis = createIsisClient({
baseUrl: ISIS_URL,
timeout: 30000, // 30 second default
});
// Override for long operations
const result = await isis.generation.waitForCompletion(jobId, {
timeout: 300000, // 5 minutes for generation
});
6. Monitor Integration Health#
Track cross-domain call metrics:
import { metrics } from '@oshun/metrics';
const domainCallDuration = metrics.histogram({
name: 'domain_call_duration_seconds',
help: 'Duration of cross-domain API calls',
labelNames: ['source', 'target', 'method', 'status'],
});
// Wrap client calls
async function callWithMetrics<T>(
target: string,
method: string,
fn: () => Promise<T>
): Promise<T> {
const timer = domainCallDuration.startTimer({ target, method });
try {
const result = await fn();
timer({ status: 'success' });
return result;
} catch (error) {
timer({ status: 'error' });
throw error;
}
}
Anti-Patterns#
1. Direct Database Access#
// ❌ NEVER do this - violates domain boundaries
import { prisma as isisPrisma } from '@isis/database';
const jobs = await isisPrisma.generationJob.findMany();
// ✅ Always use domain clients
const jobs = await isis.generation.list();
2. Synchronous Event Processing#
// ❌ Don't block on event processing
await eventBus.publish('domain.event', payload);
await waitForAllConsumers(); // Blocks, defeats purpose of events
// ✅ Fire-and-forget, let consumers process async
await eventBus.publish('domain.event', payload);
// Continue immediately
3. Tight Coupling to Event Payloads#
// ❌ Assumes specific payload structure
eventBus.subscribe('isis.asset.*', async (event) => {
const url = event.payload.asset.variants[0].url; // Breaks if structure changes
});
// ✅ Validate and handle gracefully
eventBus.subscribe('isis.asset.*', async (event) => {
const validated = AssetEventSchema.safeParse(event);
if (!validated.success) {
logger.warn('Unknown event format', { event });
return;
}
const asset = validated.data.payload.asset;
});
4. Ignoring Errors#
// ❌ Silent failures hide problems
eventBus.subscribe('isis.asset.generated', async (event) => {
try {
await processAsset(event);
} catch (error) {
// Swallowed! Problem hidden forever
}
});
// ✅ Log, alert, and potentially retry
eventBus.subscribe('isis.asset.generated', async (event) => {
try {
await processAsset(event);
} catch (error) {
logger.error('Failed to process asset', { error, eventId: event.id });
throw error; // Let event bus handle retry/DLQ
}
});
5. Missing Idempotency#
// ❌ Not idempotent - duplicate events cause duplicates
eventBus.subscribe('isis.asset.generated', async (event) => {
await db.projectAssets.create({ assetId: event.payload.assetId });
});
// ✅ Idempotent - safe for reprocessing
eventBus.subscribe('isis.asset.generated', async (event) => {
await db.projectAssets.upsert({
where: { assetId: event.payload.assetId },
create: { assetId: event.payload.assetId },
update: {}, // No-op if exists
});
});
6. Circular Dependencies#
// ❌ Circular: Yemaya → Isis → Yemaya
// Isis event handler calling Yemaya, which calls Isis again
// ✅ Break cycles with:
// 1. Events instead of sync calls
// 2. Saga/orchestration pattern
// 3. Shared contracts without implementation dependencies
Checklist#
Before adding a new cross-domain integration:
- Identified which domain owns the data/capability
- Using domain client (not raw HTTP or direct DB)
- Correlation IDs propagated for tracing
- Appropriate communication pattern (sync vs async)
- Error handling with retries and circuit breakers
- Event handlers are idempotent
- Timeouts configured appropriately
- Monitoring and metrics in place
- Contract tests written
- Integration tests written
- Documentation updated