Yemaya — Technical Specifications

Technical reference covering data models, database schema, API contracts, events, worker queues, configuration, environment variables, and integration points. Every entry below is traceable to source code in libs/yemaya/* or apps/yemaya/*.

This document is the implementation contract for Yemaya. It defines the exact data model (table names, field types, foreign key strategies), the API surface (route files, mount paths, middleware chain), the event bus contracts (12 published events, 6 subscribed events, consumer group names), the BullMQ queue configuration (queue names, retry policies), and the environment variables required to run the API and worker processes.

Everything here is derived directly from source: the Prisma schema at libs/yemaya/database/prisma/schema.prisma, the app entry points at apps/yemaya/api/src/app.ts and apps/yemaya/workers/src/queues.ts, and the contracts package at libs/contracts/src/events/yemaya.ts.

Database Configuration#

Yemaya uses a dedicated PostgreSQL database, separate from the databases of every other Oshun domain. The Prisma schema lives at libs/yemaya/database/prisma/schema.prisma and the generated client at libs/yemaya/database/src/generated/client.

Source: libs/yemaya/database/prisma/schema.prisma lines 21–30.

The schema is intentionally scoped to platform concerns. Capability-domain data lives in separate databases and is referenced by ID, not by foreign key. This means no Yemaya migration can break Isis, Hathor, Sophia, or Bellona, and vice versa:

Data Models#

All models below are defined in libs/yemaya/database/prisma/schema.prisma. Primary keys default to cuid() (@db.VarChar(25)) unless otherwise noted; the canonical-library models (Collection, Incident, SupportCase) use uuid(). Most platform models carry createdAt, updatedAt, and a nullable deletedAt for soft deletion.

User#

The core user account model — authentication identity, display profile, and session tracking all hang off this record.

Table: users Relations: accounts, sessions, memberships, ownedProjects, projectMembers, assets, comments, activities, notifications, preferences, apiKeys.

UserPreference#

Per-user UI and notification preferences, stored as a one-to-one extension of the User record so the core User table stays lean.

Table: user_preferences.

Account#

OAuth and social provider accounts linked to a user. A single user can have multiple OAuth accounts from different providers (Google, GitHub, etc.).

Unique: [provider, providerAccountId] Table: accounts.

Session and ApiKey#

These two models handle the two authentication mechanisms: browser sessions (JWT-backed, expiring) and server-to-server API keys (hashed, scoped).

Tables: sessions, api_keys.

Organization#

Multi-tenant organizations are the top-level billing and access boundary. Every project belongs either to an organization or directly to a user, and plan-based feature gating applies at the organization level.

Table: organizations Relations: members, teams, projects, invitations, subscription.

Team and Membership Models#

Teams sit between organizations and projects, providing an intermediate grouping level for large studios. The join tables below track who belongs to what and at what access level.

OrganizationMember is unique on [organizationId, userId]; TeamMember on [teamId, userId]; Team on [organizationId, slug]; TeamProject on [teamId, projectId].

Project#

The central creative container in Yemaya. A project holds all assets, scripts, storyboards, and configuration for a production. Cross-domain capability references (hathorWorldId, isisWorkflowIds[], sophiaPackIds[]) are stored here as plain ID fields, not foreign keys, preserving database isolation.

Unique: [organizationId, slug] Table: projects Relations: organization, owner, members, teams, assets, folders, tags, activities.

Cross-Domain Reference Fields#

hathorWorldId, isisWorkflowIds[], and sophiaPackIds[] are resolved via internal API calls (or the capability proxy), never by database foreign key, so each domain database stays isolated.

Asset#

A digital file managed within a project's asset library. Assets carry both a reference to object storage (storageKey) and cross-domain references into Isis and Bellona for the generation job that produced them and the engine-processed variant respectively.

Table: assets Relations: project, folder, creator, tags (AssetTag[]), usages (AssetUsage[]), comments.

Folder#

Hierarchical self-referential folder structure for organizing assets within a project. The parentId self-reference creates an arbitrarily deep tree.

Table: folders.

Tagging Models#

Tags are organization-scoped and applied to both assets and projects through join tables. The AssetUsage model tracks where an asset is referenced so safe-to-delete analysis is always available.

Tag is unique on [organizationId, slug]. Tables: tags, asset_tags, project_tags, asset_usages.

Comment#

Threaded comments on assets, with resolution and pinning support for formal review workflows.

Table: comments.

Activity and Notification#

Activity records form the project-level audit feed visible to team members. Notifications are per-user inbox items delivered to the UI and optionally via email or push.

Tables: activities, notifications.

Canonical Library, Support, and Incident Records#

These three UUID-keyed models back cross-domain library curation and operational support workflows. They use uuid() primary keys (rather than cuid()) to align with cross-domain reference conventions.

Collection#

A curated library collection (@@map("collections")).

Incident#

An operational incident record (@@map("incidents")).

SupportCase#

A customer support case (@@map("support_cases")).

Webhooks#

Webhooks allow external systems to receive Yemaya events via HTTP. Each Webhook configuration has a companion WebhookDelivery record for every delivery attempt, enabling retry tracking and audit.

Tables: webhooks, webhook_deliveries.

Subscription and Billing#

The billing model tracks the organization's active subscription plan, invoices, and credit balance. UsageRecord provides per-metric metering for credit-based services like AI generation.

Tables: subscriptions, invoices, invoice_line_items, usage_records, credit_balances, credit_transactions.

Plugin Marketplace#

The plugin data model stores the plugin registry, version history, per-scope installation records, and community reviews.

Tables: plugins, plugin_versions, plugin_installations, plugin_reviews.

AuditLog#

The AuditLog model captures an immutable before/after snapshot of every audited action for GDPR and SOC2 compliance reporting.

Table: audit_logs.

Key Enumerations#

All Prisma enums below are defined in libs/yemaya/database/prisma/schema.prisma. These values are the only valid states for their respective fields — the database and Prisma client both enforce membership.

UserStatus#

ACTIVE, INACTIVE, SUSPENDED, PENDING_VERIFICATION

OrganizationPlan / SubscriptionPlan#

FREE, STARTER, PROFESSIONAL, ENTERPRISE (two separate enums with the same members).

OrganizationRole#

OWNER, ADMIN, MEMBER, VIEWER

TeamRole#

LEAD, MEMBER

ProjectAccess#

VIEW, EDIT, ADMIN

ProjectType#

FILM, GAME, ANIMATION, COMMERCIAL, MUSIC_VIDEO, DOCUMENTARY, OTHER

ProjectStatus#

DRAFT, ACTIVE, REVIEW, COMPLETED, ARCHIVED

ProjectRole#

OWNER, ADMIN, EDITOR, CONTRIBUTOR, VIEWER

Visibility#

PRIVATE, TEAM, ORGANIZATION, PUBLIC

AssetType (15 values)#

IMAGE, VIDEO, AUDIO, MODEL_3D, DOCUMENT, SCRIPT, STORYBOARD, CHARACTER_DESIGN, ENVIRONMENT_DESIGN, ANIMATION, VFX, MATERIAL, TEXTURE, FONT, OTHER

AssetStatus#

DRAFT, PROCESSING, REVIEW, APPROVED, REJECTED, ARCHIVED

SubscriptionStatus#

ACTIVE, PAST_DUE, CANCELLED, TRIALING, PAUSED

InvoiceStatus#

DRAFT, OPEN, PAID, VOID, UNCOLLECTIBLE

CreditTransactionType#

PURCHASE, USAGE, REFUND, BONUS, ADJUSTMENT, EXPIRATION

ActivityType#

PROJECT, ASSET, USER, TEAM, ORGANIZATION, SYSTEM

NotificationType#

MENTION, COMMENT, INVITATION, ASSIGNMENT, APPROVAL, SYSTEM, ALERT

WebhookDeliveryStatus#

PENDING, SUCCESS, FAILED, RETRYING

PluginCategory#

GENERATION, EDITING, EXPORT, INTEGRATION, UTILITY, COLLABORATION, ANALYTICS, AUTOMATION, OTHER

PluginPricing#

FREE, PAID, FREEMIUM, SUBSCRIPTION

PluginStatus#

PENDING, APPROVED, REJECTED, PUBLISHED, DEPRECATED, REMOVED

Library-Level Enumerations#

Several Yemaya libraries define their own typed constant objects (TypeScript as const unions, not Prisma enums). These live inside individual library packages rather than in the database schema.

@yemaya/canon-enforcement (libs/yemaya/canon-enforcement/src/types.ts):

These constants drive the tier hierarchy and workflow states used by the canon management system to classify established creative facts.

@yemaya/human-override (libs/yemaya/human-override/src/types.ts):

These constants define the types of human interventions available, the conditions that trigger automatic escalation, and the recovery lifecycle.

@yemaya/production-verification (libs/yemaya/production-verification/src/types.ts):

These constants model the asset completeness taxonomy and the four-state sign-off workflow used by pre-delivery verification checks.

Pipeline Schema#

The orchestration pipeline model is defined as Zod schemas in libs/yemaya/orchestration/src/schemas/pipeline.ts. These schemas are the runtime contract: every pipeline created or received by the orchestration layer is validated against them.

Pipeline#

A pipeline is the top-level execution unit: it holds a list of steps with explicit dependencies, a progress counter, and the variables injected at runtime.

PipelineStatus: draft, ready, running, paused, completed, failed, cancelled.

PipelineStep#

Each step in a pipeline carries its own retry configuration, timeout, and dependency list. The dependsOn field specifies which step IDs must complete before this step can start, enabling the orchestrator to build a correct topological execution order.

StepStatus: pending, queued, running, completed, failed, skipped, cancelled.

StepType (40 values)#

The StepType enum defines every action a pipeline can request. Values are namespaced by domain so it is always clear which system will execute a given step. The 40 values span generation (Isis), research (Sophia), worldbuilding (Hathor), engine integration (Bellona), DCC orchestration (MCP), studio actions (Yemaya), and data-flow utilities.

PipelineTemplate#

A PipelineTemplate is a runtime-state-free pipeline definition — it stores the step graph, variable schema, and metadata but no execution state. Templates can be public (shared across the platform), built-in (shipped with Yemaya), or private to an organization.

A reusable, runtime-state-free pipeline definition: id, name, description?, category, stepTemplates[] (name/type/config/ dependsOn only), variables (typed schema with type ∈ string/number/boolean/array/object, optional default, required flag), isPublic, isBuiltIn, tags[], usageCount.

Built-In Pipelines#

@yemaya/autonomous-pipelines exports factory functions that produce CreatePipelineInput objects. These are the ready-made production pipeline definitions shipped with Yemaya:

• Film pipeline (createFilmPipeline) — FilmPipelineStage = pre_production, world_building, asset_generation, scene_assembly, animation, rendering, post_production, export. Steps cover script analysis (sophia:analyze), visual research (sophia:research), character/ location creation (hathor:*), 3D asset generation (isis:generate_3d), audio (isis:generate_audio), scene assembly / camera / lighting / render / composite / export via mcp:execute_dcc against Blender and DaVinci, and a closing yemaya:review step.
• Game pipeline (createGamePipeline) — libs/yemaya/autonomous-pipelines/src/pipelines/game.ts.
• Curation and QC pipelines — libs/yemaya/autonomous-pipelines/src/pipelines/curation.ts and qc.ts.

The film pipeline's qualityPreset accepts draft, preview, production, final.

API Surface#

The Yemaya REST API is an OpenAPIHono (@hono/zod-openapi) application defined in apps/yemaya/api/src/app.ts, listening on port 3000 by default. Routes respond with RFC 7807 problem details on error (problemDetailsErrorHandler). The OpenAPI document is served at GET /openapi.json; interactive docs at GET /docs (Swagger UI), GET /reference and GET /playground (Scalar).

Route Mounting#

app.ts makes 30 app.route() calls across 24 distinct base paths. Several base paths host multiple routers (for example /v1/schedules and /v1/budgets), and one router file (websocket.ts) provides the /ws upgrade endpoint. The table below maps every router file to its mount path and describes its responsibility:

Engine integrations (Blender, Godot, Unreal, Houdini, DaVinci) and AI generation (image/3D/audio/video) are not separate Yemaya route files. They are reached either through the /v1/capabilities/{domain}/... proxy or through pipeline steps (isis:*, bellona:*, mcp:execute_dcc).

Project Endpoints (representative)#

projects.ts defines the following createRoute operations. The pattern follows OpenAPIHono convention and every operation gets a unique operationId in the generated OpenAPI document:

The remaining routers follow the same Zod-OpenAPI createRoute pattern; the OpenAPI tag list in app.ts enumerates the full set of resource groups (Authentication, Sessions, API Keys, Users, Organizations, Teams, Projects, Assets, Scripts, Scenes, Characters, Worlds, Storyboards, Schedules, Budgets, Crew, Call Sheets, Locations, Collaboration, Agents, Plugins, Webhooks, Search, Analytics, Admin, API Versioning, Capabilities, WebSocket events, and more).

Score Editor Entitlement Gate#

The score-editor BFF surface is tier-gated to prevent free or starter plan users from accessing AAA features. score-editor.ts resolves an EditorTier from request headers (x-yemaya-tier ∈ contemplative / aaa-creator / operator) and a roles list (x-yemaya-roles / x-user-role). Access to the scene-score editor requires the aaa-creator or operator tier, or an admin role.

Health and Metrics#

These endpoints do not require authentication and are used by load balancers, monitoring systems, and CI/CD pipelines:

Middleware Chain#

The middleware chain runs in a fixed order on every request. Understanding this order is important when debugging: each middleware may short-circuit the chain if it rejects a request.

app.ts applies, in order: request ID (X-Request-ID), request context (async local storage), performance metrics (p95 target 100 ms), RFC 7807 error handler, security headers (CSP, HSTS, frame options), CORS (origin allowlist), request logging (non-production only), gzip compression, server timing, input sanitization (/v1/*), asset metadata sanitization (/v1/assets/*), rate limiting (/v1/*), and API versioning (path + header, header X-API-Version). Additional middleware modules in apps/yemaya/api/src/middleware/ include circuit-breaker.ts, gdpr-compliance.ts, soc2-compliance.ts, and proxy-auth.ts.

Event Contracts#

Yemaya publishes and consumes events via @oshun/event-bus, which is Redis-backed — it uses ioredis for Redis pub/sub fan-out plus TTL-bounded keys for replay, durable sorted sets for delayed/retried delivery, and consumer groups for at-most-once delivery. It is not Kafka. (Source: libs/shared/event-bus/src/event-bus.ts, module header and import { Redis } from 'ioredis'.)

The API server initializes the event bus with keyPrefix: 'oshun:events', persistence: true, eventTtl: 86400 (24 h), exponential retry (maxAttempts: 3), and a 7-day dead-letter retention (apps/yemaya/api/src/index.ts).

Events Published by Yemaya#

YemayaEventTypes is defined in libs/contracts/src/events/yemaya.ts and is the canonical source of published event names. Publishing is handled by YemayaEventPublisher (libs/yemaya/event-publisher/src/yemaya-event-publisher.ts), which exposes a typed method per event and fails soft — a publish error is logged but does not break the request flow. Each payload has a corresponding Zod schema in libs/contracts/src/events/yemaya.ts.

The 12 events and their default notification targets are:

Events Consumed by Yemaya#

YEMAYA_SUBSCRIPTIONS in libs/yemaya/event-handlers/src/index.ts is the canonical list of subscribed events. Handlers run under the consumer group yemaya-api (set by the API server) or yemaya-handlers (default). Each handler updates the Yemaya project state in response to a completed action in a capability domain.

The 6 subscribed events and their handlers are:

Worker Queues#

Background job processing uses BullMQ over Redis. Queue names and worker types are defined in apps/yemaya/workers/src/queues.ts and apps/yemaya/workers/src/config.ts.

Per the queues.ts header comment, capability-domain execution queues live with their owning domains (Isis owns generative/3D execution; Bellona owns media, render, export, and engine execution). Yemaya workers only coordinate those queues and process studio-level events.

Queues (QUEUE_NAMES)#

Each queue has its own retry and backoff policy tuned to the expected job duration. Notification retries are faster (short backoff, 5 attempts) while pipeline orchestration jobs are slower and less likely to benefit from rapid retries.

Worker Processes#

Worker implementations in apps/yemaya/workers/src/workers/: notification-worker.ts, rendering-orchestration-worker.ts, export-orchestration-worker.ts, pipeline-orchestration-worker.ts, event-consumer.ts.

The WORKER_TYPE value (parsed from a --worker= CLI argument; defaults to all) selects which worker(s) run. Valid values from the config Zod enum: all, notification, rendering, export, pipeline, event-consumer, orchestration. Concurrency is set via WORKER_CONCURRENCY (1–100, default 5).

Notification Job Types#

The notification queue handles four job types dispatched under different delivery channels. These job type strings are the discriminant used inside the notification worker to route each job to the right delivery path:

notification.send-email, notification.send-webhook, notification.send-push, notification.digest.

Environment Variables#

API Server (apps/yemaya/api)#

In production, CORS defaults to https://yemaya.io, https://app.yemaya.io, https://api.yemaya.io plus any ALLOWED_ORIGINS entries.

Workers (apps/yemaya/workers)#

The worker type is selected by a --worker=<type> process argument.

Integration Points#

Storage#

Asset files are stored in MinIO (development) or S3-compatible storage (production). Each Asset row references storage by storageKey; storageUrl holds a direct (pre-signed in production) access URL. The API's storage client lives in apps/yemaya/api/src/services/storage-client.ts and storage-utils.ts; chunked uploads are handled by chunked-upload.ts.

Authentication#

Yemaya supports three authentication mechanisms, all registering as OpenAPI security schemes in app.ts:

• JWT Bearer tokens — registered OpenAPI security scheme bearerAuth (http / bearer / JWT). Obtained via login or OAuth.
• API keys — registered scheme apiKey, header X-API-Key, for server-to-server access. Backed by the ApiKey model (hashed keyHash, keyPrefix, scopes[]).
• OAuth 2.0 — social login; provider records in the accounts table. API OAuth logic in apps/yemaya/api/src/services/oauth.ts.
• The @yemaya/auth library provides JWT and session primitives.

Capability Domain Proxy#

The capabilities proxy is the synchronous integration path to the four capability domains. It handles authentication, circuit-breaking, and health checking so that individual feature routes do not need to implement these concerns.

apps/yemaya/api/src/routes/capabilities.ts plus apps/yemaya/api/src/services/proxy.ts implement a reverse proxy to the four capability domains. A CapabilityDomain request is forwarded with service headers built from the caller's proxy-auth claims; proxy-auth.ts middleware gates access. The proxy includes circuit-breaker protection and health checks (OpenAPI tag Capabilities - Health).

Cross-Domain References#

The Yemaya database stores only lightweight IDs pointing into other domain databases (Project.hathorWorldId, Project.isisWorkflowIds[], Project.sophiaPackIds[], Asset.isisGenerationId, Asset.bellonaAssetId). Resolution happens at query time through internal service calls or the capability proxy — never through SQL joins.

Real-Time Collaboration#

WebSocket connections terminate at /ws on the same Hono process as the REST API (apps/yemaya/api/src/routes/websocket.ts, apps/yemaya/api/src/websocket/). The @yemaya/collaboration library provides the Yjs-CRDT sync, presence, and cursor logic. Presence data is ephemeral and is not persisted to PostgreSQL.

Webhook Delivery#

Webhook delivery runs asynchronously through the BullMQ notification queue. Each configured Webhook row is delivered by the notification worker (notification.send-webhook job). Each attempt creates a WebhookDelivery row; failures retry with exponential backoff. Webhook.retryCount defaults to 3 and retryDelay to 60 seconds. Delivery status follows WebhookDeliveryStatus: PENDING → SUCCESS | FAILED → RETRYING → ….

Authentication and Security#

All API routes are protected by the layered middleware described in the Middleware Chain section. The key security rules are:

• All /v1/* routes pass through input sanitization (XSS / SQL / command injection protection) and rate limiting; documentation, health, and metrics endpoints are exempt from rate limiting.
• Security headers: Content-Security-Policy, Strict-Transport-Security (max-age=31536000; includeSubDomains), X-Content-Type-Options: nosniff, X-Frame-Options: DENY, X-XSS-Protection, and Referrer-Policy: strict-origin-when-cross-origin.
• Compliance middleware (gdpr-compliance.ts, soc2-compliance.ts) applies audit logging uniformly; the AuditLog model captures actor, action, entity, before/after snapshots, IP, and user agent.
• Errors are RFC 7807 problem details with a propagated requestId.

Acceptance Criteria#

A Yemaya deployment is correct when all of the following conditions hold simultaneously. These criteria are the observable contract between the specifications and a running system:

1. Schema parity — the running database matches libs/yemaya/database/prisma/schema.prisma; GET /health/ready reports the database check healthy.
2. API contract — GET /openapi.json returns a valid OpenAPI 3.1 document covering all 30 mounted route operations; every operation has a unique operationId.
3. Event bus — the API logs the count of active event subscriptions on startup; all 6 entries of YEMAYA_SUBSCRIPTIONS subscribe successfully, and the bus connects to Redis with prefix oshun:events.
4. Workers — each BullMQ queue in QUEUE_NAMES is created with its configured retry/backoff policy; the selected WORKER_TYPE starts without error.
5. Pipeline validity — pipelines created from @yemaya/autonomous-pipelines factories validate against PipelineSchema; every step type is a member of the 40-value StepType enum.
6. Cross-domain isolation — no foreign keys exist from the Yemaya database into Isis/Hathor/Sophia/Bellona databases; all such references are stored as ID columns and resolved over the network.
7. Capability proxy — /v1/capabilities/{domain}/... forwards to the configured domain API, with circuit-breaker protection and a passing health check.
8. Compliance — GDPR/SOC2 middleware records an AuditLog entry for audited actions; problem-details responses carry a requestId.