Eve library

Oshun V1 Phase-2 Architecture Overview

Guide

Revision 1

On this page

Status: Living document  Date: 2026-04-27  Owner: eng-platform  Backlog item: V1-P2-0371

This is the canonical view of how data, requests, and events flow across the Oshun platform once the V1 Phase-2 foundational work has landed. It is the "start here" doc for anyone joining a domain — it states what the moving parts are, who owns each one, and what end-to-end flows look like in practice. The root ARCHITECTURE.md is the reference for the V1 Phase-1 system as it shipped; this document is the delta — what changes once the Phase-2 contracts (LLM gateway, embedding provider, cost-tracking, deterministic mode, scanner conventions, external-API guard) are wired up.

Domains at a glance#

Six bounded contexts, each with its own data ownership, libraries, and event topic prefix:

Domain Topic prefix Owns Read-only consumers
Yemaya — creative studio yemaya.* Projects, scripts, scenes, characters, shots, sequences, budgets, team membership Lilith (course assets), Bellona (exports)
Lilith — consciousness experience lilith.* Meditation sessions, courses, conversations, progress, biometrics, subscriptions Sophia (knowledge queries), Tara (App-Store-safe variant)
Isis — generative factory isis.* Generation jobs, assets, GPU workflows, provenance Yemaya (asset feed), Lilith (meditation generation)
Sophia — research / knowledge sophia.* Documents, embeddings index, citations, query history Lilith (BFF knowledge route), Iris (RAG)
Hathor — worldbuilding hathor.* Worlds, narrative lattices, NPC agents, simulation state Yemaya (worldbuilding)
Bellona — build / integration bellona.* Engine adapters, export packages, build jobs Yemaya (export feed)

In addition, three platform domains provide shared capabilities consumed by all six:

  • Iris — embeddings, OCR, vision LLM, document layout analysis, desktop automation, A2A privacy primitives, semantic memory.
  • Aja — motion capture, GPU motion pipeline, real-time pose / face / hand tracking. Consumed by Yemaya and Isis.
  • Aje / Aphrodite / Airmid — cryptographic primitives (signing, key exchange, anonymization) consumed everywhere user PII or content provenance crosses a trust boundary.

Phase-2 foundational contracts#

These contracts are what changes between the Phase-1 system in ARCHITECTURE.md and the Phase-2 end state. Every domain consumes them; they bind the six contexts into one platform.

1. LLM gateway — IsisLLMClient#

Every prompt that crosses a process boundary flows through Isis. Phase-1 allowed direct vendor calls (OpenAI, Anthropic, ComfyUI); Phase-2 requires IsisLLMClient (V1-P2-0001 audit; isis-llm-gateway-audit.md) so that:

  1. Cost accounting is centralized — every token is charged to a tenant via the contract in isis-cost-tracking-contract.md (V1-P2-0005).
  2. Deterministic-mode replays work — the snapshot/replay semantics in isis-deterministic-mode.md (V1-P2-0021) require the gateway to be the single point where requests are recorded and replayed.
  3. Provider failover is uniform — libs/iris/failover/src/failover-manager.ts provides multi-provider orchestration that the gateway uses internally.
text
Caller (any domain)
   │   Request: { tenantId, model, messages, traceId }
   ▼
IsisLLMClient (libs/isis/llm-providers)
   │   ├── tenant quota check  (cost-tracking-contract.md)
   │   ├── deterministic-mode hook (isis-deterministic-mode.md)
   │   ├── failover-manager + circuit breaker
   │   └── provider call via @oshun/http-client (V1-P2-0358)
   ▼
Provider (OpenAI / Anthropic / local ONNX)
   │   Response
   ▼
IsisLLMClient
   │   ├── token accounting → bills to tenant
   │   ├── trace span → otel collector
   │   └── deterministic-mode write (when active)
   ▼
Caller receives { content, usage, costMicros, traceId }

2. Embedding gateway — IrisEmbeddingClient#

Sophia, Lilith (BFF knowledge), Iris (semantic memory), and Hathor (narrative-lattice retrieval) all call IrisEmbeddingClient. The provider chain is documented in embeddings-provider-decision.md (V1-P2-0030). The client wraps:

  • The configured embedding provider (bge-m3 default, vendor fallback).
  • A request-level cache keyed by (model, sha256(text)).
  • The same cost-tracking middleware as the LLM gateway, so embedding calls are billed alongside LLM calls.
  • The same @oshun/http-client retry + circuit breaker + telemetry stack.

3. ML / on-device runtime#

Browser- and Node-side inference flows through the runtime and provider stack defined in V1-P2-021x:

For Aja motion capture in particular, the GPU-pipeline plan is in aja-gpu-pipeline-strategy.md (V1-P2-0334) and the Kubernetes deployment posture is in aja-runtime-strategy.md (V1-P2-0333).

4. HTTP transport — @oshun/http-client#

Every outbound HTTP call uses libs/shared/http-client (createHttpClient / HttpClient / withRetry / withCircuitBreaker / tracedRequest) or one of the domain wrappers (libs/lilith/service-lib/circuit-breaker, libs/iris/failover, libs/hathor/client, libs/sophia/client, libs/bellona/client, libs/iris/agents/tools/web/http-client). The CI gate in .github/workflows/external-api-guard.yml (V1-P2-0358) refuses any new raw fetch( / axios( / got( / https.request( call that lacks both retry and circuit breaker and telemetry.

5. Persistence — Drizzle migration#

libs/yemaya/database, libs/lilith/service-lib/database, the Isis, Sophia, Hathor, and Bellona databases all run on PostgreSQL. The Phase-2 work tracked in drizzle-completion-targets.md (V1-P2-0233) walks every domain off placeholder persistence layers onto Drizzle ORM.

6. Cryptography — placeholder removal#

Every site flagged in crypto-replacement-targets.md (V1-P2-0176) — libs/aje/, libs/aphrodite/, libs/airmid/, libs/iris/memory/privacy/, libs/iris/a2a/, libs/iris/privacy/security/dlp/ — moves from stubbed crypto to the real primitive (@oshun/aje for signing/key-exchange, @oshun/aphrodite for homomorphic operations, @oshun/airmid for differential privacy / noise injection).

7. Event envelope contract#

Every cross-domain event uses the envelope from ARCHITECTURE.md § Communication Patterns:

typescript
interface DomainEvent<T> {
  id: string; // UUIDv7 (sortable)
  type: string; // e.g. "isis.asset.generated"
  source: string; // owning domain
  timestamp: Date;
  correlationId: string; // request-trace ID
  payload: T;
  metadata: Record<string, unknown>;
}

Producers commit owned data first, then publish. Subscribers idempotently key side effects on event.id and store processed-event records in the subscriber's own database. The Redis Streams transport is durable; failed subscribers are replayed until the projection succeeds.

8. Quality / scanner contracts#

Phase-2 introduces seven CI workflows that gate or report on the §2.6 quality bar. They span every domain — see the README for the index.

Workflow Tickets What it gates
stub-indicator-scan V1-P2-0350 New stub-vocabulary in PR diff
silent-random-scan V1-P2-0351 New Math.random() in deterministic functions
test-coherence-check V1-P2-0352 Per-file SHAPE/BOOLEAN/COMPUTED ratio (reporting)
adversarial-grep-trend V1-P2-0355 Nightly per-domain count + week-over-week delta (reporting)
precommit-hook-verification V1-P2-0356 Husky stub scan contract
stub-indicator-pr-comment V1-P2-0357 Sticky PR comment with file:line + remediation
external-api-guard V1-P2-0358 New raw HTTP calls without retry+CB+telemetry

The // stub:legitimate <reason>, // random:legitimate <reason>, and // http:guarded <reason> allow-list conventions are documented at stub-indicator-allowlist.md and inline in scripts/external-api-guard.mjs.

Canonical end-to-end flows#

These are the data flows the platform supports once the foundational work is complete. Each flow lists the originating user action, the participating domains, the events emitted, and the durability boundary.

Flow 1 — Yemaya generates an asset via Isis#

text
User in studio-web requests "generate concept art for shot 12"
   │
   ▼
Yemaya BFF (apps/yemaya/api)
   │   Validate: project membership, content-policy gate, budget remaining
   │   Persist: GenerationRequest row in yemaya_db
   │
   ▼  POST /isis/jobs (gRPC, mutual TLS)
Isis API (apps/isis/api)
   │   Persist: Job row in isis_db
   │   Enqueue: BullMQ "generation" queue, attempt budget = 3
   │   Publish: isis.job.queued
   │
   ▼  worker
Isis worker (apps/isis/workers)
   │   Pull image-gen pipeline from libs/isis/pipelines
   │   Call provider via IsisLLMClient OR ComfyUI (libs/isis/comfyui)
   │     ├── @oshun/http-client retries on 5xx / network
   │     ├── Circuit breaker skips dead providers (failover-manager)
   │     ├── HttpTracing emits OTel span; cost-tracker bills tenant
   │   Stream resulting image bytes to MinIO bucket isis-assets/
   │   Persist: Asset row in isis_db with provenance metadata
   │   Publish: isis.asset.generated
   │
   ▼  Redis Streams subscriber
Yemaya event-handler (libs/yemaya/event-handlers)
   │   Project: asset_link row in yemaya_db, key=event.id (idempotent)
   │   Notify: WebSocket push to studio-web via svc-real-time-sync
   ▼
User sees asset in shot 12's gallery.

Durability boundary: the Isis Job row + MinIO object survive a crash mid-flow. Failed worker attempts are retried by BullMQ; failed Yemaya projections are replayed from the Redis Stream until the row is written.

Flow 2 — Lilith generates a personalized meditation#

text
User in Lilith app taps "today's meditation"
   │
   ▼
Lilith BFF (apps/lilith/bff)
   │   Resolve user profile + progress + biometrics from lilith_db
   │   Resolve relevant teaching from Sophia knowledge graph:
   │     ├── Sophia search uses IrisEmbeddingClient (V1-P2-0030)
   │     └── Returns top-K passages with citations
   │
   ▼  POST /isis/meditation (gRPC)
Isis svc-meditation-generation
   │   IsisLLMClient generates script (cost-tracking-contract enforces quota)
   │   Isis TTS gateway (V1-P2-0140) renders audio (provider chain)
   │   Determinism: when seed is fixed, replay path stores prompt + audio
   │     fingerprint per isis-deterministic-mode.md
   │   Stream audio to MinIO bucket lilith-meditations/
   │   Publish: isis.meditation.ready
   │
   ▼  Redis Streams subscriber
Lilith svc-daily-content
   │   Project: today_meditation row in lilith_db
   │   Push to user via svc-notification (idempotent on event.id)

Flow 3 — Sophia ingests + indexes a document#

text
User uploads PDF to apps/sophia/api/ingest
   │
   ▼  Sync intake
Sophia api persists Document row in sophia_db with status=ingesting
Sophia publishes sophia.document.intake on Redis Stream
   │
   ▼  worker
Sophia ingestion worker
   │   Document layout analysis: libs/iris/doc-layout (V1-P2-0066)
   │   OCR pass for scanned pages: libs/iris/ocr (V1-P2-0060)
   │   Vision LLM caption pass: libs/iris/vision-llm (V1-P2-0064)
   │   Chunk + embed via IrisEmbeddingClient
   │   Write: sophia.embeddings index (PostgreSQL pgvector or external store)
   │   Update: Document row status=indexed
   │   Publish: sophia.document.ingested + sophia.index.updated
   │
   ▼ Subscribers
Lilith BFF (knowledge route) projects updated index pointer
Iris RAG cache invalidates per document_id

Flow 4 — Hathor simulates a worldbuilding tick#

text
Yemaya world editor calls Hathor "simulate next tick"
   │
   ▼
Hathor api persists SimulationRequest in hathor_db
Hathor publishes hathor.simulation.requested
   │
   ▼ worker
Hathor narrative-lattice engine
   │   Pull world state from hathor_db
   │   For each NPC: IsisLLMClient call shaped by lattice constraints
   │   Detect contradictions, resolve via Sophia citation lookup
   │   Persist: WorldState row + NPC actions in hathor_db
   │   Publish: hathor.simulation.completed
   │
   ▼ Subscribers
Yemaya world editor projects updated state
Sophia indexes new lore entries

Flow 5 — Bellona builds + exports#

text
Yemaya project lead clicks "export to Unreal 5"
   │
   ▼
Yemaya api packages export bundle metadata, persists ExportRequest
Yemaya publishes yemaya.export.requested
   │
   ▼ worker
Bellona engine adapter (libs/bellona/adapters/unreal)
   │   Pull authoritative assets from Isis MinIO buckets
   │   Transform to engine-specific format (FBX, USD, …)
   │   Stream to bellona-exports/ MinIO bucket
   │   Persist: Export row in bellona_db
   │   Publish: bellona.export.ready
   │
   ▼ Subscribers
Yemaya UI shows download link
Sophia indexes export manifest for citation

Shared infrastructure topology#

text
┌──────────────────────────────────────────────────────────────────┐
│                        CLIENT LAYER                              │
│ studio-web · studio-desktop · iris-mobile · cli · sdks (ts/cpp/py) │
└─────────────────────────────────────────────────────┬────────────┘
                                                       │
                  ┌────────────────────────────────────┴────────────────────────────────────┐
                  │                          API GATEWAY (Hono + OpenAPI)                    │
                  │       JWT (auth-orchestrator) · rate-limit · @oshun/http-client          │
                  └─────────────────────────────────┬────────────────────────────────────────┘
                                                    │
        ┌───────────────────┬──────────────────────┬┴──────────────────┬──────────────────┬───────────────────┐
        ▼                   ▼                      ▼                   ▼                  ▼                   ▼
     Yemaya               Lilith                  Isis                Sophia            Hathor              Bellona
    (yemaya_db)         (lilith_db)            (isis_db)            (sophia_db)       (hathor_db)         (bellona_db)
        │                   │                      │                   │                  │                   │
        └─────┬─────────────┴──────────────────────┴───────────────────┴──────────────────┴───────────────────┘
              │
    ┌─────────┴────────────────────────────────────────────────────────────────────────────────────────────┐
    │                          SHARED INFRASTRUCTURE                                                        │
    │                                                                                                       │
    │  Redis Streams + Pub/Sub  (event bus)                                                                 │
    │  Redis  (cache / rate-limit buckets)                                                                  │
    │  BullMQ on Redis  (job queues per domain)                                                             │
    │  PostgreSQL  (one DB per domain — Drizzle ORM, V1-P2-0233)                                            │
    │  pgvector  (Sophia embeddings index; optional Qdrant via observability profile)                       │
    │  MinIO  (asset object store: yemaya-assets/, isis-assets/, lilith-meditations/, bellona-exports/)     │
    │  Vault / VaultClient  (secrets, with retry + cache; libs/lilith/service-lib/vault-client.ts)          │
    │  OpenTelemetry collector → Prometheus + Tempo + Loki                                                  │
    │     dashboards live at docker/observability/grafana/dashboards/<domain>/                              │
    │  Mailpit (dev) / SES (prod)  (notifications)                                                          │
    └───────────────────────────────────────────────────────────────────────────────────────────────────────┘

Per-domain database isolation is enforced at the schema level — domains do not query each other's tables. Cross-domain reads happen via API or via projected read-models written by event-handler subscribers.

Trust boundaries#

The Phase-2 system has three trust boundaries that gate every flow:

  1. External user → API gateway. JWT auth, content-policy filtering, rate limit per tenant. The gateway is also where @oshun/http-client is wired up — every outbound provider call from a domain goes through the same retry + circuit-breaker + telemetry stack as the inbound request.
  2. Domain → Vault. No domain stores raw vendor API keys; every key is fetched via VaultClient.getSecret() with the 5-minute in-memory cache + 3x retry. Per V1-P2-0358 the vault client itself is on the wrapper allow-list (it's a primitive; it can call https.request directly).
  3. Domain → Domain. mTLS gRPC for synchronous call-outs; signed event envelopes for asynchronous broadcast. Subscribers verify the event's source and idempotently apply by event.id.

Observability surface#

Every flow emits the same three signal types:

  • Logs — pino structured JSON, scoped by correlationId == event.correlationId so a single user action can be reconstructed end to end across domain boundaries.
  • Metrics — Prometheus scrape on every apps/*/api service. Per-domain dashboards under docker/observability/grafana/dashboards/<domain>/ enumerated in the README.
  • Traces — OpenTelemetry OTLP via the HttpTracing interceptor in @oshun/http-client. Tempo retains 30 days; Grafana oshun/platform-overview.json is the top-of-funnel view.

For per-subsystem operational reference, every domain owns a runbook generated from the Runbook Template (V1-P2-0372), and every Tier-A subsystem owns a threat model generated from the Threat Model Template (V1-P2-0373).

Differences from ARCHITECTURE.md#

This document is the Phase-2 view; the table below highlights what changes from the Phase-1 baseline in ARCHITECTURE.md:

Concern Phase-1 baseline Phase-2 contract
LLM provider Direct vendor SDK calls scattered across domains Centralized IsisLLMClient (V1-P2-0001), cost-tracked, deterministic-replayable
Embeddings Per-domain vendor calls IrisEmbeddingClient (V1-P2-0030)
HTTP transport Mix of raw fetch/axios and ad-hoc wrappers @oshun/http-client everywhere, gated by external-api-guard (V1-P2-0358)
Crypto Stub primitives in privacy/security paths Real primitives per crypto-replacement-targets.md (V1-P2-0176)
Persistence Mixed (some Drizzle, some placeholder) Uniform Drizzle (V1-P2-0233)
Quality bar Manual review 7 CI workflows enforce §2.6 (V1-P2-0350..0358)
Documentation Free-form per domain Templated docs under docs/releases/p2/ (V1-P2-0370..0373)

When the items in the right column have all moved from [ ] to [x] in OSHUN_V1_TODOS_P2.md, this document IS the operating architecture; any divergence in code is a defect to fix, not a documentation update to make.