# Oshun V1 Phase-2 Architecture Overview

**Status:** Living document &nbsp;**Date:** 2026-04-27 &nbsp;**Owner:**
eng-platform &nbsp;**Backlog item:** V1-P2-0371

This is the canonical view of how data, requests, and events flow across the
Oshun platform once the V1 Phase-2 foundational work has landed. It is the
"start here" doc for anyone joining a domain — it states what the moving parts
are, who owns each one, and what end-to-end flows look like in practice. The
root [`ARCHITECTURE.md`](../../../ARCHITECTURE.md) is the reference for the V1
Phase-1 system as it shipped; this document is the delta — what changes once the
Phase-2 contracts (LLM gateway, embedding provider, cost-tracking, deterministic
mode, scanner conventions, external-API guard) are wired up.

## Domains at a glance

Six bounded contexts, each with its own data ownership, libraries, and event
topic prefix:

| Domain                                | Topic prefix | Owns                                                                              | Read-only consumers                                       |
| ------------------------------------- | ------------ | --------------------------------------------------------------------------------- | --------------------------------------------------------- |
| **Yemaya** — creative studio          | `yemaya.*`   | Projects, scripts, scenes, characters, shots, sequences, budgets, team membership | Lilith (course assets), Bellona (exports)                 |
| **Lilith** — consciousness experience | `lilith.*`   | Meditation sessions, courses, conversations, progress, biometrics, subscriptions  | Sophia (knowledge queries), Tara (App-Store-safe variant) |
| **Isis** — generative factory         | `isis.*`     | Generation jobs, assets, GPU workflows, provenance                                | Yemaya (asset feed), Lilith (meditation generation)       |
| **Sophia** — research / knowledge     | `sophia.*`   | Documents, embeddings index, citations, query history                             | Lilith (BFF knowledge route), Iris (RAG)                  |
| **Hathor** — worldbuilding            | `hathor.*`   | Worlds, narrative lattices, NPC agents, simulation state                          | Yemaya (worldbuilding)                                    |
| **Bellona** — build / integration     | `bellona.*`  | Engine adapters, export packages, build jobs                                      | Yemaya (export feed)                                      |

In addition, three platform domains provide shared capabilities consumed by all
six:

- **Iris** — embeddings, OCR, vision LLM, document layout analysis, desktop
  automation, A2A privacy primitives, semantic memory.
- **Aja** — motion capture, GPU motion pipeline, real-time pose / face / hand
  tracking. Consumed by Yemaya and Isis.
- **Aje / Aphrodite / Airmid** — cryptographic primitives (signing, key
  exchange, anonymization) consumed everywhere user PII or content provenance
  crosses a trust boundary.

## Phase-2 foundational contracts

These contracts are what changes between the Phase-1 system in
[`ARCHITECTURE.md`](../../../ARCHITECTURE.md) and the Phase-2 end state. Every
domain consumes them; they bind the six contexts into one platform.

### 1. LLM gateway — `IsisLLMClient`

Every prompt that crosses a process boundary flows through Isis. Phase-1 allowed
direct vendor calls (OpenAI, Anthropic, ComfyUI); Phase-2 requires
[`IsisLLMClient`](../OSHUN_V1_TODOS_P2.md) (V1-P2-0001 audit;
[`isis-llm-gateway-audit.md`](./isis-llm-gateway-audit.md)) so that:

1. Cost accounting is centralized — every token is charged to a tenant via the
   contract in
   [`isis-cost-tracking-contract.md`](./isis-cost-tracking-contract.md)
   (V1-P2-0005).
2. Deterministic-mode replays work — the snapshot/replay semantics in
   [`isis-deterministic-mode.md`](./isis-deterministic-mode.md) (V1-P2-0021)
   require the gateway to be the single point where requests are recorded and
   replayed.
3. Provider failover is uniform — `libs/iris/failover/src/failover-manager.ts`
   provides multi-provider orchestration that the gateway uses internally.

```
Caller (any domain)
   │   Request: { tenantId, model, messages, traceId }
   ▼
IsisLLMClient (libs/isis/llm-providers)
   │   ├── tenant quota check  (cost-tracking-contract.md)
   │   ├── deterministic-mode hook (isis-deterministic-mode.md)
   │   ├── failover-manager + circuit breaker
   │   └── provider call via @oshun/http-client (V1-P2-0358)
   ▼
Provider (OpenAI / Anthropic / local ONNX)
   │   Response
   ▼
IsisLLMClient
   │   ├── token accounting → bills to tenant
   │   ├── trace span → otel collector
   │   └── deterministic-mode write (when active)
   ▼
Caller receives { content, usage, costMicros, traceId }
```

### 2. Embedding gateway — `IrisEmbeddingClient`

Sophia, Lilith (BFF knowledge), Iris (semantic memory), and Hathor
(narrative-lattice retrieval) all call `IrisEmbeddingClient`. The provider chain
is documented in
[`embeddings-provider-decision.md`](./embeddings-provider-decision.md)
(V1-P2-0030). The client wraps:

- The configured embedding provider (`bge-m3` default, vendor fallback).
- A request-level cache keyed by `(model, sha256(text))`.
- The same cost-tracking middleware as the LLM gateway, so embedding calls are
  billed alongside LLM calls.
- The same `@oshun/http-client` retry + circuit breaker + telemetry stack.

### 3. ML / on-device runtime

Browser- and Node-side inference flows through the runtime and provider stack
defined in V1-P2-021x:

- [`ml-runtime-strategy.md`](./ml-runtime-strategy.md) — ONNX Runtime Web for
  browser, ONNX Runtime Node for server, WebGPU acceleration where available.
- [`pose-estimation-provider.md`](./pose-estimation-provider.md) (V1-P2-0211),
  [`object-detection-provider.md`](./object-detection-provider.md) (V1-P2-0212),
  [`asr-vad-provider.md`](./asr-vad-provider.md) (V1-P2-0213),
  [`speaker-id-provider.md`](./speaker-id-provider.md) (V1-P2-0214),
  [`emotion-detection-provider.md`](./emotion-detection-provider.md)
  (V1-P2-0215), [`cross-encoder-reranker.md`](./cross-encoder-reranker.md)
  (V1-P2-0216).

For Aja motion capture in particular, the GPU-pipeline plan is in
[`aja-gpu-pipeline-strategy.md`](./aja-gpu-pipeline-strategy.md) (V1-P2-0334)
and the Kubernetes deployment posture is in
[`aja-runtime-strategy.md`](./aja-runtime-strategy.md) (V1-P2-0333).

### 4. HTTP transport — `@oshun/http-client`

Every outbound HTTP call uses `libs/shared/http-client` (`createHttpClient` /
`HttpClient` / `withRetry` / `withCircuitBreaker` / `tracedRequest`) or one of
the domain wrappers (`libs/lilith/service-lib/circuit-breaker`,
`libs/iris/failover`, `libs/hathor/client`, `libs/sophia/client`,
`libs/bellona/client`, `libs/iris/agents/tools/web/http-client`). The CI gate in
[`.github/workflows/external-api-guard.yml`](../../../.github/workflows/external-api-guard.yml)
(V1-P2-0358) refuses any new raw `fetch(` / `axios(` / `got(` / `https.request(`
call that lacks both retry and circuit breaker and telemetry.

### 5. Persistence — Drizzle migration

`libs/yemaya/database`, `libs/lilith/service-lib/database`, the Isis, Sophia,
Hathor, and Bellona databases all run on PostgreSQL. The Phase-2 work tracked in
[`drizzle-completion-targets.md`](./drizzle-completion-targets.md) (V1-P2-0233)
walks every domain off placeholder persistence layers onto Drizzle ORM.

### 6. Cryptography — placeholder removal

Every site flagged in
[`crypto-replacement-targets.md`](./crypto-replacement-targets.md) (V1-P2-0176)
— `libs/aje/`, `libs/aphrodite/`, `libs/airmid/`, `libs/iris/memory/privacy/`,
`libs/iris/a2a/`, `libs/iris/privacy/security/dlp/` — moves from stubbed crypto
to the real primitive (`@oshun/aje` for signing/key-exchange, `@oshun/aphrodite`
for homomorphic operations, `@oshun/airmid` for differential privacy / noise
injection).

### 7. Event envelope contract

Every cross-domain event uses the envelope from
`ARCHITECTURE.md § Communication Patterns`:

```typescript
interface DomainEvent<T> {
  id: string; // UUIDv7 (sortable)
  type: string; // e.g. "isis.asset.generated"
  source: string; // owning domain
  timestamp: Date;
  correlationId: string; // request-trace ID
  payload: T;
  metadata: Record<string, unknown>;
}
```

Producers commit owned data first, then publish. Subscribers idempotently key
side effects on `event.id` and store processed-event records in the subscriber's
own database. The Redis Streams transport is durable; failed subscribers are
replayed until the projection succeeds.

### 8. Quality / scanner contracts

Phase-2 introduces seven CI workflows that gate or report on the §2.6 quality
bar. They span every domain — see the
[README](./README.md#quality--scanner-workflows) for the index.

| Workflow                      | Tickets    | What it gates                                               |
| ----------------------------- | ---------- | ----------------------------------------------------------- |
| `stub-indicator-scan`         | V1-P2-0350 | New stub-vocabulary in PR diff                              |
| `silent-random-scan`          | V1-P2-0351 | New `Math.random()` in deterministic functions              |
| `test-coherence-check`        | V1-P2-0352 | Per-file SHAPE/BOOLEAN/COMPUTED ratio (reporting)           |
| `adversarial-grep-trend`      | V1-P2-0355 | Nightly per-domain count + week-over-week delta (reporting) |
| `precommit-hook-verification` | V1-P2-0356 | Husky stub scan contract                                    |
| `stub-indicator-pr-comment`   | V1-P2-0357 | Sticky PR comment with file:line + remediation              |
| `external-api-guard`          | V1-P2-0358 | New raw HTTP calls without retry+CB+telemetry               |

The `// stub:legitimate <reason>`, `// random:legitimate <reason>`, and
`// http:guarded <reason>` allow-list conventions are documented at
[`stub-indicator-allowlist.md`](./stub-indicator-allowlist.md) and inline in
`scripts/external-api-guard.mjs`.

## Canonical end-to-end flows

These are the data flows the platform supports once the foundational work is
complete. Each flow lists the originating user action, the participating
domains, the events emitted, and the durability boundary.

### Flow 1 — Yemaya generates an asset via Isis

```
User in studio-web requests "generate concept art for shot 12"
   │
   ▼
Yemaya BFF (apps/yemaya/api)
   │   Validate: project membership, content-policy gate, budget remaining
   │   Persist: GenerationRequest row in yemaya_db
   │
   ▼  POST /isis/jobs (gRPC, mutual TLS)
Isis API (apps/isis/api)
   │   Persist: Job row in isis_db
   │   Enqueue: BullMQ "generation" queue, attempt budget = 3
   │   Publish: isis.job.queued
   │
   ▼  worker
Isis worker (apps/isis/workers)
   │   Pull image-gen pipeline from libs/isis/pipelines
   │   Call provider via IsisLLMClient OR ComfyUI (libs/isis/comfyui)
   │     ├── @oshun/http-client retries on 5xx / network
   │     ├── Circuit breaker skips dead providers (failover-manager)
   │     ├── HttpTracing emits OTel span; cost-tracker bills tenant
   │   Stream resulting image bytes to MinIO bucket isis-assets/
   │   Persist: Asset row in isis_db with provenance metadata
   │   Publish: isis.asset.generated
   │
   ▼  Redis Streams subscriber
Yemaya event-handler (libs/yemaya/event-handlers)
   │   Project: asset_link row in yemaya_db, key=event.id (idempotent)
   │   Notify: WebSocket push to studio-web via svc-real-time-sync
   ▼
User sees asset in shot 12's gallery.
```

Durability boundary: the Isis Job row + MinIO object survive a crash mid-flow.
Failed worker attempts are retried by BullMQ; failed Yemaya projections are
replayed from the Redis Stream until the row is written.

### Flow 2 — Lilith generates a personalized meditation

```
User in Lilith app taps "today's meditation"
   │
   ▼
Lilith BFF (apps/lilith/bff)
   │   Resolve user profile + progress + biometrics from lilith_db
   │   Resolve relevant teaching from Sophia knowledge graph:
   │     ├── Sophia search uses IrisEmbeddingClient (V1-P2-0030)
   │     └── Returns top-K passages with citations
   │
   ▼  POST /isis/meditation (gRPC)
Isis svc-meditation-generation
   │   IsisLLMClient generates script (cost-tracking-contract enforces quota)
   │   Isis TTS gateway (V1-P2-0140) renders audio (provider chain)
   │   Determinism: when seed is fixed, replay path stores prompt + audio
   │     fingerprint per isis-deterministic-mode.md
   │   Stream audio to MinIO bucket lilith-meditations/
   │   Publish: isis.meditation.ready
   │
   ▼  Redis Streams subscriber
Lilith svc-daily-content
   │   Project: today_meditation row in lilith_db
   │   Push to user via svc-notification (idempotent on event.id)
```

### Flow 3 — Sophia ingests + indexes a document

```
User uploads PDF to apps/sophia/api/ingest
   │
   ▼  Sync intake
Sophia api persists Document row in sophia_db with status=ingesting
Sophia publishes sophia.document.intake on Redis Stream
   │
   ▼  worker
Sophia ingestion worker
   │   Document layout analysis: libs/iris/doc-layout (V1-P2-0066)
   │   OCR pass for scanned pages: libs/iris/ocr (V1-P2-0060)
   │   Vision LLM caption pass: libs/iris/vision-llm (V1-P2-0064)
   │   Chunk + embed via IrisEmbeddingClient
   │   Write: sophia.embeddings index (PostgreSQL pgvector or external store)
   │   Update: Document row status=indexed
   │   Publish: sophia.document.ingested + sophia.index.updated
   │
   ▼ Subscribers
Lilith BFF (knowledge route) projects updated index pointer
Iris RAG cache invalidates per document_id
```

### Flow 4 — Hathor simulates a worldbuilding tick

```
Yemaya world editor calls Hathor "simulate next tick"
   │
   ▼
Hathor api persists SimulationRequest in hathor_db
Hathor publishes hathor.simulation.requested
   │
   ▼ worker
Hathor narrative-lattice engine
   │   Pull world state from hathor_db
   │   For each NPC: IsisLLMClient call shaped by lattice constraints
   │   Detect contradictions, resolve via Sophia citation lookup
   │   Persist: WorldState row + NPC actions in hathor_db
   │   Publish: hathor.simulation.completed
   │
   ▼ Subscribers
Yemaya world editor projects updated state
Sophia indexes new lore entries
```

### Flow 5 — Bellona builds + exports

```
Yemaya project lead clicks "export to Unreal 5"
   │
   ▼
Yemaya api packages export bundle metadata, persists ExportRequest
Yemaya publishes yemaya.export.requested
   │
   ▼ worker
Bellona engine adapter (libs/bellona/adapters/unreal)
   │   Pull authoritative assets from Isis MinIO buckets
   │   Transform to engine-specific format (FBX, USD, …)
   │   Stream to bellona-exports/ MinIO bucket
   │   Persist: Export row in bellona_db
   │   Publish: bellona.export.ready
   │
   ▼ Subscribers
Yemaya UI shows download link
Sophia indexes export manifest for citation
```

## Shared infrastructure topology

```
┌──────────────────────────────────────────────────────────────────┐
│                        CLIENT LAYER                              │
│ studio-web · studio-desktop · iris-mobile · cli · sdks (ts/cpp/py) │
└─────────────────────────────────────────────────────┬────────────┘
                                                       │
                  ┌────────────────────────────────────┴────────────────────────────────────┐
                  │                          API GATEWAY (Hono + OpenAPI)                    │
                  │       JWT (auth-orchestrator) · rate-limit · @oshun/http-client          │
                  └─────────────────────────────────┬────────────────────────────────────────┘
                                                    │
        ┌───────────────────┬──────────────────────┬┴──────────────────┬──────────────────┬───────────────────┐
        ▼                   ▼                      ▼                   ▼                  ▼                   ▼
     Yemaya               Lilith                  Isis                Sophia            Hathor              Bellona
    (yemaya_db)         (lilith_db)            (isis_db)            (sophia_db)       (hathor_db)         (bellona_db)
        │                   │                      │                   │                  │                   │
        └─────┬─────────────┴──────────────────────┴───────────────────┴──────────────────┴───────────────────┘
              │
    ┌─────────┴────────────────────────────────────────────────────────────────────────────────────────────┐
    │                          SHARED INFRASTRUCTURE                                                        │
    │                                                                                                       │
    │  Redis Streams + Pub/Sub  (event bus)                                                                 │
    │  Redis  (cache / rate-limit buckets)                                                                  │
    │  BullMQ on Redis  (job queues per domain)                                                             │
    │  PostgreSQL  (one DB per domain — Drizzle ORM, V1-P2-0233)                                            │
    │  pgvector  (Sophia embeddings index; optional Qdrant via observability profile)                       │
    │  MinIO  (asset object store: yemaya-assets/, isis-assets/, lilith-meditations/, bellona-exports/)     │
    │  Vault / VaultClient  (secrets, with retry + cache; libs/lilith/service-lib/vault-client.ts)          │
    │  OpenTelemetry collector → Prometheus + Tempo + Loki                                                  │
    │     dashboards live at docker/observability/grafana/dashboards/<domain>/                              │
    │  Mailpit (dev) / SES (prod)  (notifications)                                                          │
    └───────────────────────────────────────────────────────────────────────────────────────────────────────┘
```

Per-domain database isolation is enforced at the schema level — domains do not
query each other's tables. Cross-domain reads happen via API or via projected
read-models written by event-handler subscribers.

## Trust boundaries

The Phase-2 system has three trust boundaries that gate every flow:

1. **External user → API gateway.** JWT auth, content-policy filtering, rate
   limit per tenant. The gateway is also where `@oshun/http-client` is wired up
   — every outbound provider call from a domain goes through the same retry +
   circuit-breaker + telemetry stack as the inbound request.
2. **Domain → Vault.** No domain stores raw vendor API keys; every key is
   fetched via `VaultClient.getSecret()` with the 5-minute in-memory cache + 3x
   retry. Per V1-P2-0358 the vault client itself is on the wrapper allow-list
   (it's a primitive; it can call `https.request` directly).
3. **Domain → Domain.** mTLS gRPC for synchronous call-outs; signed event
   envelopes for asynchronous broadcast. Subscribers verify the event's `source`
   and idempotently apply by `event.id`.

## Observability surface

Every flow emits the same three signal types:

- **Logs** — `pino` structured JSON, scoped by `correlationId` ==
  `event.correlationId` so a single user action can be reconstructed end to end
  across domain boundaries.
- **Metrics** — Prometheus scrape on every `apps/*/api` service. Per-domain
  dashboards under `docker/observability/grafana/dashboards/<domain>/`
  enumerated in the [README](./README.md#per-domain-grafana-dashboards).
- **Traces** — OpenTelemetry OTLP via the `HttpTracing` interceptor in
  `@oshun/http-client`. Tempo retains 30 days; Grafana
  `oshun/platform-overview.json` is the top-of-funnel view.

For per-subsystem operational reference, every domain owns a runbook generated
from the [Runbook Template](./runbook-template.md) (V1-P2-0372), and every
Tier-A subsystem owns a threat model generated from the
[Threat Model Template](./threat-model-template.md) (V1-P2-0373).

## Differences from `ARCHITECTURE.md`

This document is the Phase-2 view; the table below highlights what changes from
the Phase-1 baseline in [`ARCHITECTURE.md`](../../../ARCHITECTURE.md):

| Concern        | Phase-1 baseline                                 | Phase-2 contract                                                                 |
| -------------- | ------------------------------------------------ | -------------------------------------------------------------------------------- |
| LLM provider   | Direct vendor SDK calls scattered across domains | Centralized `IsisLLMClient` (V1-P2-0001), cost-tracked, deterministic-replayable |
| Embeddings     | Per-domain vendor calls                          | `IrisEmbeddingClient` (V1-P2-0030)                                               |
| HTTP transport | Mix of raw `fetch`/`axios` and ad-hoc wrappers   | `@oshun/http-client` everywhere, gated by `external-api-guard` (V1-P2-0358)      |
| Crypto         | Stub primitives in privacy/security paths        | Real primitives per `crypto-replacement-targets.md` (V1-P2-0176)                 |
| Persistence    | Mixed (some Drizzle, some placeholder)           | Uniform Drizzle (V1-P2-0233)                                                     |
| Quality bar    | Manual review                                    | 7 CI workflows enforce §2.6 (V1-P2-0350..0358)                                   |
| Documentation  | Free-form per domain                             | Templated docs under `docs/releases/p2/` (V1-P2-0370..0373)                      |

When the items in the right column have all moved from `[ ]` to `[x]` in
[`OSHUN_V1_TODOS_P2.md`](../OSHUN_V1_TODOS_P2.md), this document IS the
operating architecture; any divergence in code is a defect to fix, not a
documentation update to make.
