Oshun Platform · Architecture

Foundations

A focused page within the Oshun Platform Architecture documentation. The full map and every sibling page live in the Architecture hub.

12sections14 minread1diagram6tables

On this page

The foundation layer is the shared substrate every V1 surface stands on: the contracts that define each object once, the persistence machinery that keeps those contracts and the database in lockstep, the BFF middleware chain that guards every write, and the queue/event/integration/identity plumbing that makes the customer-facing domains and product surfaces interoperable. It serves engineers across every domain — none of them re-implement idempotency, residency, audit, or eventing — and it sits underneath the High-Level Architecture, between the domain adapters and the raw infrastructure. The work is tracked in TODOS § 1, and almost every piece below is real code with tests already in the repo. Where something is contract-level rather than a live deployment, this page says so.

Status candor. This area is overwhelmingly implemented, not aspirational. Contracts, the persistence drift/index/migration/tombstone test quartet, the OAuth 2.1/PKCE module, the role/scope model, the partial-failure envelope contract, the durable queue + DLQ + SLA monitor, the topic-registry + outbound-delivery + webhook-simulator event bus, the inbound connectors, the data-residency enforcer, the idempotency middleware, and the audit-platform hash chain all exist with tests. The two honestly-scoped caveats: the OAuth module is contracts/state-machine level (token types, PKCE validation patterns, refresh/revocation surface) and does not stand up a live authorization-server deployment. The BFF request-lifecycle diagram below describes a real composition of building blocks that are all present, but no single production route was traced end-to-end proving every middleware fires in exactly that order.


1. Contracts — libs/contracts/#

V1 holds to one source of truth per object class. Each contract is a Zod schema with a contract-test fixture that round-trips a known-good payload, so a schema change that breaks an existing object fails CI rather than silently shipping.

  • Common contracts live under libs/contracts/src/common/. That directory now contains 285 files161 implementation .ts modules plus their co-located .spec.ts/.test.ts siblings — covering admin, billing, incident, persona, voice, persistence, and many more cross-cutting object classes. (Earlier drafts of this doc cited "≈265 files," which was stale and low.)
  • Domain contracts live under libs/contracts/src/<domain>/.
  • Everything re-exports through libs/contracts/src/index.ts as @oshun/contracts. To avoid naming collisions across families, several domains are exported as namespaces rather than flat symbols. The index uses export * as for NisabaContracts (./nisaba), MetisContracts (./metis), and VeritasContracts (./veritas) — beyond the three customer domains, also for V3Contracts (./v3/index), V6Contracts (./v6/index), V9Contracts (./v9/index), and LivingSceneContracts (./living-scene/index). Where a generic name would collide, the index re-exports under an explicit alias — e.g. ScoreSchema as LivingSceneScoreSchema and deepParseScore as deepParseLivingSceneScore — so consumers can import a Living Scene score without shadowing another domain's ScoreSchema.
  • Specialized contract packages exist for Veritas, Psyche, Iris, Maat, Cybele, Concordia, and Saraswati when a consumer needs a smaller surface than the umbrella package.

The partial-failure envelope contract#

The standardized partial-failure envelope (§5, below) is not a convention — it is a first-class Zod contract at libs/contracts/src/common/partial-failure-envelope.ts. Its exported surface:

Symbol What it is
PartialFailureEnvelopeSchema The base schema: { results: unknown[], errors: PartialFailureError[], partial: boolean }.
PartialFailureErrorSchema / PartialFailureError A single failure: { domain: string, stage?: string, message: string } (each string non-empty; stage optional).
createPartialFailureEnvelopeSchema(resultSchema, errorSchema?) Generic factory that produces a typed envelope schema for a specific result/error shape, defaulting the error schema to PartialFailureErrorSchema.
buildPartialFailureEnvelope({ results, errors, partial? }) Constructs a well-formed envelope; throws if a supplied partial flag disagrees with whether errors is non-empty.
isPartialFailureEnvelope(value) Structural type guard.

The base schema's cross-field refinement is what makes it trustworthy: a superRefine enforces that partial: true requires at least one error and partial: false requires an empty errors array. An envelope that claims to be partial but carries no errors — or claims success while carrying errors — is rejected at parse time:

ts
PartialFailureEnvelopeSchema.parse({
  results: [
    /* 8 ok */
  ],
  errors: [{ domain: 'metis', stage: 'grading', message: 'timeout' }],
  partial: true, // valid: partial=true ⇒ errors non-empty
});

This is why the BFF can normalize a fan-out across several domains into one response that is honest about which slices succeeded — the contract makes a lie unrepresentable.


2. Adapters — libs/oshun/domain-* and substrate adapters#

  • Every customer-facing domain exports one canonical adapter (the typed read APIs the shared shell, admin, and assistant consume) and one adapter variant for boundary-specific concerns (BFF routes, fixtures).

  • Substrate adapters present the same shape regardless of whether the underlying substrate is local-only or distributed, so callers don't branch on deployment topology.

  • Adapters are wired through the BFF via the domain registry. The registry at libs/oshun/domain-registry/src/registry.ts exports OSHUN_DOMAIN_IDS and DOMAIN_REGISTRY, and pins the shell's primary domain via OSHUN_SHELL_PRIMARY_DOMAIN = 'tara'. Each of the six customer domains carries a 'bff-base-path':

    Domain bff-base-path
    Tara /api/oshun/domains/tara
    Veritas /api/oshun/domains/veritas
    Nyx /api/oshun/domains/nyx
    Arete /api/oshun/domains/arete
    Nisaba /api/oshun/domains/nisaba
    Metis /api/oshun/domains/metis

See the Subsystem Glossary and Customer-Facing Domains for what each domain owns.

The platform-foundations package#

Several of these adapter-adjacent concerns are not scattered across the tree — they have a single home. libs/oshun/platform-foundations/ (@oshun/platform-foundations) re-exports exactly nine subsystems from its src/index.ts:

Subsystem Concern
service-discovery Resolving and addressing domain services.
public-api The §27 Public API platform, incl. OAuth 2.1/PKCE (§10).
shared-contracts Cross-subsystem contract surface.
role-model Canonical roles, scopes, field filtering (§10).
step-up Step-up authentication for sensitive actions.
secrets Secret handling.
configs Configuration.
rollback Rollback workflows.
abuse-controls Abuse/throttling controls.

3. Persistence — libs/oshun/persistence/#

The persistence layer's job is to keep Zod contracts and the database from drifting apart, and to make user-data deletion both safe and auditable.

  • Schema rendering. prisma-renderer.ts and zod-prisma-introspection.ts drive Prisma schema from Zod, not the reverse, so the contract remains the single source of truth.
  • Drift test. contract-persistence-registry.test.ts (backed by contract-persistence-registry.ts) fails on field divergence between the Zod contract and the persistence layer — a renamed or dropped field is a red build, not a runtime surprise.
  • Tombstone semantics. Mandatory on every user-data table: deletion is a soft tombstone that propagates and is audit-logged; there is no silent re-creation. Covered by tombstone-semantics.test.ts.
  • Required indexes. index-requirements.test.ts validates per-table indexes via EXPLAIN checks in CI, so an unindexed hot path can't land quietly.
  • Migration plans. migration-plan.test.ts keeps migrations idempotent and dry-runnable.

Beyond the core quartet, the same library houses the DSAR deletion machinerydsar-deletion-cascade.ts (with both unit and .integration tests) and dsar-erasure-runtime.ts — which is how a privacy-operator's erasure request fans the tombstone across every owning table. This is the persistence-side counterpart to the Trust, Safety, and Privacy workflows.


4. OpenAPI and codegen — libs/openapi/#

  • Specs live under libs/openapi/src/specs/. There are 15 real spec directories: arete, bellona, calliope, concordia, hathor, isis, lilith, metis, nisaba, nyx, oshun-bff, sophia, tara, veritas, and yemaya — plus top-level main.yaml and v3.yaml.
  • Schemas drive specs (Zod → OpenAPI), never the reverse, so the contract layer remains authoritative.
  • Typed clients are generated per domain into libs/<domain>/api-client/.
  • CI gate: spec drift against runtime fails the build.

5. Idempotency, partial-failure, response normalization#

  • Idempotency middleware lives in libs/shared/http-client/src/idempotency.ts. It defaults to the Idempotency-Key header (DEFAULT_IDEMPOTENCY_HEADER) with a 24-hour replay window (DEFAULT_IDEMPOTENCY_TTL_MS = 24 * 60 * 60 * 1000). The IdempotencyMiddleware class — constructed via createIdempotencyMiddleware() — gives request deduplication and replay-safe semantics: a repeated key returns the stored envelope instead of re-executing the write.
  • The same libs/shared/http-client/ package ships the rest of the outbound hardening: circuit-breaker.ts, ssrf-guard.ts, retry.ts, tenant-context.ts, and tracing.ts. So an outbound call from the BFF to a domain (or to an external connector) is idempotent, breaker-protected, SSRF-guarded, retried, tenant-scoped, and traced through one composed client.
  • Partial-failure envelope. The standardized shape { results[], errors[], partial } is the §1 contract above — used across the BFF and domain services so a multi-domain fan-out can report exactly which slices failed without collapsing into a single opaque error.
  • Job contracts. Retry/dedup/replay/dead-letter contract tests run on every job submitter.

6. Multi-tenant routing — libs/shared/data-residency/#

  • Tenant-aware middleware reads X-Tenant-ID (or the token claim) and propagates the tenant context through every downstream call.
  • Residency-aware traffic shaping. traffic-shaping.ts, home-zone.ts, and enforcer.ts route a request to the user's primary (home-zone) data plane unless explicit cross-region consent is present. dsr-routing.ts handles data-subject-request routing, and an injected ResidencyAuditPublisher seam (exercised by audit-platform-bridge.spec.ts) ties residency decisions into the audit substrate (§10).
  • Tenant-aware caching keys prevent cross-tenant cache pollution.
  • Tenant-aware feature flags and experiment scoping.
  • Cross-tenant leakage tests live under tests/security/tenant-isolation/.

This is the enforcement spine behind Data Architecture and Tenancy and the residency obligations in Security, Privacy, and Compliance.


7. Background-job substrate — libs/shared/queue/#

A single durable queue is shared by editorial, asset, agentic-AI, notification, billing, and integration jobs — there is not a bespoke queue per domain.

  • durable-queue.ts — priority classes, replay, deduplication, observability hooks.
  • dead-letter-queue.ts — the DLQ for jobs that exhaust retries.
  • sla-monitor.ts — a per-job-class SLA monitor.
  • memory-queue.ts — an in-memory implementation for tests/local development.
  • worker.ts — the worker loop that drains the queue.

8. Webhook and event-bus plumbing — libs/shared/event-bus/#

The event bus is @oshun/event-bus; its surface is export class EventBus implements IEventBus with a createEventBus(config: EventBusConfig) factory. Alongside it:

  • topic-registry.ts — topics with schema versioning (DEFAULT_EVENT_TOPIC_REGISTRY), so a payload that doesn't match the registered topic schema is rejected before fan-out.
  • outbound-delivery.ts — outbound signing keys, retry/backoff, dead-letter inspection, replay, and per-event audit for tenant-facing webhooks.
  • webhook-simulator.ts — a simulator tenant integrations can use in a dev sandbox.

How the bus actually works — it is not Redis Streams#

Earlier drafts (and the cascade diagram below) labeled the bus "over Redis Streams." That is the one outright-wrong technical claim, and it's worth correcting precisely, because the real design is more interesting. The implementation imports import { Redis } from 'ioredis' and opens two separate connectionsthis.pub and this.sub — but it uses none of the native Streams commands (no XADD, XREAD, XREADGROUP, or XGROUP). Instead it composes Streams-like guarantees out of simpler primitives:

Concern Mechanism
Fan-out Redis pub/subthis.pub.publish(channel, …) out, pattern-subscribe (pmessage) in.
Replay source A TTL-bounded key per event (setex, default eventTtl = 24h), so a crash-restart can re-deliver unacked work.
Delay / nack A durable sorted set named scheduled (zadd(scheduledKey, deliverAt, …)); a scheduler loop (default 250 ms) pulls due entries. setTimeout state is never the only copy.
Consumer groups A per-(eventId, group) SET … NX claim key (this.pub.set(key, '1', 'EX', eventTtl, 'NX')): only the winner runs the handler. Without a group, every matching subscription runs (classic broadcast).
Dead letter A list (ordered pagination) plus a hash (O(1) id lookup); removeDeadLetter uses LREM for atomic removal.

So the bus is genuinely durable and consumer-group-capable, but via pub/sub + TTL keys + a sorted set + SET NX claims — not XADD/XREADGROUP. Treat any remaining "Redis Streams" wording in the hub doc as a known label error.


9. Inbound integration plumbing — libs/shared/inbound-integrations/#

The package's src/index.ts re-exports 15 connector modules — more than the nine the original table listed. Several are substantial real implementations (e.g. oneroster.ts ≈1081 LOC, lms.ts ≈1425 LOC, identity.ts ≈1048 LOC), not thin wrappers.

Module Connector family / role V1 use
types.ts Shared types Connector contracts used by all the modules below.
lms.ts LMS LTI 1.3, LTI Advantage, SCORM fallback for Metis institutions.
lti-verification.ts LMS LTI launch/credential verification, split out from lms.ts.
scorm-rte.ts LMS SCORM run-time environment (legacy).
scorm-2004-rte.ts LMS SCORM 2004 run-time environment (second, distinct RTE).
oneroster.ts OneRoster Rostering sync with conflict reporting and dry-run.
identity.ts Identity SAML 2.0, OIDC, SCIM 2.0 for tenant SSO.
calendar.ts Calendar Google/Apple/Outlook two-way sync for Tara/Arete/Nyx/Metis.
calendar-google-transport.ts Calendar Google-specific transport for the calendar connector.
payment.ts Payment (fiat) Fiat-rail entitlement upgrades (Stripe-class; Telegram payments). V1.x optional — the V1 primary path is non-custodial crypto via the Aje domain (libs/aje/) plus libs/oshun/payments-bridge/.
telemetry.ts Telemetry xAPI/cmi5/Caliper export to institutional sinks.
byom.ts BYOM Tenant-provided model endpoints for Metis study.
byom-model.ts BYOM The model abstraction backing byom.ts, exported separately.
notification.ts Notification Slack/Teams notification sinks for institutional surfaces.
health.ts Health Connector health probes, circuit breakers, version pinning.

See Messaging Channels and the Aje bridge in Support, Entitlements, Billing, and the Aje Entitlement Bridge for how these surface to customers.


10. Identity, auth, and audit#

Auth primitives — libs/shared/auth-primitives/#

Richer than "JWT and session primitives." The package ships, among others: jwt.ts, session.ts, api-key.ts, password.ts, oauth-client.ts, oauth-revoke.ts, token-refresh.ts, token-audit.ts, totp.ts (TOTP for step-up), platform-roles.ts, and tenant-isolation.ts — each with a co-located .spec.ts. So API keys, password hashing, OAuth client + revocation, refresh rotation, token auditing, TOTP step-up, and tenant isolation are all primitives, not bespoke per-app code.

Workspace identity client — libs/oshun/auth/#

@oshun/auth-client is the workspace-side identity and session client every Oshun app consumes.

The role/scope model — platform-foundations/.../role-model/#

libs/oshun/platform-foundations/src/role-model/role-model.ts defines the ten CANONICAL_ROLES — note the precise names; prose elsewhere uses looser labels like "support" that don't match the code:

Canonical role Scopes granted (scopesFor(role))
customer customer.self, customer.shell
creator customer.self, customer.shell, creator.studio
support-agent support.case.read, support.case.write
reviewer review.queue.read, review.queue.decide
moderator review.queue.read, review.queue.decide, moderation.decide
privacy-operator privacy.dsar.execute, privacy.audit.read, admin.audit.global
model-operator model.registry.read, model.registry.promote
persona-operator persona.registry.read, persona.registry.publish
tenant-admin tenant.console, admin.audit.global
admin-leadership admin.users.act, admin.scope.grant, admin.breakglass.act, admin.audit.global

The full SCOPE_KEYS list (21 scopes) is the vocabulary authorizeAction(), authorizeObjectAccess(), and filterFieldsByRole() enforce. creator and tenant-admin are distinct canonical roles — a fact older role prose omits. The support role is support-agent, not support. Use these exact strings when wiring RBAC.

OAuth 2.1 / PKCE — platform-foundations/.../public-api/oauth.ts#

The §27 Public API platform's OAuth module is contracts/state-machine level: it validates and computes, but does not host a running authorization server. Concretely it exports OAUTH_GRANT_TYPES, TOKEN_TYPES = ['access', 'refresh'], and a PKCE validator built on PKCE_CODE_CHALLENGE_PATTERN = /^[A-Za-z0-9_-]{43,128}$/ (the RFC 7636 URL-safe-Base64 challenge length window). Its functions cover the real surface: validateAuthorizationCodeRequest, validateAuthorizationCodeRedeem, verifyPkceChallenge, validateAccessToken, evaluatePublicApiQuota, buildPublicApiAuditEvent, redeemRefreshToken, and revocationCascade. Error codes include 'pkce-required' (authorization code request with no challenge) and 'pkce-verification-failed' (redeem whose verifier doesn't match). PKCE validation patterns and the refresh/revocation surface are confirmed; a live authorization-server deployment is not part of this module.

RBAC checks — libs/shared/identity/#

RBAC and permission checks are consumed across domains.

Audit substrate — libs/shared/audit-platform/#

Far more than one sentence's worth: libs/shared/audit-platform/ is a large real substrate (~40 implementation modules plus tests). At its core is hash-chain.ts, an append-only hash chain, so an audit record cannot be silently altered or deleted without breaking the chain. Around it: schema-versioning.ts, retention.ts, escalation-rules.ts, support-escalation.ts, compliance-attestation.ts, watermark-verification.ts, synthetic-media-labeling.ts, voice-likeness-consent.ts, and provenance-bundle validation (with .integration specs). This is the substrate behind Iris admin inspection, DSAR review, the deletion workflow, review packages, synthetic-media labeling, and voice-likeness consent — the trust-and-safety record of last resort.


How the foundation composes — the BFF request lifecycle#

Every customer- or operator-issued write passes the same middleware sequence before reaching a domain adapter. Idempotency is checked first (replays return the cached envelope); tenant and residency resolve next so the rest of the pipeline can short-circuit on policy. The body is validated against the Zod contract; the adapter commits owned data with tombstone semantics; an event is published and an audit record appended. Each station maps to a real building block above — idempotency → http-client/idempotency.ts; tenant + residency → data-residency/; validation → @oshun/contracts; adapter → the domain adapters keyed by DOMAIN_REGISTRY; event → @oshun/event-bus; audit → audit-platform/hash-chain.ts. (The ordering shown is the intended composition; no single production route was traced proving every middleware fires in exactly this order.)

sequenceDiagram autonumber actor C as Client participant BFF as Oshun BFF participant Idem as Idempotency Cache participant T as Tenant Middleware participant R as Residency Enforcer participant V as Zod Contract Validator participant D as Domain Adapter participant DB as Domain DB participant EB as Event Bus participant A as Audit Platform C->>BFF: POST /api/oshun/domains/X<br/>Idempotency-Key · X-Tenant-ID · JWT BFF->>Idem: lookup(idempotencyKey) alt cache hit Idem-->>BFF: stored response BFF-->>C: 200 OK (replayed) else cache miss BFF->>T: resolve tenant + scopes T-->>BFF: tenant context · homeZone BFF->>R: check residency vs request region R-->>BFF: allow / require consent BFF->>V: validate body V-->>BFF: parsed payload BFF->>D: invoke(operation, payload) D->>DB: write owned data (tombstone-aware) DB-->>D: ack D->>EB: publish domain event<br/>{correlationId} D->>A: append mutation record D-->>BFF: result BFF->>Idem: store(key, response) BFF-->>C: 200 OK end

A partial fan-out at the BFF — say one operation that touches three domains — collapses into the §1 partial-failure envelope rather than a single opaque error, so the client learns exactly which slice failed.

The asynchronous side and the Veritas retraction-cascade walkthrough live on the dedicated Communication Patterns page (which also inherits the corrected "not Redis Streams" framing from §8). For internal service-to-service calls, the gRPC layer is real too: libs/proto/ carries buf.work.yaml, a generated/ tree, and ~24 proto domain directories under src/agent, ai, asset, auth, bridge, collaboration, concordia, generation3d, hathor, health, isis, loadbalancing, oshun, oya, pipeline, procedural, project, reflection, rendering, shared, sophia, splatting, user, and common — well beyond the Psyche/Isis/Sophia trio commonly cited.