Shared Domain — Architecture

Architectural overview of the @oshun/* shared infrastructure layer: 42 libraries, dependency philosophy, and how domains consume shared infrastructure.

The shared domain (libs/shared/) is the platform's horizontal foundation. It solves a common problem in multi-team monorepos: without a shared layer, each domain team reinvents authentication, logging, error handling, and metrics in slightly incompatible ways. The result is operational chaos — logs that can't be correlated, errors that serialize differently per service, and security primitives that drift out of sync.

The shared domain prevents that by owning every cross-cutting concern once. A domain engineer should never write their own JWT verification, Redis connection pool, or Pino configuration — they import from @oshun/* and stay focused on domain logic.

Every @oshun/* library is a pure infrastructure or utility library. Business logic, user-facing features, and domain-specific data models live in their respective domain directories (libs/isis/, libs/lilith/, etc.).

Role of the Shared Domain#

The shared domain establishes four system-wide contracts:

1. A consistent runtime contract — every service logs the same way, traces the same way, handles errors the same way
2. Infrastructure abstraction — domains write to @oshun/database, not raw pg; they publish via @oshun/event-bus, not raw Redis pub/sub
3. Security baseline — authentication, authorization, rate limiting, and audit logging are infrastructure, not each domain's responsibility
4. Operational visibility — metrics, tracing, and health checks are wired in uniformly across all services

Library Map#

The 42 packages are organized into functional groups. The tree below shows the directory name under libs/shared/, the npm package name, and a brief note on what each package provides.

libs/shared/   (42 packages)
│
├── CORE TYPES AND ERRORS
│   ├── types/              → @oshun/types        (branded IDs, Result, pagination, envelopes)
│   ├── errors/             → @oshun/errors       (OshunError hierarchy, code registry)
│   └── config/             → @oshun/config       (env-var loading, Zod validation, flags)
│
├── AUTHENTICATION
│   ├── auth/               → @oshun/auth         (auth service, RBAC, lockout, OAuth helpers)
│   ├── auth-primitives/    → @oshun/auth-primitives (JWT, session, password, API key, TOTP)
│   └── identity/           → @oshun/identity     (verify-only JWT, gateway auth, mTLS)
│
├── DATA AND STATE
│   ├── database/           → @oshun/database     (PostgreSQL pool, query builder, migrations)
│   ├── cache/              → @oshun/cache        (Redis cache-aside, locks, circuit breaker)
│   ├── storage/            → @oshun/storage      (S3/MinIO, pre-signed URLs, multipart)
│   ├── queue/              → @oshun/queue        (BullMQ job queue, durable substrate, DLQ)
│   └── migration/          → @oshun/migration    (cross-domain migration framework)
│
├── COMMUNICATION
│   ├── event-bus/          → @oshun/event-bus    (Redis pub/sub, typed envelopes, topic registry)
│   ├── websocket/          → @oshun/websocket    (WebSocket server, rooms, Redis adapter)
│   └── http-client/        → @oshun/http-client  (fetch wrapper, circuit breaker, retry, SSRF)
│
├── OBSERVABILITY
│   ├── logging/            → @oshun/logging      (Pino, structured, context propagation)
│   ├── metrics/            → @oshun/metrics      (Prometheus, OpenTelemetry metrics)
│   └── tracing/            → @oshun/tracing      (OpenTelemetry traces, Jaeger/X-Ray)
│
├── PLATFORM SERVICES
│   ├── gateway/            → @oshun/traefik-config      (Traefik v3 config builders/generators)
│   ├── health/             → @oshun/health       (liveness/readiness probes, aggregator)
│   ├── rate-limit/         → @oshun/rate-limit   (sliding/fixed/token-bucket, quotas, abuse)
│   ├── security/           → @oshun/security     (audit log, content scanning, secrets)
│   └── service-discovery/  → @oshun/service-discovery (Redis registry, load balancing)
│
├── AI AND GPU
│   ├── ai/                 → @oshun/ai           (LLM providers, routing, prompts, local LLM)
│   ├── ai-advanced/        → @oshun/ai-advanced  (adapters, benchmarking, edge, research)
│   ├── gpu-dispatcher/     → @oshun/gpu-dispatcher (RunPod job dispatch, queuing, fallback)
│   └── runpod-client/      → @oshun/runpod-client  (typed RunPod Serverless API client)
│
├── SECURITY, AUDIT, AND COMPLIANCE
│   ├── crypto/             → @oshun/crypto        (noble-backed hashing/signing/AEAD/KDF)
│   ├── audit-platform/     → @oshun/audit-platform (canonical audit ingest, append-only store)
│   ├── data-residency/     → @oshun/data-residency (region/residency enforcement)
│   ├── region-rules/       → @oshun/region-rules  (V2 regional content rules)
│   └── review-persistence/ → @oshun/review-persistence (review-package persistence)
│
├── DOCUMENT AND MEDIA PROCESSING
│   ├── vision-llm/         → @oshun/vision-llm    (vision-locate wrapper over IsisLLMClient)
│   ├── layout-analyzer/    → @oshun/layout-analyzer (PubLayNet layout analysis, mAP eval)
│   ├── ocr/                → @oshun/ocr           (tiered OCR client; Tesseract/vision/cloud)
│   ├── ml/                 → @oshun/ml            (ONNX runtime, model loading, benchmark)
│   └── native-libs/        → @oshun/native-libs   (typed wrappers over pinned native deps)
│
├── INFRASTRUCTURE AND OPERATIONS
│   ├── infrastructure/     → @oshun/infrastructure (performance/security/monitoring managers)
│   ├── release-management/ → @oshun/release-management (rollback plans, rehearsal engine)
│   └── documentation/      → @oshun/documentation  (architecture-documentation models)
│
├── INTEGRATION SURFACES
│   ├── inbound-integrations/   → @oshun/inbound-integrations (LMS/OneRoster/calendar/BYOM/…)
│   └── tara-live-class-booking/ → @oshun/tara-live-class-booking (Tara live-class primitives)
│
└── DEVELOPER TOOLS
    └── testing/            → @oshun/testing      (mocks, fixtures, test containers, fuzz)

A few important clarifications about packages that are sometimes mischaracterized:

@oshun/event-bus uses Redis pub/sub (ioredis), not Kafka. Its at-least-once replay source is a TTL-bounded Redis key plus a per-event processed-marker hash — there is no Kafka topic and no transactional outbox worker. @oshun/release-management implements rollback-plan validation and rehearsal for six v1 surfaces; it is not a versioning/changelog/beta-program tool. @oshun/documentation implements architecture-documentation models only; API-reference generation, tutorials, and certification are not present in source. @oshun/errors serializes via toJSON() / toResponse() — there is no RFC 7807 (toProblemDetails) serializer.

Dependency Hierarchy#

The shared domain sits at the very bottom of the monorepo's internal dependency graph. No @oshun/* library may depend on any domain library. Arrows flow only downward, ensuring domain teams can upgrade shared libraries without circular rebuild problems.

All domain libraries (@isis/*, @lilith/*, @yemaya/*, ...)
        │
        ▼
@oshun/* shared libraries
        │
        ▼
npm packages (pg, ioredis, bullmq, pino, prom-client, @opentelemetry/*,
              @noble/*, ws, AWS SDK S3, etc.)
        │
        ▼
Infrastructure (PostgreSQL, Redis, S3-compatible storage, RunPod)

A few @oshun/* packages depend on @oshun/contracts (the shared Zod contract package) — specifically audit-platform, data-residency, layout-analyzer, review-persistence, and vision-llm. These remain horizontal: @oshun/contracts is itself a shared contract package, not a domain library.

Service Integration Pattern#

Every Oshun domain service follows the same four-phase startup sequence when integrating shared infrastructure. The pattern ensures that configuration is validated before anything connects, and that all observability and security middleware is installed consistently across every service.

// 1. Load configuration
//    @oshun/config validates all env vars at startup, failing fast with a clear
//    error rather than silently using a zero-value or undefined.
const config = loadConfig();

// 2. Initialize shared infrastructure
//    Each client is created against the validated config. Connection pooling,
//    circuit breakers, and retry policies are all wired here.
const db = createPostgresClient(config.database);
const cache = createRedisClient(config.redis);
const eventBus = createEventBus({
  redisUrl: config.redisUrl,
  sourceDomain: 'isis',
});
const logger = createLogger({ service: 'isis-api' });
const metrics = createMetricsServer({ port: 9090 });
const tracer = createTracer({ service: 'isis-api' });

// 3. Build application
//    Auth, rate limiting, logging, and tracing middleware are registered once
//    at the application level — domain route handlers do not need to re-install them.
const app = new Hono();
app.use(authMiddleware()); // from @oshun/auth
app.use(rateLimitMiddleware()); // from @oshun/rate-limit
app.use(loggingMiddleware()); // from @oshun/logging
app.use(tracingMiddleware()); // from @oshun/tracing
app.get('/health', healthHandler(db, cache)); // from @oshun/health

// 4. Register domain routes
//    Domain logic receives the already-initialized infrastructure clients as
//    constructor arguments, keeping domain code free of infrastructure concerns.
app.route('/api/v1', domainRoutes(db, cache, eventBus));

Multi-Database Architecture#

Each domain has its own isolated PostgreSQL database to prevent accidental cross-domain data coupling. A bug in one domain's migrations cannot corrupt another domain's schema, and each domain can scale its database independently.

@oshun/database and @oshun/config provide the connection tooling; domains own their schemas entirely. The core domain-isolated databases used in local development are:

Additional domains carry their own <DOMAIN>_DATABASE_URL connection strings in the root .env. Each service loads its database config with loadDatabaseConfig() from @oshun/config; there is no per-domain loader function — all domain databases use the same loader with the appropriate connection-string environment variable.

Cross-domain data references use userId (a shared identifier) rather than foreign keys across database boundaries. This is the intentional isolation boundary: domain databases contain only domain-specific tables.

Event Bus Architecture#

The event bus solves inter-domain communication without tight coupling. When Isis generates an asset, it publishes a typed event. Sophia, Yemaya, and the shell can subscribe to that event independently, and Isis knows nothing about its subscribers.

The transport is Redis pub/sub with a TTL-bounded durability layer — not Kafka and not a transactional outbox. The flow looks like this:

Domain Service A                    Domain Service B
      │                                   │
      │ publish('isis.asset.generated')   │
      ▼                                   │
@oshun/event-bus                          │
      │                                   │
      ├── SETEX  oshun:events:event:<id>  │  (TTL-bounded replay copy)
      └── PUBLISH oshun:events:channel:isis.asset.generated
                  │                       │
                  └────────────────────── ▼
                                    @oshun/event-bus
                                    PSUBSCRIBE 'isis.asset.*'
                                    consumer group claims via SET NX EX
                                    (one member runs the handler)

Delivery is at-least-once. The durability mechanism works in four steps:

1. publish() stores a TTL-bounded copy of the event under oshun:events:event:<id> (default TTL 24 h) before fanning it out over Redis pub/sub. This is the replay source.
2. A per-event Redis HASH records which subscriptions and consumer groups have acked. This is what makes replay safe — already-processed events are not re-delivered.
3. replayUnacked(), called on boot after wiring subscriptions, scans the persisted events and redelivers any that still lack a processed marker — recovering work lost to a crash between publish and ack.
4. Delayed and nacked events live in a durable Redis sorted set (oshun:events:scheduled) drained by a scheduler loop every 250 ms, so retry state survives a process restart.

Observability Architecture#

All three observability signals flow through shared libraries to a common backend. This means every service's logs, metrics, and traces can be correlated in a single dashboard — a feature that would be impossible if each team chose their own logging framework.

Every Domain Service
      │
      ├── @oshun/logging → Pino JSON → Log aggregator (Loki/ELK)
      │
      ├── @oshun/metrics → Prometheus /metrics → Prometheus scraper → Grafana
      │
      └── @oshun/tracing → OTLP → Jaeger (dev) / AWS X-Ray (prod)

Trace spans carry OpenTelemetry semantic-convention attributes so Jaeger can build accurate service maps and identify bottlenecks by operation type:

• service.name: identifying the specific service
• http.method and http.url for HTTP spans (HttpSpanAttributes)
• db.system and db.statement for database spans (DbSpanAttributes)
• RPC attributes for service-to-service calls (RpcSpanAttributes)
• correlation-id and user-context attributes where available

Security Architecture#

Security controls are layered at the platform level so a domain engineer cannot accidentally skip authentication or rate limiting on a new route. Each layer has a specific responsibility and passes the request to the next layer only on success.

Inbound request
      │
      ▼
Traefik gateway  (config generated by @oshun/traefik-config)
  - TLS termination
  - routing per generated dynamic config
      │
      ▼
Auth Middleware (@oshun/auth / @oshun/identity)
  - JWT signature validation
  - Token expiry check
  - Role and scope extraction
  - mTLS peer verification for service-to-service calls (@oshun/identity)
      │
      ▼
Rate Limit Middleware (@oshun/rate-limit)
  - Per-user sliding window
  - Per-IP / per-API-key limits, abuse controls
      │
      ▼
Request Handler
  - Domain business logic
      │
      ▼
Audit Log (@oshun/security and @oshun/audit-platform)
  - Every privileged action recorded
  - Append-only, tamper-evident

Testing Architecture (@oshun/testing)#

Tests are structured in two tiers: unit tests using in-memory mocks for everything, and integration tests using real Docker-backed infrastructure. The split allows the unit test suite to run in milliseconds with no external dependencies while still exercising real database and cache behavior in the integration suite.

Unit tests use in-memory mocks for all infrastructure:

• @oshun/testing provides createMockDatabaseClient(), createMockRedisClient(), createMockHttpClient(), createMockLogger(), createMockEventEmitter(), and createMockTimers().
• Event-driven logic is asserted via MockEventEmitter, which records emitted events for assertion. The @oshun/event-bus package also ships in-memory- friendly behaviour, and @oshun/queue ships MemoryQueue / MemoryWorker.
• Fixtures come from createFixtureFactory and the user/content factories, producing realistic test data with configurable overrides.
• createVitestConfig / createServiceVitestConfig / createIntegrationVitestConfig produce consistent Vitest configs with enforced coverage thresholds.

Integration tests use real infrastructure:

• docker/docker-compose.dev.yml starts PostgreSQL, Redis, MinIO, and Mailpit (Kafka and other engines are optional Compose profiles).
• @oshun/testing containers (PostgresTestContainer, RedisTestContainer, ContainerManager) manage Docker container lifecycle for tests that need real external services, starting and stopping containers around the test suite.

Planned verification strategy (Phase 13, Phase 18.12) — beyond the two implemented tiers, the roadmap assigns Shared the platform-wide verification envelope:

• End-to-end testing — Playwright suites for the web shells and Detox for the React Native mobile apps, run against production builds.
• Consumer-driven contract testing — Pact (or equivalent) contracts between domain APIs and their consumers, verified in CI on both sides, so a provider cannot break a consumer without a failing contract build.
• Performance and load testing — load baselines per public API with regression gates, plus soak tests for long-running workers.
• Security testing — dependency and container scanning, SAST (CodeQL), and periodic dynamic scans of exposed surfaces.
• Boundary validation — automated Nx module-boundary checks (@nx/enforce-module-boundaries) and dependency-graph audits, enforcing the tag taxonomy described under Workspace Foundation below.
• Coverage expansion — a coordinated strategy to raise coverage across domains with thresholds enforced by the shared Vitest config factories.

Workspace Foundation (Phases 0–2)#

The Shared domain owns the monorepo's foundational tooling — the layer every other domain builds on but must never re-implement:

• Package management — a single pnpm workspace (pnpm-workspace.yaml) with a central dependency catalog: packages reference catalog: versions so the workspace pins each third-party dependency exactly once. .npmrc uses the hoisted node-linker, which the workspace's phantom-dependency profile depends on. Turbo was removed in favor of Nx as the single task runner.
• Nx workspace — nx.json defines the task runner, caching, and affected configuration. Projects carry a tag taxonomy (scope:<domain>, type:app|lib|e2e, layer:*) enforced by @nx/enforce-module-boundaries depConstraints in the root ESLint config: domain code may depend on Shared and Contracts, never on another domain's internals.
• TypeScript configuration — tsconfig.base.json holds the compiler baseline and the @oshun/* / @<domain>/* path aliases that make library imports resolve from source across the workspace.
• Code quality gates — root ESLint and Prettier configs, plus husky-driven git hooks (staged-file typecheck/lint, commitlint conventional-commit enforcement, stub-indicator scan) run before every commit.
• Architecture decision records (Phase 0) — the monorepo-level decisions (git strategy, package-manager choice, single-provider identity, event-bus strategy, Zod+OpenAPI+Proto contract approach, versioning/release policy) and the consolidated data-ownership matrix: each domain owns its own database (oshun_dev, yemaya, lilith, isis, sophia, hathor, bellona, …) and no domain reads another domain's tables directly — cross-domain data moves over events and APIs only.

Infrastructure and Delivery Topology (Phases 12, 18, 19)#

Shared also owns the deployment and delivery trees (docker/*, deploy/*, infra/*) and the platform CI/CD posture:

• CI/CD (Phase 12.1) — consolidated GitHub Actions workflows (ci.yml, deploy.yml) built on Nx-affected pipelines so pull requests build and test only what they touch, with CodeQL security analysis and accessibility/ performance gates on web surfaces. A shared setup composite action and the Nx cache keep per-PR cost bounded.
• Container strategy (Phase 12.2) — base Dockerfiles for services, web apps, and workers that domain images extend, so patching a base image patches the fleet.
• Orchestration and IaC (Phases 12.3–12.6, 19) — the compute target is AWS ECS on Fargate (clusters, task definitions, and services for domain APIs and workers) with Terraform as the IaC source of truth under infrastructure/terraform; Kubernetes manifests and Helm charts remain for the services still deployed to K8s (infra/k8s/*, deploy/helm/*). GPU workloads dispatch to RunPod Serverless via @oshun/runpod-client and @oshun/gpu-dispatcher rather than resident GPU nodes.
• Observability stack (Phase 12.7) — beyond the in-process libraries (@oshun/metrics, @oshun/tracing, @oshun/logging), Shared owns the hosted stack: Prometheus scraping, Grafana dashboards per domain, Jaeger tracing, and centralized log aggregation (Loki/ELK).
• Secrets management (Phase 12.8) — a central secrets store (Vault or the cloud-native equivalent) feeding runtime configuration; secrets never live in the repository or images.
• Production readiness and operations (Phases 18, 19.10–19.11) — the critical-blocker remediation posture for shared libraries (connection pooling, retry/backoff, graceful shutdown, health probes on every service), deployment runbooks per service, and the decommissioning procedure for superseded infrastructure so replaced stacks are torn down rather than left running.

The Neith-stack counterparts of this layer (@neith/cloud, @neith/observability, @neith/security, @neith/testing, @neith/qa, @neith/docs, @neith/training, @neith/release — Phases 49–52) are owned by Neith and documented in DOMAINS/neith/features.md.