# Nous Domain — Architecture > **Nous** - LLM Inference and Reasoning Engine This document describes the technical architecture of the Nous domain: library organization, module topology, design patterns, data flows, and integration architecture. Nous is the AI infrastructure backbone of the Oshun monorepo. It answers the question: _how do you run large language models, vision models, and audio models on real hardware, at production throughput, without paying per-token cloud API costs or sending user data off-premises?_ The answer is a carefully layered set of TypeScript libraries that abstract hardware specifics away from every caller. An Iris assistant feature, a Sophia research synthesis pipeline, or a Maya NPC dialogue system all reach Nous through the same interfaces — and Nous figures out whether to run on a GPU, an Apple Neural Engine, or a plain CPU thread pool. Because Nous ships no services or applications of its own, there is no Nous deployment to operate separately. Domains import `@nous/*` packages directly and the inference work happens wherever those domains run. --- ## 1. Domain Overview Nous is a **library-only domain** — it ships no applications, no standalone services, and no APIs exposed over the network. It is purely a set of TypeScript libraries consumed by other domains and applications in the Oshun monorepo. `libs/nous/` holds twenty package directories; nineteen contain implemented source and one (`@nous/platform`) is a config-only scaffold (no `src/`). **What Nous provides:** - A hardware-abstracted ONNX inference runtime, session manager, and model registry (`@nous/core`) - High-throughput inference primitives — batching, KV cache, speculative decoding, multi-GPU parallelism, multi-LoRA serving (`@nous/core`) - An embedding pipeline with multi-modal and multi-provider support, plus similarity search and reranking (`@nous/core`) - Provider-registry-based LLM completion APIs, prompt templates, an agent runtime, and a document-ingestion/RAG surface (`@nous/llm`) - Image and video computer vision — detection, segmentation, OCR, scene understanding, diffusion generation (`@nous/vision`) - Speech, music, and audio analysis (`@nous/audio`) - A model-training surface — distributed training, fine-tuning, preference alignment, diffusion training (`@nous/training`) - Safety classifiers, output enforcement, interpretability, formal verification, and V2 Model Card hosting (`@nous/safety`) - Controllable generative-media packages — image/video control, video object removal, video-to-audio, advanced speech, relighting, portrait animation, joint A/V generation - Cooperative-intelligence primitives for the Concordia domain — preference inference, agreement search, bargaining sessions, sealed memory **What Nous does not provide:** - Network APIs (no REST, WebSocket, or gRPC server) - Persistent storage (consuming applications manage storage) - Authentication or authorization - User-facing applications --- ## 2. Design Principles These six principles shape every architectural decision in Nous. They explain why the domain is structured the way it is, and what invariants consuming domains can depend on. 1. **Hardware Abstraction** — All inference runs behind a unified provider interface. The calling application never writes CUDA-specific or Metal-specific code. 2. **Optional Runtime** — `onnxruntime-node` is an optional dependency. Nous can be imported without installing the GPU runtime (useful in browser/edge environments using the WebGPU provider). 3. **Zero Cloud Lock-In** — Every capability in Nous runs locally. No API keys, no external services, no internet required. 4. **Progressive Enhancement** — Applications start with CPU inference and add GPU acceleration, batching, and parallelism incrementally as hardware allows. 5. **Composable Primitives** — Each module (`dynamic-batching`, `kv-cache`, `flash-attention`, etc.) is independently importable. Applications compose the primitives they need, not a monolithic inference stack. 6. **ESM-First** — All three libraries are built as ESM modules for compatibility with modern runtimes (Node.js 22+, browsers via bundlers, Deno, Bun). --- ## 3. Technology Stack The table below summarises the tooling choices for every `@nous/*` package. Nothing here is domain-specific configuration — these defaults apply uniformly across all nineteen implemented packages. | Layer | Technology | | ----------------- | ------------------------------------------------------------------- | | Language | TypeScript 5+ (strict mode) | | Module format | ESM (`"type": "module"` in every package) | | Build | `@nx/esbuild:esbuild` (`format: ["esm"]`, `bundle: false`) | | Package shape | Source-only — `main`/`types` point at `./src/index.ts` | | Testing | Vitest (`@nx/vite:test`) | | Type checking | Explicit `typecheck` target via `tsc --noEmit -p tsconfig.lib.json` | | Schema validation | `zod` (Concordia packages and some V2 contracts) | | Inference backend | ONNX Runtime (optional dependency: `onnxruntime-node ^1.20.0`) | ### Source-Only Packages and esbuild Every `libs/nous/*/package.json` sets `"main"` and `"types"` to `./src/index.ts` — packages are consumed directly as TypeScript source through the workspace. The Nx `build` target uses `@nx/esbuild:esbuild` with `format: ["esm"]` and `bundle: false`, producing unbundled ESM module files so consumers can tree-shake. Each `project.json` also defines `lint`, `test`, and `typecheck` targets. --- ## 4. Library Structure `libs/nous/` contains twenty package directories. Every implemented package keeps its modules **flat** under `src/` (no nested subdirectories) — each module is a single `.ts` file with a sibling `*.spec.ts`, and `src/index.ts` is a barrel that re-exports every module. ``` libs/nous/ ├── core/ # @nous/core — 84 source modules (flat src/) ├── llm/ # @nous/llm — 78 source modules ├── vision/ # @nous/vision — 103 source modules ├── audio/ # @nous/audio — 81 source modules ├── training/ # @nous/training — 137 source modules ├── safety/ # @nous/safety — 64 source modules ├── image-control/ # @nous/image-control — 50 source modules ├── video-control/ # @nous/video-control — 43 source modules ├── video-removal/ # @nous/video-removal — 48 source modules ├── video-to-audio/ # @nous/video-to-audio — 21 source modules ├── advanced-speech/ # @nous/advanced-speech — 31 source modules ├── generative-relighting/ # @nous/generative-relighting — 23 source modules ├── portrait-animation/ # @nous/portrait-animation — 23 source modules ├── joint-av-generation/ # @nous/joint-av-generation — 34 source modules ├── agreement-search/ # @nous/agreement-search — 22 source modules ├── preference-inference/ # @nous/preference-inference — 24 source modules ├── cooperative-bargaining/ # @nous/cooperative-bargaining — 2 source modules ├── concordia-sealed-memory/ # @nous/concordia-sealed-memory — 5 source modules └── platform/ # @nous/platform — config-only scaffold (no src/) ``` ### `@nous/core` Module Inventory `@nous/core` is the foundational package. Its 84 modules fall into five functional groups, all residing flat in `core/src/`: **Runtime and session management** — the entry types and runtime (`types.ts`, `runtime-loader.ts`, `session-manager.ts`, `inference-engine.ts`, `model-loader.ts`). **Execution providers** — one module per hardware backend (`cpu-`, `cuda-`, `rocm-`, `metal-`, `webgpu-`, `vulkan-`, `dml-`, `npu-execution-provider.ts`) plus the shared `execution-providers.ts` registry. **Inference primitives** — throughput and memory optimization techniques: `dynamic-batching`, `continuous-batching`, `llm-serving`, `speculative-decoding`, `kv-cache`, `paged-attention`, `flash-attention`, `tensor-parallelism`, `pipeline-parallelism`, `model-sharding`, `moe-parallelism`, `prefix-caching`, `radix-attention`, `prompt-caching`, `multi-lora-serving`, `adapter-switching`, `context-extension`. **Generation control** — sampling and stopping: `streaming-generation`, `stop-sequences`, `logit-processors`, `temperature-sampling`, `top-k-top-p-sampling`, `beam-search`. **Graph optimization** — compile-time improvements: `graph-optimization`, `memory-planning`, `operator-fusion`, `quantization`. **Model management** — the full lifecycle from discovery to eviction: `model-registry`, `model-versioning`, `model-metadata`, `model-discovery`, `model-download`, `checksum-verification`, `model-conversion`, `format-detection`, `model-caching`, `model-eviction`, `model-preloading`, `warm-model-pool`, `model-benchmarking`, `model-profiling`, `model-optimization-pipeline`. **Format readers** — direct file format support: `safetensors`, `gguf`, `ggml`, `pytorch-loading`, `tensorflow-loading`, `jax-loading`, `tensor`. **Embedding and search pipeline** — vector generation and retrieval: `embedding-generation`, `text-embeddings`, `image-embeddings`, `audio-embeddings`, `multi-modal-embeddings`, `embedding-normalization`, `dimensionality-reduction`, `batch-embedding`, `streaming-embedding`, `embedding-caching`, `embedding-compression`, `binary-embeddings`, `matryoshka-embeddings`, `similarity-search`, `approximate-nearest-neighbor`, `exact-nearest-neighbor`, `distance-metrics`, `hybrid-search`, `reranking`, `cross-encoder-reranking`. See `DOMAINS/nous/specifications.md` §10–§19 for the `@nous/llm`, `@nous/vision`, `@nous/audio`, `@nous/training`, `@nous/safety`, generative-media, and Concordia module groups. --- ## 5. Module Topology The diagram below shows how the packages depend on each other. `@nous/core` is the foundation: it provides the inference engine, providers, and embedding pipeline that all higher-level packages build on. `@nous/llm`, `@nous/vision`, `@nous/audio`, and `@nous/training` each sit one layer above core, composing its primitives for their respective domains. `@nous/safety` is a cross-cutting layer that can consume any package. The generative-media and Concordia packages sit at the edges — they wrap specific capabilities without creating circular dependencies. ``` @nous/core (84 source modules) ┌──────────────────┼──────────────────┐ | | | Execution Inference Model Providers Pipeline Management | | | cpu, cuda, dynamic-batching, model-registry, rocm, metal, continuous-batching, model-versioning, webgpu, vulkan, speculative-decoding, model-download, dml, npu llm-serving, checksum-verification, (8 modules + kv-cache, model-conversion, 12 known paged-attention, format-detection, provider flash-attention, safetensors, gguf, names) | ggml, pytorch, … Graph | Optimization Parallelism / Caching / Adapters | | memory-planning, tensor-parallelism, operator-fusion, pipeline-parallelism, quantization model-sharding, moe-parallelism, prefix-caching, radix-attention, multi-lora-serving, … | ┌──────────────┼───────────────┬─────────────┐ | | | | @nous/llm @nous/vision @nous/audio @nous/training (78 modules) (103 modules) (81 modules) (137 modules) | | | | completion, detection, ASR, TTS, distributed agents, segmentation, music, training, prompts, OCR, scene, audio fine-tuning, reasoning, diffusion, analysis preference RAG, NLP video alignment @nous/safety (64 modules) classifiers, output enforcement, interpretability, formal verification, V2 Model Card hosting Generative-media packages Concordia packages image-control, video-control, preference-inference, video-removal, video-to-audio, agreement-search, advanced-speech, cooperative-bargaining, generative-relighting, concordia-sealed-memory portrait-animation, joint-av-generation ``` --- ## 6. nous-core Architecture `@nous/core` implements two interlocking concerns: _how a session is acquired and reused_ (the session manager), and _how hardware is selected_ (provider resolution). Understanding these two flows is the key to understanding how all higher-level Nous packages work. ### Inference Session Lifecycle The `OnnxSessionManager` maintains a reference-counted cache of live ONNX sessions. Each entry is keyed by a fingerprint derived from the model source, the resolved provider list, and the session options. When two parts of the application ask for the same model and configuration, they share a single underlying session rather than loading the model twice. ``` Application │ │ sessionManager.acquire({ model, preferredProviders, ... }) ▼ ┌──────────────────────────────────────────┐ │ OnnxSessionManager │ │ │ │ Reference-counted session cache │ │ (keyed by model fingerprint + providers)│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │ │ Session │ │ Session │ │ ... │ │ │ │ refs: 2 │ │ refs: 0 │ │ │ │ │ └─────────┘ └─────────┘ └─────────┘ │ │ │ │ On acquire(): │ │ 1. resolveExecutionProviders() │ │ 2. Build cache key; hit → refcount++ │ │ 3. Miss → create (dedup in-flight │ │ creations); evict LRU zero-ref │ │ entry if cache is full │ │ 4. CPU fallback if primary fails │ └──────────────────────────────────────────┘ │ │ ManagedOnnxSession { session, key, release() } ▼ Application runs inference, then release() │ │ (decrements refcount; zero-ref entries │ stay cached until LRU eviction) ▼ closeSession(key) releases a zero-ref entry ``` ### Provider Resolution Chain Before a session can be created, Nous must decide which hardware backend to use. The resolver walks the caller's preferred provider list in order, testing each against the detected hardware environment, and always appends a `cpu` fallback so the chain can never be exhausted without producing a working session. ``` Preferred providers: ['cuda', 'metal', 'cpu'] | v Is provider available on this hardware? | Yes ──────────┼──────────── No | | Use this provider Try next in list | | OnnxRuntime (fallback loop) .create() | | Exhausted all → Session throw error ``` --- ## 7. nous-llm Architecture `@nous/llm` adds two major abstractions on top of `@nous/core`: a **provider-registry pattern** for LLM completions (so the calling code never hard-codes a backend), and an **agent runtime** for orchestrating multi-step reasoning tasks. ### LLM Completion Flow The completion layer is provider-registry based: callers register `ChatCompletionProvider` / `CompletionProvider` implementations on a `ChatCompletionAPI` / `CompletionAPI` instance, then dispatch a request. An ONNX-backed provider can compose `@nous/core` primitives internally — the caller is shielded from that complexity. ``` caller: chatCompletionAPI.complete({ messages, providerId?, ... }) │ v ┌──────────────────────┐ │ ChatCompletionAPI │ │ │ │ 1. Select provider │ │ (providerId or │ │ default) │ │ 2. Try provider; │ │ fall back to │ │ fallbackProviderIds │ │ 3. Record latency / │ │ usage stats │ └──────────┬───────────┘ │ provider.complete(request) v ┌──────────────────────────────────────┐ │ ChatCompletionProvider implementation │ │ │ │ A provider is supplied by the │ │ consumer. An ONNX-backed provider │ │ may compose @nous/core primitives: │ │ OnnxSessionManager.acquire → │ │ DynamicBatchScheduler → session.run →│ │ logit processors / stop sequences / │ │ token sampling. │ └──────────┬───────────────────────────┘ │ v ChatCompletionResponse { choices, usage, latencyMs } ``` ### Agent Task Execution Flow The agent runtime follows a plan-then-execute model. The planning phase (optional, enabled per agent) translates a high-level goal into an ordered list of concrete steps with a confidence score before any tool is called. The execution loop then steps through those tasks, using memory recall and tool selection to resolve each one. ``` submit(agentId, task) │ v ┌─────────────┐ │ Task Queue │ (priority-ordered) └──────┬──────┘ │ v ┌─────────────────────────┐ │ Planning Module │ │ (if enabled) │ │ - summarize goal │ │ - decompose to steps │ │ - estimate confidence │ └──────────┬──────────────┘ │ v ┌──────────────────────────┐ │ Execution Loop │ │ │ │ for each step: │ │ 1. Recall memory │ │ 2. Generate action │ │ 3. Select tool │ │ 4. Execute tool │ │ 5. Store result │ │ 6. Update state │ │ │ │ On error: │ │ - retry with backtrack│ │ - emit failure event │ └──────────┬───────────────┘ │ v TaskResult { state, output, steps } ``` --- ## 8. nous-vision Architecture `@nous/vision` builds on the same `@nous/core` session infrastructure as the LLM stack, but adds vision-specific pre- and post-processing steps: image buffers are resized, normalised, and tensor-encoded before inference, and raw model outputs are decoded through non-maximum suppression, mask decoding, or text parsers depending on the task. ### Standard Vision Inference Pipeline ``` Input Image (Buffer | ImageData) │ v ┌──────────────────┐ │ Image Preprocess│ │ - resize │ │ - normalize │ │ - tensor encode │ └────────┬─────────┘ │ v ┌──────────────────┐ │ Model Session │ (@nous/core session-manager) │ Provider: │ │ cuda/metal/cpu │ └────────┬─────────┘ │ v ┌──────────────────┐ │ ONNX Inference │ │ (run) │ └────────┬─────────┘ │ v ┌──────────────────┐ │ Output Decode │ │ - postprocess │ │ - NMS (detect.) │ │ - decode masks │ │ - parse text │ └────────┬─────────┘ │ v Typed result object ``` ### Diffusion Pipeline Image generation with Stable Diffusion or Flux requires a multi-stage pipeline that interleaves a text encoder, a noise scheduler, many denoising iterations through a UNet, and a final VAE decode. Nous models each stage as a separate ONNX session so the hardware provider can be selected per stage. ``` text prompt │ v ┌─────────────────────────────────┐ │ Diffusion Pipeline │ │ │ │ 1. Text Encoder (CLIP) │ │ prompt → conditioning │ │ │ │ 2. Scheduler Init │ │ noise schedule, timesteps │ │ │ │ 3. Denoising Loop (N steps) │ │ ┌────────────────────────┐ │ │ │ UNet forward pass │ │ │ │ (ONNX session) │ │ │ │ predict noise │ │ │ │ scheduler step │ │ │ └────────────────────────┘ │ │ │ │ 4. VAE Decode │ │ latent → pixel image │ │ │ └─────────────────────────────────┘ │ v Buffer (PNG/JPEG) ``` --- ## 9. Key Design Patterns The following patterns recur across all `@nous/*` packages. A new engineer implementing a feature or a new provider should follow them to stay consistent with the existing codebase. ### Provider Interface Pattern Provider names are an open union (`OnnxExecutionProviderName`). `resolveExecutionProviders` ranks a requested list against detected hardware and always appends a `cpu` fallback; `OnnxSessionManager.acquire` takes a `NousOnnxSessionConfig` and never needs backend-specific calling code. The calling site looks like this regardless of whether the runtime machine has a GPU: ```typescript // Calling code — no CUDA-specific logic const managed = await sessionManager.acquire({ model: modelPathOrBytes, preferredProviders: ['cuda', 'metal', 'cpu'], // priority list allowProviderFallback: true, }); const outputs = await managed.session.run(feeds); await managed.release(); ``` ### Optional Dependency Pattern `runtime-loader.ts` loads `onnxruntime-node` lazily and throws a typed error when it is absent, so `@nous/core` compiles without the native runtime. This is what enables `@nous/core` to be imported in browser or serverless targets that cannot install a native binary — the error is only thrown at the moment inference is actually attempted: ```typescript // libs/nous/core/src/runtime-loader.ts (shape) export async function loadOnnxRuntime( options: OnnxRuntimeLoaderOptions = {} ): Promise { const runtimePackage = options.runtimePackage ?? 'onnxruntime-node'; try { const candidate = await (options.load ?? defaultRuntimeLoader)( runtimePackage ); return resolveOnnxRuntimeModule(candidate); // validates Tensor + InferenceSession } catch (error) { if (isMissingModuleError(error, runtimePackage)) { throw new OnnxRuntimeNotAvailableError(runtimePackage, error); } throw error; } } ``` ### Composable Pipeline Pattern Rather than a single monolithic inference class, consumers assemble a pipeline from the specific primitives they need. A high-throughput serving setup might combine a dynamic batch scheduler, a KV cache manager, and tensor parallelism; a lightweight edge deployment might use only the session manager: ```typescript // Consumer assembles their own pipeline const batcher = new DynamicBatchScheduler(engine, { maxBatchSize: 16, maxQueueDelayMs: 8, }); const kvCache = new KvCacheManager({ maxTotalBytesReserved: 512 * 1024 * 1024, }); const cache = kvCache.allocate({ modelId: 'llama-3-8b', layerCount: 32, numHeads: 32, headSize: 128, }); ``` ### Agent Event Sourcing Pattern The `AgentArchitecture` runtime (`libs/nous/llm/src/agent-architecture.ts`) records typed `AgentRuntimeEvent`s as agents and tasks transition through their lifecycle states. The event log is queryable for replay, debugging, and monitoring. Because every state change is recorded as an event (not mutated in place), the full history of an agent run is always recoverable: ```typescript // Query the runtime event log const events = agentRuntime.getEvents(100); // last 100 events const stats = agentRuntime.getStats(); // AgentArchitectureStats // Agent lifecycle: stopped | idle | running | paused | error // Task lifecycle: queued | running | completed | failed | cancelled | timeout ``` --- ## 10. Data Flow Diagrams ### Embedding Generation Flow Embeddings are the bridge between raw text or media and vector search. The `EmbeddingGenerator` abstracts multiple providers (a local sentence-transformer, an OpenAI embedding endpoint, a Cohere model) behind a single interface, routing by modality and falling back across providers on failure. ``` Application │ generate({ modelId, input: { content, modality: 'text' } }) ▼ ┌──────────────────────────────┐ │ EmbeddingGenerator │ │ │ │ 1. Select provider │ │ (by id / priority + │ │ supported modality) │ │ │ │ 2. Call provider.generate() │ │ fall back on failure │ │ │ │ 3. Truncate to │ │ outputDimensions │ │ 4. Record provider stats │ └──────────────┬───────────────┘ │ v EmbeddingGenerationResult { vector, dimensions, ... } ``` ### RAG Pipeline Flow RAG (Retrieval-Augmented Generation) grounds an LLM response in specific documents by retrieving relevant passages and injecting them into the context window. The pipeline runs through five stages: embedding the query, searching the vector index, reranking candidates with a cross-encoder, building an injection context, and finally calling the LLM with that context attached. ``` Query: "What is the refund policy?" │ v ┌────────────────────┐ │ Query Embedding │ (EmbeddingGenerator) └────────┬───────────┘ │ vector v ┌────────────────────┐ │ Vector Index │ (hybridSearch) │ search(topK=5) │ └────────┬───────────┘ │ SearchResult[] v ┌────────────────────┐ │ Reranker │ (cross-encoder) └────────┬───────────┘ │ reranked SearchResult[] v ┌────────────────────┐ │ Context Builder │ │ chunk[0..k] → │ │ injected context │ └────────┬───────────┘ │ v ┌────────────────────┐ │ LLM Completion │ │ (with RAG context)│ └────────┬───────────┘ │ v Grounded response ``` --- ## 11. Deployment Characteristics ### Runtime Environments Nous supports a wide range of deployment targets. The runtime environment determines which execution provider is selected; no code changes are needed between targets — only the hardware and the installed ONNX Runtime variant differ. | Environment | Provider Used | Notes | | --------------------- | ------------- | ---------------------------------------------- | | NVIDIA GPU server | `cuda` | Requires CUDA 12.x and `onnxruntime-node` | | AMD GPU server | `rocm` | Requires ROCm 5.6+ and `onnxruntime-node` | | macOS (Apple Silicon) | `metal` | Metal acceleration via `onnxruntime-node` | | Windows desktop | `dml` | DirectML via DirectX 12 | | Browser (WebGPU) | `webgpu` | Uses `onnxruntime-web`, not `onnxruntime-node` | | Any CPU | `cpu` | Universal fallback, no GPU drivers needed | | Serverless/edge | `cpu` | Use quantized models (INT8/INT4) for size | ### WASM Considerations When bundling `@nous/core` for browser use (e.g., in a PWA), use `webgpu` or `cpu` providers. The `onnxruntime-web` package includes WASM binaries. The `metal`, `cuda`, and `rocm` providers are Node.js-only. ### Memory Sizing Guidelines Model memory requirements grow with parameter count and numerical precision. The table below gives approximate VRAM floors for common configurations — the actual requirement depends on the KV cache allocation on top of these weights. | Model Size | Min GPU VRAM | Recommended Config | | ---------- | ------------ | --------------------------------------------------- | | 7B (FP16) | 14 GB | Single GPU, no parallelism needed | | 7B (INT4) | 4 GB | Single GPU with quantization | | 13B (INT8) | 13 GB | Single GPU or 2-GPU tensor parallelism | | 70B (INT4) | 40 GB | Multi-GPU tensor parallelism (2× 24 GB or 4× 12 GB) | ### Package Entry Points `@nous/core` (and every other `@nous/*` package) is a source-only workspace package — `package.json` sets `main` and `types` to `./src/index.ts` and declares `"type": "module"`. There is no `exports` map and no published `dist/`; the Nx `build` target emits unbundled ESM to `libs/nous//dist` only when explicitly run. Consumers resolve `@nous/*` through the `tsconfig.base.json` path mapping `"@nous/*": ["libs/nous/*/src/index.ts"]`. The Concordia packages (`@nous/agreement-search`, `@nous/cooperative-bargaining`, `@nous/preference-inference`, `@nous/concordia-sealed-memory`) additionally declare an explicit `exports` field pointing at `./src/index.ts`. --- ## 12. Domain Dependencies ### External Dependencies The only external runtime dependency is `onnxruntime-node`, and even that is optional. Nous deliberately avoids third-party service dependencies at the library level — consuming applications wire in any external providers (such as an OpenAI embedding endpoint) through the provider registration interfaces. | Dependency | Type | Notes | | ------------------ | -------- | ---------------------------------------------------------------------------- | | `onnxruntime-node` | Optional | Core inference backend. Not required in browser environments using `webgpu`. | ### Domains That Depend on Nous Each consuming domain uses Nous for a different purpose, which is why the boundary is clean: Nous knows nothing about Iris assistant UX, Sophia research workflows, or Maya world generation. It exposes primitives; the consuming domain decides what to do with them. | Domain | Libraries Used | How Nous Is Consumed | | ---------- | ------------------------- | -------------------------------------------------------------------------------------- | | **Iris** | `@nous/core`, `@nous/llm` | Local LLM serving for privacy mode; embedding generation for knowledge base and memory | | **Sophia** | `@nous/llm` | LLM reasoning for research synthesis | | **Psyche** | `@nous/vision` | Facial emotion recognition and perception models | | **Maya** | `@nous/llm` | NPC dialogue generation and world content AI | ### Cross-Domain Dependencies Within Nous Most `@nous/*` packages are self-contained — they bring only `devDependencies` and depend on nothing outside the package at runtime. The two exceptions exist for concrete architectural reasons: - `@nous/safety` depends on `@kalika/core` and `@kalika/formal-verification` because formal safety verification requires the Kalika theorem-prover runtime. The safety invariant proofs are not a Nous concern to implement from scratch; they delegate to Kalika's proven core. - The three Concordia packages (`@nous/preference-inference`, `@nous/agreement-search`, `@nous/cooperative-bargaining`) depend on `@concordia/contracts` because they implement the _computational_ half of a bargaining session — the preference models and search kernels — while Concordia owns the type contracts that describe parties, cases, and outcomes. Nous cannot define those contracts without creating a circular dependency, so it consumes them from `@concordia/contracts` instead. | `@nous` package | Workspace dependency | | ------------------------------ | --------------------------------------------- | | `@nous/safety` | `@kalika/core`, `@kalika/formal-verification` | | `@nous/preference-inference` | `@concordia/contracts` | | `@nous/agreement-search` | `@concordia/contracts` | | `@nous/cooperative-bargaining` | `@concordia/contracts` | --- ## 13. Concordia Cooperative-Intelligence Packages Four `@nous/*` packages provide reusable cooperative-intelligence primitives consumed by the Concordia domain (`DOMAINS/concordia`). These packages sit at the intersection of Nous and Concordia: Nous owns the _algorithms_ (utility model fitting, multi-objective optimisation, privacy primitives), while Concordia owns the _policy_ (legal boundaries, settlement authority, user experience). The boundary exists precisely so that the algorithms can be tested, versioned, and reused independently of any specific dispute or governance domain. Data that crosses the boundary: `@concordia/contracts` types for parties, cases, and outcomes flow _into_ Nous as inputs to the preference and search kernels. Calibrated utility estimates and Pareto-frontier candidate agreements flow _out_ to Concordia as results. No settlement decisions, legal boundaries, or product UX logic cross into Nous. | Package | Architectural role | | ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `@nous/preference-inference` | Pairwise preference elicitation and five calibrated utility-model estimators (Bradley-Terry, Thurstone-Mosteller, Plackett-Luce, Gaussian-process, neural ranker); calibration, fairness metrics, BATNA analysis | | `@nous/agreement-search` | Search-kernel registry plus seven search kernels (Nash genetic, NSGA-II, MAP-Elites, MCTS/LATS, CP-SAT, Bayesian optimization, PSRO) and a coalition-stability search; candidate generation and clause mutators | | `@nous/cooperative-bargaining` | Session-scoped bargaining substrate bridging `@concordia/contracts` and the search kernels (phase machine, stakeholder registry, budget) | | `@nous/concordia-sealed-memory` | Privacy primitives — envelope-encrypted sealed-memory store, zero-retention routing/attestation, confidential-compute scoring | These packages provide reusable preference/search/bargaining/privacy primitives only. Domain policy, legal boundaries, settlement authority, and user experience are owned by the Concordia domain and its peer domains (Themis, Kuanyin, Maat, Aje, and others). --- ## Document Provenance This architecture document was written against the source under `libs/nous/`. Package inventory and module counts come from `ls libs/nous/` and the `src/index.ts` barrels; the dependency table is read from each `libs/nous/*/package.json`; build targets from each `project.json`. It describes the AI-infrastructure module topology, the inference/LLM/vision/audio architecture, execution providers, session/model/inference/embedding/search/ agent/document pipelines, deployment characteristics, and the cross-domain and Concordia dependency structure.