Nous Domain — Technical Specifications

Technical specifications for the Nous AI model infrastructure domain: library inventory, type contracts, class APIs, configuration objects, and integration points. Every type, field, and identifier in this document is traceable to source under libs/nous/.

This document is the authoritative type-level reference for the Nous domain. It complements the higher-level features.md (what Nous does) and architecture.md (how it is structured) by providing the exact TypeScript interfaces, enums, class signatures, and constant values that are exported from each @nous/* package. Engineers implementing a new Nous feature, writing a consumer in another domain, or reviewing a change for correctness should use this document alongside the actual source files under libs/nous/.

The document is organised in dependency order: §1–§2 cover the package inventory and build system; §3–§9 cover @nous/core (the inference foundation); §10–§13 cover @nous/llm; §14–§17 cover the specialised packages (vision, audio, training, safety); §18–§19 cover generative media and Concordia primitives; and §20–§22 cover configuration, integration points, and acceptance criteria.

1. Library Inventory#

Nous is a library-only domain. It ships no apps/nous and no services/nous — the entire domain is a set of TypeScript workspace packages under libs/nous/. Twenty package directories exist; nineteen contain implemented source, and one (platform) is a config-only scaffold. The module counts below are non-spec .ts files in each src/; every implemented module in the six core packages has a sibling *.spec.ts.

* "Source modules" counts non-spec .ts files under src/ (each module has a sibling *.spec.ts in core/llm/vision/training/safety/audio/the media packages; the four Concordia packages currently have no .spec.ts files). @nous/platform has package.json, project.json, three tsconfig*.json, and vitest.config.ts, but no src/ directory.

tsconfig.base.json declares a single wildcard path mapping: "@nous/*": ["libs/nous/*/src/index.ts"].

2. Build System and Package Shape#

Source-Only Workspace Packages#

Every libs/nous/*/package.json sets "main" and "types" to ./src/index.ts — packages are consumed directly as TypeScript source through the workspace; there is no published dist/. All packages declare "type": "module" and "private": true, version 0.1.0.

Package dependencies (package.json):

• @nous/core — optionalDependencies: { "onnxruntime-node": "^1.20.0" }.
• @nous/safety — dependencies: { "@kalika/core", "@kalika/formal-verification" } (both workspace:*).
• @nous/video-removal — dependencies: { "bullmq": "catalog:" }.
• @nous/agreement-search, @nous/cooperative-bargaining, @nous/preference-inference — each depends on @concordia/contracts (workspace:*) and zod (catalog:).
• @nous/concordia-sealed-memory — depends on zod (catalog:).
• All others — only devDependencies (@types/node, typescript, vitest, all catalog:).

Nx Build Targets#

Each project.json (e.g. libs/nous/core/project.json) defines four targets that apply uniformly across all implemented packages. The typecheck target runs tsc --noEmit in isolation, which is important because Nx's duplicate-project detection can fail in worktree environments — in that case run the typecheck directly with npx tsc --noEmit from the package directory.

Project tags are ["scope:nous", "layer:infra", "type:lib"]. Project names are hyphenated (nous-core, nous-llm, …).

3. Core Type System (@nous/core)#

Source: libs/nous/core/src/types.ts. The barrel libs/nous/core/src/index.ts re-exports 83 modules plus types.ts. These base types are the currency of the entire @nous/core API — every execution provider, session, inference request, and batch result is built from them. Understanding OnnxExecutionProviderName (an open union that allows unknown providers), OnnxTensorLike (the lightweight tensor shape the engine accepts), and ManagedOnnxSession (the reference- counted session handle) is the prerequisite for reading any other section.

ONNX Provider and Tensor Types#

type OnnxExecutionProviderName =
  | 'cpu'
  | 'cuda'
  | 'rocm'
  | 'dml'
  | 'coreml'
  | 'metal'
  | 'webgpu'
  | 'vulkan'
  | 'xnnpack'
  | 'nnapi'
  | 'qnn'
  | 'npu'
  | (string & {}); // open union — unknown providers permitted

type OnnxTensorType =
  | 'float32'
  | 'float64'
  | 'int8'
  | 'uint8'
  | 'int16'
  | 'uint16'
  | 'int32'
  | 'uint32'
  | 'int64'
  | 'uint64'
  | 'bool';

type OnnxTensorData =
  | Float32Array
  | Float64Array
  | Int8Array
  | Uint8Array
  | Int16Array
  | Uint16Array
  | Int32Array
  | Uint32Array
  | BigInt64Array
  | BigUint64Array;

Core Interfaces#

The following interfaces are the building blocks for every @nous/core operation. OnnxSessionOptions maps directly to ONNX Runtime session configuration; NousOnnxSessionConfig is the higher-level config passed to OnnxSessionManager.acquire; ManagedOnnxSession is the handle that callers receive and must release() when done.

interface OnnxExecutionProviderConfig {
  readonly name: OnnxExecutionProviderName;
  readonly options?: Readonly<Record<string, string | number | boolean>>;
}

interface OnnxTensorLike<TData extends OnnxTensorData = OnnxTensorData> {
  readonly type: string;
  readonly data: TData;
  readonly dims: readonly number[];
  readonly size?: number;
}

interface OnnxSessionOptions {
  readonly executionProviders?: readonly OnnxExecutionProviderConfig[];
  readonly graphOptimizationLevel?: 'disabled' | 'basic' | 'extended' | 'all';
  readonly executionMode?: 'sequential' | 'parallel';
  readonly enableCpuMemArena?: boolean;
  readonly enableMemPattern?: boolean;
  readonly interOpNumThreads?: number;
  readonly intraOpNumThreads?: number;
  readonly extra?: Readonly<Record<string, string>>;
  readonly logSeverityLevel?: number;
  readonly logVerbosityLevel?: number;
}

type OnnxModelSource = string | ArrayBuffer | Uint8Array;

interface OnnxInferenceSessionLike {
  readonly inputNames: readonly string[];
  readonly outputNames: readonly string[];
  run(
    feeds: Record<string, OnnxTensorLike>,
    fetches?: readonly string[]
  ): Promise<Record<string, OnnxTensorLike>>;
  release?: () => void | Promise<void>;
  dispose?: () => void | Promise<void>;
}

interface OnnxRuntimeModule {
  readonly Tensor: new (
    type: OnnxTensorType,
    data: OnnxTensorData,
    dims: readonly number[]
  ) => OnnxTensorLike;
  readonly InferenceSession: {
    create(
      source: OnnxModelSource,
      options?: OnnxSessionOptions | Record<string, unknown>
    ): Promise<OnnxInferenceSessionLike>;
  };
  readonly env?: Record<string, unknown>;
}

interface NousOnnxSessionConfig {
  readonly model: OnnxModelSource;
  readonly preferredProviders?: readonly OnnxExecutionProviderName[];
  readonly sessionOptions?: Omit<OnnxSessionOptions, 'executionProviders'>;
  readonly allowProviderFallback?: boolean;
}

interface ManagedOnnxSession {
  readonly key: string;
  readonly session: OnnxInferenceSessionLike;
  readonly providers: readonly OnnxExecutionProviderConfig[];
  release(): Promise<void>;
}

Inference Request/Result Types#

OnnxInferenceRequest packages a session configuration, feed tensors, and optional fetch names into a single value that OnnxInferenceEngine.run consumes. OnnxBatchInferenceRequest adds a list of items and a mode: independent runs them concurrently with optional error tolerance, while stacked concatenates all items into one batch-dimension call.

interface TensorInput {
  readonly type?: OnnxTensorType;
  readonly data: ArrayLike<number | bigint>;
  readonly dims: readonly number[];
}

interface OnnxInferenceRequest {
  readonly session: NousOnnxSessionConfig;
  readonly feeds: Readonly<Record<string, OnnxTensorLike | TensorInput>>;
  readonly fetches?: readonly string[];
  readonly validateFeeds?: boolean;
  readonly validateFetches?: boolean;
}

interface OnnxInferenceResult {
  readonly outputs: Readonly<Record<string, OnnxTensorLike>>;
  readonly sessionKey: string;
  readonly providers: readonly OnnxExecutionProviderConfig[];
}

type OnnxBatchInferenceMode = 'independent' | 'stacked';

interface OnnxBatchInferenceRequest {
  readonly session: NousOnnxSessionConfig;
  readonly items: readonly OnnxBatchInferenceItem[];
  readonly fetches?: readonly string[];
  readonly validateFeeds?: boolean;
  readonly validateFetches?: boolean;
  readonly mode?: OnnxBatchInferenceMode;
  readonly maxConcurrency?: number;
  readonly continueOnError?: boolean;
}

interface OnnxBatchInferenceItemResult {
  readonly index: number;
  readonly outputs?: Readonly<Record<string, OnnxTensorLike>>;
  readonly error?: Error;
}

Runtime Loader#

loadOnnxRuntime is what makes onnxruntime-node optional. It loads the runtime lazily via createRequire and validates the module shape before returning it. If the package is not installed, it throws OnnxRuntimeNotAvailableError — a typed error that callers can catch and handle gracefully (e.g., by reporting a clear "GPU inference unavailable" message rather than an unhandled MODULE_NOT_FOUND). Callers can also supply a custom loader via OnnxRuntimeLoaderOptions.load for testing or alternative runtimes.

Source: libs/nous/core/src/runtime-loader.ts.

const DEFAULT_ONNX_RUNTIME_PACKAGE = 'onnxruntime-node';

interface OnnxRuntimeLoaderOptions {
  readonly runtimePackage?: string;
  readonly load?: (moduleSpecifier: string) => Promise<unknown>;
}

class OnnxRuntimeNotAvailableError extends Error {
  readonly runtimePackage: string;
}

function loadOnnxRuntime(
  options?: OnnxRuntimeLoaderOptions
): Promise<OnnxRuntimeModule>;

loadOnnxRuntime resolves onnxruntime-node (or a custom package) via createRequire, validates the module shape (Tensor constructor + InferenceSession.create), unwraps a default export if needed, and throws OnnxRuntimeNotAvailableError when the module is missing. The runtime is an optional dependency, so packages compile without it installed.

4. Execution Providers#

Each execution provider is a dedicated module that knows how to detect its own hardware, recommend a configuration, and produce OnnxSessionOptions with the correct executionProviders array. The shared execution-providers.ts module provides the registry and resolution logic that selects providers at session creation time.

Source: libs/nous/core/src/execution-providers.ts plus per-provider modules (cpu-execution-provider.ts, cuda-execution-provider.ts, rocm-execution-provider.ts, metal-execution-provider.ts, webgpu-execution-provider.ts, vulkan-execution-provider.ts, dml-execution-provider.ts, npu-execution-provider.ts).

Provider Registry and Resolution#

KNOWN_EXECUTION_PROVIDERS is the canonical list of twelve known backend names. DEFAULT_PROVIDER_PREFERENCE is the order in which they are tried when the caller does not specify a preference. PROVIDER_ALIASES maps informal spellings to canonical names so callers are not penalised for using directml instead of dml. resolveExecutionProviders is the core function — it walks the requested list, drops anything unavailable, and appends cpu if it is not already present.

const KNOWN_EXECUTION_PROVIDERS = [
  'cpu',
  'cuda',
  'rocm',
  'dml',
  'coreml',
  'metal',
  'webgpu',
  'vulkan',
  'xnnpack',
  'nnapi',
  'qnn',
  'npu',
] as const; // 12 known providers

const DEFAULT_PROVIDER_PREFERENCE = [
  'cuda',
  'rocm',
  'dml',
  'coreml',
  'metal',
  'webgpu',
  'vulkan',
  'nnapi',
  'qnn',
  'xnnpack',
  'cpu',
] as const;

execution-providers.ts also defines a PROVIDER_ALIASES map (e.g. directml/dx12 → dml, mps → metal, ane/neural_engine → npu, core-ml → coreml).

interface ExecutionProviderEnvironment {
  readonly platform: NodeJS.Platform;
  readonly arch: NodeJS.Architecture;
  readonly cudaAvailable: boolean;
  readonly rocmAvailable: boolean;
  readonly dmlAvailable: boolean;
  readonly coremlAvailable: boolean;
  readonly metalAvailable: boolean;
  readonly webgpuAvailable: boolean;
  readonly vulkanAvailable: boolean;
  readonly nnapiAvailable: boolean;
  readonly qnnAvailable: boolean;
  readonly npuAvailable: boolean;
}

interface ExecutionProviderResolution {
  readonly requested: readonly OnnxExecutionProviderName[];
  readonly resolved: readonly OnnxExecutionProviderName[];
  readonly unavailable: readonly ExecutionProviderUnavailable[];
  readonly fallbackApplied: boolean;
  readonly environment: ExecutionProviderEnvironment;
}

interface ExecutionProviderUnavailable {
  readonly provider: OnnxExecutionProviderName;
  readonly reason: string; // 'unknown_provider' | 'unavailable_in_environment'
}

Exported functions: detectExecutionProviderEnvironment, isExecutionProviderAvailable, getDefaultProviderPreference, normalizeExecutionProviderName, normalizeExecutionProviders, toExecutionProviderConfigs, resolveExecutionProviders, applyExecutionProvidersToSessionOptions, resolveSessionExecutionProviders.

detectExecutionProviderEnvironment reads the process environment for hardware signals (e.g. CUDA_PATH, CUDA_HOME, ROCM_HOME, VULKAN_SDK, LD_LIBRARY_PATH tokens) and process.platform to populate the environment struct. resolveExecutionProviders walks the requested list (or DEFAULT_PROVIDER_PREFERENCE), drops unavailable/unknown providers, and always appends a cpu fallback unless one is present.

CPU Provider#

The CPU provider module detects the hardware profile (core count, platform, architecture) and translates an optimization goal — latency, throughput, or balanced — into specific thread counts for intra-op and inter-op parallelism. All other provider modules follow the same shape: a hardware-profile detector, a recommend*Config function, and a provider class that produces OnnxSessionOptions.

Source: cpu-execution-provider.ts.

type CpuOptimizationGoal = 'latency' | 'throughput' | 'balanced';
type CpuExecutionMode = 'sequential' | 'parallel';

interface CpuHardwareProfile {
  readonly logicalCores: number;
  readonly estimatedPhysicalCores: number;
  readonly model: string;
  readonly platform: NodeJS.Platform;
  readonly arch: NodeJS.Architecture;
}

interface CpuExecutionProviderConfig {
  readonly optimizationGoal?: CpuOptimizationGoal;
  readonly useXnnpack?: boolean;
  readonly useArena?: boolean;
  readonly intraOpNumThreads?: number;
  readonly interOpNumThreads?: number;
  readonly executionMode?: CpuExecutionMode;
  readonly enableCpuMemArena?: boolean;
  readonly enableMemPattern?: boolean;
}

class CpuExecutionProvider {
  constructor(profile?: CpuHardwareProfile);
  getHardwareProfile(): CpuHardwareProfile;
  recommendConfig(
    goal?: CpuOptimizationGoal
  ): Required<CpuExecutionProviderConfig>;
  createSessionOptions(
    config?: CpuExecutionProviderConfig,
    baseOptions?: Omit<OnnxSessionOptions, 'executionProviders'>
  ): CpuSessionBuildResult;
}

detectCpuHardwareProfile reads node:os cpus(); thread-count recommendations are derived from estimated physical cores and the optimization goal. The other provider modules follow the same shape — each exposes a hardware-profile detector, a recommend*Config function, and a provider class producing OnnxSessionOptions with executionProviders populated.

5. Inference Engine and Session Manager#

OnnxSessionManager and OnnxInferenceEngine are the two central classes for running inference. The session manager is the lower-level object: it owns the reference-counted cache of live ONNX sessions. The inference engine is the higher-level facade: it wraps the session manager and adds model loading, feed/fetch validation, and batch modes. Most consumers use the inference engine directly rather than the session manager.

OnnxSessionManager#

Source: libs/nous/core/src/session-manager.ts.

interface OnnxSessionManagerOptions {
  readonly preferredProviders?: readonly OnnxExecutionProviderName[];
  readonly allowProviderFallback?: boolean; // default true
  readonly maxCachedSessions?: number; // default 8
  readonly providerEnvironment?: ExecutionProviderEnvironment;
  readonly providerOverrides?: ExecutionProviderOverrides;
  readonly strictKnownProviders?: boolean; // default false
}

interface OnnxSessionManagerStats {
  readonly totalSessions: number;
  readonly totalReferences: number;
  readonly sessions: readonly {
    key: string;
    providers: readonly OnnxExecutionProviderConfig[];
    references: number;
  }[];
}

class OnnxSessionManager {
  constructor(runtime: OnnxRuntimeModule, options?: OnnxSessionManagerOptions);
  acquire(config: NousOnnxSessionConfig): Promise<ManagedOnnxSession>;
  getStats(): OnnxSessionManagerStats;
  closeSession(key: string): Promise<boolean>;
  disposeAll(): Promise<void>;
}

Behaviour: sessions are cached under a stable key derived from the model source (path string, or an FNV-style fingerprint of the bytes), the resolved provider list, the session options, and the fallback flag. acquire reference-counts entries; concurrent acquire calls for the same key share one in-flight creation. When the cache reaches maxCachedSessions, the least-recently-used zero-reference entry is evicted and its session released. If the primary provider fails and fallback is allowed, the manager retries with a cpu-only configuration. closeSession only succeeds when references are 0.

OnnxInferenceEngine#

Source: libs/nous/core/src/inference-engine.ts.

interface OnnxInferenceEngineOptions {
  readonly runtime?: OnnxRuntimeModule;
  readonly loaderOptions?: OnnxRuntimeLoaderOptions;
  readonly preferredProviders?: readonly OnnxExecutionProviderName[];
  readonly allowProviderFallback?: boolean;
  readonly maxCachedSessions?: number;
  readonly validateInputNames?: boolean;          // default true
  readonly validateOutputNames?: boolean;         // default true
  readonly modelLoader?: OnnxModelLoader;
  readonly modelLoaderOptions?: OnnxModelLoaderOptions;
}

class OnnxInferenceEngine {
  static create(options?: OnnxInferenceEngineOptions): Promise<OnnxInferenceEngine>;
  getRuntime(): OnnxRuntimeModule;
  run(request: OnnxInferenceRequest): Promise<OnnxInferenceResult>;
  runBatch(request: OnnxBatchInferenceRequest): Promise<OnnxBatchInferenceResult>;
  getSessionStats(): OnnxSessionManagerStats;
  registerModel(model: OnnxModelRegistration): void;
  unregisterModel(modelId: string): boolean;
  listRegisteredModels(): /* registered models */;
  preloadModels(modelIds: readonly string[]): Promise<...>;
  resolveModel(model: string): Promise<ResolvedOnnxModel>;
  closeIdleSession(sessionKey: string): Promise<boolean>;
  dispose(): Promise<void>;
}

run resolves the model through OnnxModelLoader, acquires a managed session, optionally validates feed/fetch names against session.inputNames / session.outputNames (throwing on unknown names), converts each feed to a runtime Tensor, and runs inference. runBatch supports two modes: independent (each item run concurrently up to maxConcurrency, with optional continueOnError) and stacked (items concatenated along the batch dimension into a single run call, then split back out — requires matching feed keys, tensor types, and non-batch dimensions, and rejects continueOnError).

OnnxModelLoader#

Source: libs/nous/core/src/model-loader.ts.

interface OnnxModelRegistration {
  readonly id: string;
  readonly source: OnnxModelSource;
  readonly checksumSha256?: string;
  readonly metadata?: Readonly<Record<string, string>>;
}

interface ResolvedOnnxModel {
  readonly id?: string;
  readonly source: OnnxModelSource;
  readonly localPath?: string;
  readonly fromCache: boolean;
  readonly checksumSha256?: string;
}

interface OnnxModelLoaderOptions {
  readonly cacheDir?: string; // default <cwd>/.cache/nous/models
  readonly fetch?: FetchLike;
}

type FetchLike = (url: string) => Promise<FetchResponseLike>;

The loader registers models by id, resolves string sources (treating HTTP URLs as downloadable, caching to cacheDir), verifies SHA-256 checksums via node:crypto, and exposes preload. Buffer/ArrayBuffer sources pass through in memory.

6. Model Management#

The model management layer sits between the application's model configuration and the inference engine's session creation. ModelRegistry is the central catalogue — it maps model IDs and aliases to metadata records. OnnxModelLoader handles the actual loading: string sources are treated as paths or downloadable URLs; ArrayBuffer/Uint8Array sources pass through in memory. The other model modules in @nous/core (versioning, download, checksum verification, conversion, benchmarking, etc.) each follow the same typed *Options/*Result pattern.

ModelRegistry#

Source: libs/nous/core/src/model-registry.ts.

type ModelModality = 'text' | 'vision' | 'audio' | 'multimodal' | 'embedding';

interface ModelRegistryModel {
  readonly id: string;
  readonly displayName?: string;
  readonly provider?: string;
  readonly family?: string;
  readonly modality?: ModelModality;
  readonly task?: string;
  readonly source?: string;
  readonly tags?: readonly string[];
  readonly capabilities?: readonly string[];
  readonly metadata?: Readonly<Record<string, string | number | boolean>>;
  readonly createdAt?: number;
  readonly updatedAt?: number;
  readonly deprecated?: boolean;
}

interface ModelRegistryRegistration extends ModelRegistryModel {
  readonly aliases?: readonly string[];
}

interface ModelRegistryListFilter {
  readonly provider?: string;
  readonly family?: string;
  readonly modality?: ModelModality;
  readonly task?: string;
  readonly tag?: string;
  readonly capability?: string;
  readonly includeDeprecated?: boolean; // default false
}

interface ModelRegistryStats {
  readonly totalModels: number;
  readonly deprecatedModels: number;
  readonly aliasCount: number;
  readonly defaultModelId?: string;
}

class ModelRegistry {
  registerModel(
    reg: ModelRegistryRegistration,
    options?: RegisterModelOptions
  ): void;
  unregisterModel(modelId: string): boolean;
  addAlias(modelId: string, alias: string): void;
  removeAlias(alias: string): boolean;
  resolveModelId(modelIdOrAlias: string): string | undefined;
  getModel(modelIdOrAlias: string): ModelRegistryModel | undefined;
  requireModel(modelIdOrAlias: string): ModelRegistryModel;
  setDefaultModel(modelIdOrAlias: string): void;
  getDefaultModel(): ModelRegistryModel | undefined;
  listModels(filter?: ModelRegistryListFilter): readonly ModelRegistryModel[];
  searchModels(
    query: string,
    options?: ModelSearchOptions
  ): readonly ModelRegistryModel[];
  listAliases(modelIdOrAlias: string): readonly string[];
  getStats(): ModelRegistryStats;
}

RegisterModelOptions is { overwrite?: boolean; setAsDefault?: boolean }. Registration normalizes string fields (rejecting empties), deduplicates and sorts tags/capabilities, and validates aliases (an alias cannot equal a model id or collide with another model's id/alias). The first registered model becomes the default unless setAsDefault overrides it; searchModels substring-matches id, displayName, provider, family, task, tags, and capabilities.

Other Model Modules#

Beyond the registry and loader, @nous/core ships a complete set of model lifecycle modules, each with its own typed *Options/*Result API: model-versioning (immutable version records), model-metadata (rich metadata storage), model-discovery (Hugging Face Hub and local directory scanning), model-download (HTTP download with progress tracking), checksum-verification (SHA-256 integrity checks), model-conversion (cross-format conversion), format-detection (header-byte-based format identification), model-caching (in-memory weight cache), model-eviction (LRU/LFU eviction policies), model-preloading (startup-time preloading), warm-model-pool (pre-initialized session pools), model-benchmarking (tokens/s, latency percentiles), and model-profiling (per-layer time and memory profiling). Format reader modules: safetensors, gguf, ggml, pytorch-loading, tensorflow-loading, jax-loading, tensor.

7. Inference Pipeline Primitives#

All inference pipeline primitives live flat in libs/nous/core/src/. The selected primitives below have their full types documented; the remaining modules follow the same typed-class pattern and are listed at the end of this section.

Dynamic Batching#

DynamicBatchScheduler buffers individual inference requests and dispatches them as groups, improving GPU utilisation by reducing per-request kernel launch overhead. Requests are keyed by a batch key (default: a stable serialisation of session + fetches + flags) so requests for different models queue independently.

Source: dynamic-batching.ts.

interface DynamicBatchExecutor {
  runBatch(
    request: OnnxBatchInferenceRequest
  ): Promise<OnnxBatchInferenceResult>;
}

interface DynamicBatchingOptions {
  readonly maxBatchSize?: number; // default 8
  readonly maxQueueDelayMs?: number; // default 8
  readonly mode?: OnnxBatchInferenceMode;
  readonly maxConcurrency?: number;
  readonly maxQueueSize?: number;
  readonly batchKeyFn?: (request: OnnxInferenceRequest) => string;
}

interface DynamicBatchingStats {
  readonly pendingRequests: number;
  readonly batchesExecuted: number;
  readonly requestsProcessed: number;
  readonly requestsFailed: number;
  readonly requestsDropped: number;
  readonly averageBatchSize: number;
  readonly maxObservedBatchSize: number;
}

class DynamicBatchScheduler {
  /* submit, stats, … */
}

Requests are grouped per batch key (default key = stable serialization of session + fetches + validation flags), flushed when maxBatchSize is reached or maxQueueDelayMs elapses.

KV Cache#

KvCacheManager explicitly manages the key-value buffers that LLM generation accumulates across forward passes. Memory is reserved up front (maxTotalBytesReserved), with per-layer Float32Array buffers allocated at allocate() time. The byte cost formula is capacityTokens × numHeads × headSize × 4 (bytes/float) × 2 (K+V) per layer, summed across all layers — use this to size the cache budget for a given model.

Source: kv-cache.ts.

interface KvCacheConfig {
  readonly modelId: string;
  readonly layerCount: number;
  readonly numHeads: number;
  readonly headSize: number;
  readonly initialCapacityTokens?: number;
}

interface KvLayerAppendInput {
  readonly layer: number;
  readonly tokenCount: number;
  readonly keys: Float32Array;
  readonly values: Float32Array;
}

interface KvCacheSnapshot {
  readonly cacheId: string;
  readonly modelId: string;
  readonly tokenCount: number;
  readonly layers: readonly KvLayerSnapshot[];
}

interface KvCacheStats {
  readonly cacheCount: number;
  readonly totalBytesReserved: number;
  readonly evictions: number;
  readonly cacheIds: readonly string[];
}

interface KvCacheManagerOptions {
  readonly maxTotalBytesReserved?: number; // default 512 MiB
}

class KvCacheManager {
  /* allocate / append / read / evict / stats */
}

Per-layer key/value buffers are Float32Arrays; reserved bytes are computed as capacityTokens * numHeads * headSize * 4 * 2 summed across layers.

Speculative Decoding#

SpeculativeDecoder coordinates a fast draft model and a slower target model. The draft model proposes a batch of candidate tokens; the target model verifies them in a single forward pass. Accepted tokens reduce the number of full-model passes, cutting wall-clock generation latency by 2–3x. createGreedyTargetModel provides a pre-built target verifier for greedy decoding.

Source: speculative-decoding.ts.

interface SpeculativeDraftModel { /* draft token proposal */ }
interface SpeculativeTargetVerification { /* per-token accept/reject */ }
interface SpeculativeTargetModel { /* full-model verification */ }
interface SpeculativeDecodingOptions { /* numSpeculativeTokens etc. */ }
interface SpeculativeDecodingRequest { … }
interface SpeculativeDecodingResult { … }
function createGreedyTargetModel(...): SpeculativeTargetModel;
class SpeculativeDecoder { … }

Multi-LoRA Serving#

MultiLoraServingManager allows multiple LoRA adapters to be registered against a single base model, then selected per-request by adapter ID. Adapter weights are a small fraction of the base model's size (roughly 1–5%), so serving ten adapters from one base model requires far less memory than loading ten separate models.

Source: multi-lora-serving.ts.

interface LoraAdapterRegistration { /* adapter id, base model, … */ }
interface MultiLoraServingOptions { … }
interface MultiLoraRequest { … }
interface MultiLoraExecutionGroup { … }
interface MultiLoraExecutionPlan { … }
interface MultiLoraServingStats { … }
class MultiLoraServingManager { … }

Additional Pipeline Modules#

The following modules are implemented with their own *.spec.ts, each following the same typed-class pattern as the modules documented above. They cover the remaining throughput, attention, parallelism, caching, generation control, and graph optimization capabilities described in features.md §3–§6:

• Throughput / serving: continuous-batching, llm-serving
• Attention and memory: paged-attention, flash-attention
• Multi-GPU parallelism: tensor-parallelism, pipeline-parallelism, model-sharding, moe-parallelism
• Caching: prefix-caching, radix-attention, prompt-caching
• Adapter management: adapter-switching, context-extension
• Generation control: streaming-generation, stop-sequences, logit-processors, temperature-sampling, top-k-top-p-sampling, beam-search
• Graph optimization: graph-optimization, memory-planning, operator-fusion

8. Quantization#

quantization.ts provides the low-level numerical conversion functions that underpin the quantization pipeline. computeSymmetricQuantParams and computeAsymmetricQuantParams derive scale and zero-point from a calibration range. The float32/float16 conversion pairs handle FP16 round-trips. quantizeInt8/dequantizeInt8 handle the INT8 path; packInt4/unpackInt4 pack two nibbles per byte for INT4. These functions are called by the higher-level model-optimization-pipeline module but can also be used directly for custom quantization workflows.

Source: libs/nous/core/src/quantization.ts.

type QuantizationPrecision = 'fp16' | 'int8' | 'int4';
type QuantizationMethod = 'dynamic' | 'static';

interface QuantizationConfig { /* precision + method */ }
interface QuantizationParams { … }
interface CalibrationRange { … }
type QuantizedTensor =
  | QuantizedTensorFp16 | QuantizedTensorInt8 | QuantizedTensorInt4;
interface QuantizationBatchResult { … }

Exported functions include computeSymmetricQuantParams, computeAsymmetricQuantParams, float32ToFloat16, float16ToFloat32, convertFloat32ToFloat16, convertFloat16ToFloat32, quantizeInt8, dequantizeInt8, packInt4, unpackInt4. INT4 values are nibble-packed two per byte.

9. Embeddings and Similarity Search#

The embedding and search modules in @nous/core form the vector search stack that underpins RAG, semantic search, and recommendation. EmbeddingGenerator is the entry point: it manages a priority-ordered provider list and dispatches requests by modality. SimilaritySearchIndex is the retrieval half: it stores (id, vector, metadata) triples and supports cosine, dot, and Euclidean queries. Companion modules add ANN indexing, BM25 hybrid search, and cross-encoder reranking on top of this foundation.

Embedding Generation#

Source: libs/nous/core/src/embedding-generation.ts.

type EmbeddingModality = 'text' | 'image' | 'audio' | 'multimodal';

type EmbeddingContent =
  | string
  | Uint8Array
  | ArrayBuffer
  | readonly (string | Uint8Array | ArrayBuffer)[];

interface EmbeddingInput {
  readonly modality: EmbeddingModality;
  readonly content: EmbeddingContent;
  readonly metadata?: Readonly<Record<string, string | number | boolean>>;
}

interface EmbeddingGenerationRequest {
  readonly modelId: string;
  readonly input: EmbeddingInput;
  readonly providerId?: string;
  readonly outputDimensions?: number;
  readonly timeoutMs?: number;
  readonly allowProviderFallback?: boolean;
  readonly metadata?: Readonly<Record<string, string | number | boolean>>;
}

interface EmbeddingProvider {
  readonly id: string;
  readonly supportedModalities: readonly EmbeddingModality[];
  readonly supportedModels?: readonly string[];
  readonly priority?: number;
  generate(request: EmbeddingProviderRequest): Promise<EmbeddingProviderResult>;
}

interface EmbeddingGenerationResult {
  readonly providerId: string;
  readonly modelId: string;
  readonly modality: EmbeddingModality;
  readonly vector: readonly number[];
  readonly dimensions: number;
  readonly createdAt: number;
  readonly metadata?: Readonly<Record<string, string | number | boolean>>;
}

interface EmbeddingGeneratorStats {
  readonly totalRequests: number;
  readonly successfulRequests: number;
  readonly failedRequests: number;
  readonly generatedVectors: number;
  readonly averageLatencyMs: number;
  readonly providers: readonly EmbeddingProviderStats[];
}

class EmbeddingGenerator {
  /* registerProvider / generate / getStats */
}

Companion modules add modality-specific input processing and post-processing capabilities: text-embeddings, image-embeddings, audio-embeddings, multi-modal-embeddings handle input encoding for each modality; embedding-normalization L2-normalises vectors; dimensionality-reduction applies PCA or random projection; batch-embedding and streaming-embedding handle bulk processing; embedding-caching deduplicates on input hash; embedding-compression, binary-embeddings, and matryoshka-embeddings reduce storage footprint.

Similarity Search#

Source: libs/nous/core/src/similarity-search.ts.

type SimilarityMetricName = 'cosine' | 'dot' | 'euclidean';

interface SimilaritySearchItem {
  readonly id: string;
  readonly embedding: readonly number[];
  readonly metadata?: Readonly<Record<string, string | number | boolean>>;
}

interface SimilaritySearchQuery {
  readonly embedding: readonly number[];
  readonly topK?: number;
  readonly minScore?: number;
  readonly filter?: Readonly<Record<string, string | number | boolean>>;
  readonly includeEmbeddings?: boolean;
}

interface SimilaritySearchResultItem {
  readonly id: string;
  readonly score: number;
  readonly rank: number;
  readonly metadata?: Readonly<Record<string, string | number | boolean>>;
  readonly embedding?: readonly number[];
}

interface SimilaritySearchResult {
  readonly metric: SimilarityMetricName;
  readonly totalCandidates: number;
  readonly evaluatedCandidates: number;
  readonly durationMs: number;
  readonly items: readonly SimilaritySearchResultItem[];
}

class SimilaritySearchIndex {
  /* index / search / stats */
}

Related modules extend SimilaritySearchIndex for different scale and accuracy requirements: approximate-nearest-neighbor (HNSW/IVF for billion-scale datasets), exact-nearest-neighbor (brute-force for small corpora where accuracy is paramount), distance-metrics (cosine, Euclidean, dot, Manhattan), hybrid-search (vector + BM25 with Reciprocal Rank Fusion), reranking (score-based re-ordering), cross-encoder-reranking (BERT cross-encoder two-stage pipeline).

10. LLM Completion APIs (@nous/llm)#

@nous/llm builds the higher-level language-model surface on top of @nous/core. Its two completion classes — CompletionAPI (text) and ChatCompletionAPI (conversation) — follow a provider registry pattern: the application registers one or more provider implementations, and the API dispatches to them by ID with an optional fallback chain. This design decouples the API surface from any specific backend: an application can register an ONNX-backed local provider, a remote API provider, or a mock for testing, and the calling code stays unchanged.

The barrel libs/nous/llm/src/index.ts re-exports 78 modules.

Text Completion#

CompletionAPI is the lower-level interface for open-ended text generation. CompletionAPIRequest extends CompletionRequest with a providerId and optional fallbackProviderIds. The response records which provider was used, the model name reported by that provider, latency, and token usage.

Source: completion-api.ts.

interface CompletionChoice {
  readonly index: number;
  readonly text: string;
  readonly finishReason?: 'stop' | 'length' | 'content_filter' | 'error';
}

interface CompletionUsage {
  readonly promptTokens: number;
  readonly completionTokens: number;
  readonly totalTokens: number;
}

interface CompletionRequest {
  readonly prompt: string;
  readonly model?: string;
  readonly maxTokens?: number;
  readonly temperature?: number;
  readonly topP?: number;
  readonly stop?: readonly string[];
  readonly metadata?: Readonly<Record<string, string | number | boolean>>;
}

interface CompletionProvider {
  readonly id: string;
  readonly models?: readonly string[];
  complete(request: CompletionRequest): Promise<CompletionProviderResult>;
}

interface CompletionAPIRequest extends CompletionRequest {
  readonly providerId?: string;
  readonly fallbackProviderIds?: readonly string[];
}

interface CompletionResponse {
  readonly id: string;
  readonly providerId: string;
  readonly model: string;
  readonly createdAt: number;
  readonly latencyMs: number;
  readonly choices: readonly CompletionChoice[];
  readonly usage: CompletionUsage;
}

class CompletionAPI {
  constructor(options?: CompletionAPIOptions);
  registerProvider(provider: CompletionProvider): void;
  unregisterProvider(providerId: string): boolean;
  setDefaultProvider(providerId: string): void;
  listProviders(): readonly string[];
  complete(request: CompletionAPIRequest): Promise<CompletionResponse>;
  completeText(request: CompletionAPIRequest): Promise<string>;
  getStats(): CompletionAPIStats;
}

The completion layer is provider-registry based: callers register CompletionProvider implementations, then dispatch by providerId (with optional fallbackProviderIds). CompletionAPIOptions is { now?: () => number; idFactory?: () => string }.

Chat Completion#

ChatCompletionAPI is the conversational interface. It uses the same provider-registry pattern as CompletionAPI. The tool_calls finish reason in ChatCompletionChoice signals that the model wants to invoke a tool rather than producing a final response — the caller is responsible for handling the tool call and continuing the conversation.

Source: chat-completion-api.ts.

type ChatMessageRole = 'system' | 'user' | 'assistant' | 'tool';

interface ChatMessage {
  readonly role: ChatMessageRole;
  readonly content: string;
  readonly name?: string;
}

interface ChatCompletionChoice {
  readonly index: number;
  readonly message: ChatMessage;
  readonly finishReason?:
    | 'stop'
    | 'length'
    | 'content_filter'
    | 'error'
    | 'tool_calls';
}

interface ChatCompletionRequest {
  readonly messages: readonly ChatMessage[];
  readonly model?: string;
  readonly maxTokens?: number;
  readonly temperature?: number;
  readonly topP?: number;
  readonly stop?: readonly string[];
  readonly metadata?: Readonly<Record<string, string | number | boolean>>;
}

interface ChatCompletionProvider {
  readonly id: string;
  readonly models?: readonly string[];
  complete(
    request: ChatCompletionRequest
  ): Promise<ChatCompletionProviderResult>;
}

ChatCompletionAPI mirrors CompletionAPI (provider registry + fallback).

Output Shaping and NLP#

The following modules cover structured output enforcement, NLP post-processing, and context management. Each is a dedicated source file with its own *.spec.ts under libs/nous/llm/src/:

• Completion modes: instruction-following, multi-turn-conversation, system-prompts, function-calling, tool-use-framework
• Structured output: json-mode, structured-output, grammar-constraints, regex-constraints, schema-validation
• Response processing: response-formatting, markdown-rendering, code-extraction, citation-extraction, fact-extraction
• NLP: entity-extraction, sentiment-analysis, classification

Reasoning and Persona#

Reasoning and persona modules add layers of structured thinking and identity management on top of the core completion interfaces:

• Reasoning strategies: chain-of-thought, tree-of-thought, graph-of-thought, self-consistency, reflection-prompts, critique-prompts
• Persona and context: persona-management, context-injection, memory-injection, retrieval-injection

11. Prompt Templates#

The prompt template system provides a versioned, testable alternative to string concatenation for prompt construction. PromptTemplateDefinition is the input shape when registering a template; PromptTemplateRecord is the stored shape that adds a revision counter, timestamps, and normalised field defaults. PromptTemplateRenderRequest controls how the template is expanded: strict mode raises an error on missing variables; missingValue provides a fallback string; stringifyObjects serialises complex values to JSON. The unresolvedVariables field in the result identifies any variable that was not provided, even in non-strict mode.

Source: libs/nous/llm/src/prompt-template-system.ts.

type PromptTemplateRole = 'system' | 'user' | 'assistant';

interface PromptTemplateMessage {
  readonly role: PromptTemplateRole;
  readonly content: string;
}

interface PromptTemplateDefinition {
  readonly id: string;
  readonly name: string;
  readonly description?: string;
  readonly version?: string;
  readonly messages: readonly PromptTemplateMessage[];
  readonly variables?: readonly string[];
  readonly tags?: readonly string[];
  readonly metadata?: Readonly<Record<string, string>>;
}

interface PromptTemplateRecord {
  readonly id: string;
  readonly name: string;
  readonly description?: string;
  readonly version: string;
  readonly messages: readonly PromptTemplateMessage[];
  readonly variables: readonly string[];
  readonly tags: readonly string[];
  readonly metadata: Readonly<Record<string, string>>;
  readonly revision: number;
  readonly createdAt: number;
  readonly updatedAt: number;
}

interface PromptTemplateRenderRequest {
  readonly templateId: string;
  readonly values?: Readonly<Record<string, unknown>>;
  readonly includeMetadata?: boolean;
  readonly strict?: boolean;
  readonly missingValue?: string;
  readonly stringifyObjects?: boolean;
}

interface PromptTemplateRenderResult {
  readonly templateId: string;
  readonly version: string;
  readonly messages: readonly PromptTemplateMessage[];
  readonly unresolvedVariables: readonly string[];
  readonly metadata?: Readonly<Record<string, string>>;
}

Each registered template carries a revision counter that increments on update. strict render mode surfaces missing variables; missingValue supplies a fallback string; stringifyObjects serializes object values. Companion modules extend the template system with lifecycle management and example-based prompting: prompt-versioning (version pinning for reproducible deployments), prompt-testing (regression detection across template changes), prompt-optimization (systematic variation search), few-shot-management (input-output example collections), example-selection (embedding-similarity retrieval of the most relevant examples), dynamic-examples (context-adaptive example generation).

12. Agent Architecture#

AgentArchitecture is a runtime that manages the lifecycle of named agents and their task queues. An agent is defined by its AgentDefinition — a set of capabilities, concurrency limits, and pluggable module implementations (planner, goal decomposer, executor). Once registered, an agent transitions through lifecycle states (stopped → idle → running → paused | error) and processes tasks submitted via submitTask. Every state change produces a typed AgentRuntimeEvent that is recorded in the event log, making the full run history queryable for debugging and monitoring.

Source: libs/nous/llm/src/agent-architecture.ts.

State and Capability Enums#

The three enum types below define the vocabulary for agent and task state machines. AgentCapability is used as a tag on AgentDefinition to declare what the agent can do; consumers can query the runtime for agents with specific capabilities.

type AgentLifecycleState = 'stopped' | 'idle' | 'running' | 'paused' | 'error';

type AgentTaskState =
  | 'queued'
  | 'running'
  | 'completed'
  | 'failed'
  | 'cancelled'
  | 'timeout';

type AgentCapability =
  | 'planning'
  | 'goal-decomposition'
  | 'tool-use'
  | 'memory'
  | 'retrieval'
  | 'communication';

Agent and Task Types#

AgentDefinition is the registration input; AgentRecord is the stored record that adds runtime state and task counters. AgentModuleSet must always include an executor (the module that actually runs a task step); the planner and goalDecomposer are optional modules for planning and decomposition respectively. The pluggable modules receive an AgentPlanningContext or AgentExecutionContext that provides state access and an event emitter — they do not call the runtime directly.

interface AgentTaskInput {
  readonly goal: string;
  readonly payload?: unknown;
  readonly context?: Readonly<Record<string, unknown>>;
}

interface AgentPlan {
  readonly summary: string;
  readonly steps: readonly string[];
  readonly confidence?: number;
}

interface AgentSubtask {
  readonly id: string;
  readonly description: string;
  readonly priority?: number;
  readonly metadata?: Readonly<Record<string, unknown>>;
}

interface AgentDefinition {
  readonly id: string;
  readonly name: string;
  readonly description?: string;
  readonly capabilities?: readonly AgentCapability[];
  readonly maxConcurrency?: number;
  readonly defaultTimeoutMs?: number;
  readonly modules: AgentModuleSet;
  readonly metadata?: Readonly<Record<string, unknown>>;
}

interface AgentModuleSet {
  readonly planner?: AgentPlannerModule;
  readonly goalDecomposer?: AgentGoalDecomposerModule;
  readonly executor: AgentExecutorModule; // required
}

interface AgentRecord {
  readonly id: string;
  readonly name: string;
  readonly description?: string;
  readonly capabilities: readonly AgentCapability[];
  readonly state: AgentLifecycleState;
  readonly maxConcurrency: number;
  readonly defaultTimeoutMs: number;
  readonly metadata: Readonly<Record<string, unknown>>;
  readonly createdAt: number;
  readonly updatedAt: number;
  readonly activeTasks: number;
  readonly queuedTasks: number;
  readonly completedTasks: number;
  readonly failedTasks: number;
}

The pluggable modules implement plan(input, context), decompose(input, context), and execute(input, context). AgentPlanningContext extends a getState/setState accessor and exposes agentId, taskId, now, and an emit(event, payload) callback; AgentExecutionContext adds the resolved plan and subtasks.

AgentArchitecture Runtime#

class AgentArchitecture {
  constructor(options?: AgentArchitectureOptions);
  registerAgent(definition: AgentDefinition): AgentRecord;
  unregisterAgent(agentId: string): boolean;
  startAgent(agentId: string): AgentRecord;
  stopAgent(agentId: string, options?: { cancelQueued?: boolean }): AgentRecord;
  pauseAgent(agentId: string): AgentRecord;
  resumeAgent(agentId: string): AgentRecord;
  submitTask(request: AgentTaskRequest): string;     // returns taskId
  runNext(agentId?: string): Promise<AgentTaskRecord | undefined>;
  runAll(...): Promise<...>;
  runTask(taskId: string): Promise<AgentTaskRecord>;
  cancelTask(taskId: string): boolean;
  getAgent(agentId: string): AgentRecord | undefined;
  listAgents(): readonly AgentRecord[];
  getTask(taskId: string): AgentTaskRecord | undefined;
  listTasks(filter?: AgentTaskFilter): readonly AgentTaskRecord[];
  getEvents(limit?: number): readonly AgentRuntimeEvent[];
  getStats(): AgentArchitectureStats;
}

The runtime records typed AgentRuntimeEvents into an event log queryable via getEvents. The companion modules implement the pluggable capabilities declared in AgentCapability:

• Execution: planning-module, goal-decomposition, task-execution, action-selection, tool-selection, multi-step-reasoning, backtracking, error-recovery
• Memory: agent-memory, short-term-memory, long-term-memory, episodic-memory, semantic-memory, working-memory
• Multi-agent: agent-communication, multi-agent-systems, agent-coordination, agent-delegation, agent-supervision

13. Document Ingestion and RAG#

DocumentIngestion is the entry point for turning raw files into indexed, searchable document chunks. It operates through a content-type-matched parser registry: the application registers parsers for application/pdf, text/html, text/markdown, etc., and the ingest() call dispatches to the matching parser, then runs chunking, embedding, and indexing. The result reports a status per document — ingested, duplicate, or failed — so batch ingestion can continue past individual failures.

Source: libs/nous/llm/src/document-ingestion.ts.

type DocumentIngestionSourceType = 'text' | 'file' | 'url' | 'binary';
type DocumentIngestionResultStatus = 'ingested' | 'duplicate' | 'failed';

interface DocumentIngestionInput { /* source type + payload + metadata */ }
type DocumentIngestionParser =
  (request: DocumentIngestionParseRequest) => DocumentIngestionParseResult
    | Promise<DocumentIngestionParseResult>;
type DocumentIngestionParserMatcher =
  string | RegExp | ((contentType: string) => boolean);

interface IngestedDocumentRecord { … }
interface DocumentIngestionResult { … }
interface DocumentIngestionBatchResult { … }
interface DocumentIngestionStats { … }
interface DocumentIngestionEvent { … }

class DocumentIngestion { /* parser registry + ingest + stats + events */ }

DocumentIngestion registers content-type-matched parsers and reports a per-document status (ingested / duplicate / failed). The companion modules cover the full pipeline from raw bytes to searchable index:

• Chunking: chunking-strategies, semantic-chunking, sentence-chunking, token-chunking, recursive-chunking
• Parsing: document-parsing, pdf-parsing, html-parsing, markdown-parsing, code-parsing
• Extraction: table-extraction, image-extraction, metadata-extraction, link-extraction
• Indexing and retrieval: vector-indexing, hybrid-indexing, incremental-indexing, index-optimization, context-retrieval

14. Vision (@nous/vision)#

@nous/vision is the largest package in the domain by module count (103 modules). It covers both understanding (detection, recognition, OCR, scene analysis) and generation (Stable Diffusion, Flux, video synthesis). All models run locally through @nous/core's session manager — there are no external vision API calls. The barrel libs/nous/vision/src/index.ts re-exports all 103 modules, covering still-image and video understanding/generation.

Object Detection#

ObjectDetection returns DetectedObject records with pixel-space bounding boxes, normalised (0..1) bounding boxes, class labels from a COCO-style category union, and confidence scores. The ObjectDetectionBatchResult variant processes multiple images in one call.

Source: object-detection.ts.

type ObjectCategory = /* fixed COCO-style category union */;

interface BoundingBox {
  readonly x: number; readonly y: number;
  readonly width: number; readonly height: number;
}
interface NormalizedBoundingBox { … }   // 0..1 coordinates

interface DetectedObject {
  /* category, confidence, bounding boxes */
}

interface ObjectDetectionRequest { … }
interface ObjectDetectionResult { … }
interface ObjectDetectionBatchResult { … }
interface ObjectDetectionStats { … }

class ObjectDetection { … }

Stable Diffusion#

StableDiffusion covers the SD 1.x and 2.x model families. StableDiffusionModelId is a closed union of the supported checkpoint identifiers. The quality_preset setting (draft, balanced, quality) trades step count against generation time. The safety_policy field controls whether NSFW-classified outputs are passed through (none), flagged (warn), or blocked (block).

Source: stable-diffusion.ts.

type StableDiffusionModelId =
  'sd-1.4' | 'sd-1.5' | 'sd-2.0' | 'sd-2.1' | 'sd-2.1-768';
type StableDiffusionQualityPreset = 'draft' | 'balanced' | 'quality';
type StableDiffusionSafetyPolicy = 'none' | 'warn' | 'block';

interface StableDiffusionGenerationRequest { … }
interface StableDiffusionSafetyAssessment { … }
interface StableDiffusionGeneratedImage { … }
interface StableDiffusionGenerationResult { … }
interface StableDiffusionBatchResult { … }
interface StableDiffusionStats { … }

class StableDiffusion { … }

@nous/vision ships dedicated modules for sdxl-support, sd3-support, flux-support, and a generic diffusion-model-framework / video-diffusion-framework.

Module Groups#

The 103 modules in @nous/vision are organised into the following functional groups. Each module has its own *.spec.ts under libs/nous/vision/src/.

• Detection / classification: object-detection, image-classification, instance-segmentation, semantic-segmentation, panoptic-segmentation, anomaly-detection, multi-object-tracking, object-tracking, re-identification, crowd-analysis, traffic-analysis.
• Facial: facial-recognition, face-landmark-detection, face-restoration, facial-reenactment, emotion-recognition, age-gender-estimation.
• OCR / documents: ocr-engine, handwriting-recognition, document-layout, table-recognition, barcode-qr-recognition, license-plate-recognition.
• Scene understanding: depth-estimation, surface-normal-estimation, pose-estimation, scene-detection, image-captioning, visual-question-answering, action-recognition, trajectory-prediction.
• Image generation / editing: stable-diffusion, sdxl-support, sd3-support, flux-support, controlnet-integration, ip-adapter-support, image-to-image, inpainting, outpainting, background-removal, style-transfer, image-composition, image-editing, upscaling-esrgan, variation-generation, progressive-generation, consistent-character-generation, regional-prompting, pose-guided-generation, depth-aware-generation, edge-guided-generation, semantic-guided-generation, reference-based-generation, lora-management, dreambooth-training, textual-inversion.
• Video: text-to-video, image-to-video, video-to-video, video-editing, video-inpainting, video-outpainting, video-upscaling, video-stylization, video-composition, video-classification, video-captioning, dense-video-captioning, video-question-answering, video-indexing, video-summarization, video-text-matching, frame-interpolation, slow-motion-generation, temporal-super-resolution, temporal-consistency, temporal-segmentation, temporal-grounding, moment-retrieval, shot-boundary-detection, scene-coherence-support, motion-transfer, rotoscoping-automation.

(Other modules — turbo-lightning-modes, sparse-dit-support, moe-routing-support, llama-text-encoder-support, text-rendering-support, prompt-adherence-support, hidream-benchmark-parity, render-mode-support, quality-speed-tradeoff-controls — are present in the barrel and implemented.)

15. Audio (@nous/audio)#

@nous/audio provides the speech, music, and audio analysis surface of Nous. All 81 modules run locally through the same ONNX session infrastructure as @nous/core. The package covers four broad areas: ASR (automatic speech recognition), TTS (text-to-speech synthesis and voice cloning), music generation, and audio analysis / restoration. Each module has its own *.spec.ts.

Source: libs/nous/audio/src/, 81 modules.

The 81 modules group into the four functional areas described below.

• Speech recognition: asr-engine, multi-language-asr, language-detection, custom-vocabulary, hotword-detection, confidence-scores, capitalization, code-switching-support.
• Speech synthesis: neural-tts, multi-speaker-tts, few-shot-cloning, emotion-control, emphasis-control, accent-adaptation, language-mixing.
• Music: music-generation-framework, melody-generation, harmony-generation, accompaniment-generation, chord-progression, chord-recognition, music-continuation, music-style-transfer, music-transcription, midi-generation, audiobook-generation, beat-detection, key-detection.
• Analysis / restoration: audio-classification, audio-scene-classification, audio-segmentation, audio-tagging, genre-classification, mood-classification, audio-search, audio-fingerprinting, content-identification, audio-forensics, audio-enhancement, audio-restoration, audio-inpainting, audio-super-resolution, bandwidth-extension, declipping, denoising, noise-reduction, dereverberation, echo-cancellation, domain-adaptation.

16. Training (@nous/training)#

@nous/training is the largest package in the domain (137 modules). It provides a full model-training surface — not only fine-tuning but also distributed training infrastructure, all major optimiser variants, alignment/preference tuning, diffusion model training, and a complete data pipeline. Two concrete V2 contracts are also implemented here: the anti-cheat classifier training loop (§16.2) and the preference alignment stack.

Source: libs/nous/training/src/, 137 modules.

Module Groups#

The 137 modules cover seven functional areas:

• Distributed training: distributed-training-system, distributed-data-loading, data-parallelism, model-parallelism, tensor-parallelism, pipeline-parallelism, zero-optimization, mixed-precision-training, gradient-checkpointing, gradient-accumulation, fault-tolerance, checkpoint-management.
• Optimizers / schedules: learning-rate-schedulers, warmup-strategies, optimizer-selection, adamw-implementation, lamb-optimizer, adafactor-optimizer, eight-bit-optimizers, loss-functions, metric-tracking.
• Fine-tuning: supervised-fine-tuning, instruction-tuning, chat-fine-tuning, lora-fine-tuning, qlora-fine-tuning, adapter-tuning, prefix-tuning, prompt-tuning, p-tuning, ia3-tuning, dora-tuning, peft-integration, multi-task-fine-tuning.
• Continual learning / distillation: continual-learning, catastrophic-forgetting-prevention, knowledge-distillation, teacher-student-training, self-distillation, progressive-training, curriculum-learning.
• Preference / alignment tuning: reward-modeling, preference-learning, comparison-data-collection, bradley-terry-model, ppo-training, dpo-training, ipo-training, kto-training, orpo-training, rlaif, rejection-sampling, best-of-n-sampling, iterative-refinement, self-play-training, constitutional-ai, red-teaming, safety-training, helpfulness-training, harmlessness-training, honesty-training, preference-aligned-inference, alignment-quality-metrics.
• Diffusion training / acceleration: diffusion-dpo-training, diffusion-curriculum-dpo, diffusion-spo-training, flux-dpo-training, diffusion-dora-training, diffusion-lokr-training, diffusion-oft-training, diffusion-boft-training, lycoris-framework-integration, consistency-model-training, lcm-lora-training, scot-training, deepcache-training-free-acceleration, streamdiffusion-realtime-generation-pipeline, hidiffusion-training-free-high-resolution-generation, senseflow-flow-matching-distillation, and related modules.
• Data pipeline: data-loading-system, data-sharding, streaming-datasets, data-preprocessing, tokenization, packing-for-efficiency, sequence-padding, dynamic-batching, data-augmentation, text-augmentation, back-translation, paraphrasing, data-filtering, quality-scoring, deduplication, data-mixing, curriculum-design, data-selection.

V2 Anti-Cheat Classifier Training Contract#

This is a concrete, domain-specific training contract for the V2 fighting-game project. It trains four classifiers (smurf, win-trading, coordinated-throw, geographic-anomaly) from event-bus labeled data. The contract enforces structural completeness before any training begins — a missing classifier, dataset, or label class causes buildNousV2AntiCheatTrainingPlan to throw NousV2AntiCheatTrainingPlanError. The mayInfluenceRollback: false field is fixed and non-overridable, ensuring trained classifiers can never affect the deterministic gameplay simulation path.

Source: libs/nous/training/src/v2-anti-cheat-classifier-training.ts.

const NOUS_TRAINING_PACKAGE_NAME = '@nous/training';
const NOUS_V2_ANTI_CHEAT_TRAINING_SOURCE_OF_TRUTH_ID =
  'nous.training.v2.anti-cheat-classifier-suite';

const NOUS_V2_ANTI_CHEAT_CLASSIFIERS = [
  'smurf',
  'win-trading',
  'coordinated-throw',
  'geographic-anomaly',
] as const;

type NousV2AntiCheatClassifierId =
  (typeof NOUS_V2_ANTI_CHEAT_CLASSIFIERS)[number];
type NousV2AntiCheatLabel = 'clean' | 'suspected' | 'confirmed';
type NousV2AntiCheatTrainingStage =
  | 'event-bus-ingest'
  | 'feature-extraction'
  | 'offline-training'
  | 'threshold-calibration'
  | 'human-review-shadow';

interface NousV2AntiCheatTrainingPlan {
  readonly schemaVersion: 1;
  readonly sourcePackageName: '@nous/training';
  readonly sourceOfTruthId: typeof NOUS_V2_ANTI_CHEAT_TRAINING_SOURCE_OF_TRUTH_ID;
  readonly suiteId: string;
  readonly generatedAt: string;
  readonly classifierIds: readonly NousV2AntiCheatClassifierId[];
  readonly datasets: readonly NousV2AntiCheatClassifierDataset[];
  readonly labeledEvents: readonly NousV2AntiCheatLabeledEvent[];
  readonly eventBusTopics: readonly string[];
  readonly trainingLoopOwnerPackageName: '@nous/training';
  readonly usesEventBusLabeledData: true;
  readonly modelCardRequired: true;
  readonly modelCardOwnerPackageName: '@nous/safety';
  readonly mayInfluenceRollback: false;
  readonly minimumPrecision: number; // default 0.99
  readonly minimumRecall: number; // default 0.75
  readonly stages: readonly NousV2AntiCheatTrainingStage[];
  readonly diagnostics: NousV2AntiCheatTrainingDiagnostics;
}

function buildNousV2AntiCheatTrainingPlan(
  input: NousV2AntiCheatTrainingPlanInput
): NousV2AntiCheatTrainingPlan;
function validateNousV2AntiCheatTrainingPlanInput(
  input: NousV2AntiCheatTrainingPlanInput
): string[];
class NousV2AntiCheatTrainingPlanError extends Error {
  readonly validationErrors: readonly string[];
}

NOUS_V2_ANTI_CHEAT_CLASSIFIER_EVENT_TOPICS maps each classifier to required v2.* event-bus topics, e.g. smurf → ['v2.match.completed', 'v2.player.skill-profile.updated', 'v2.account.linked']. buildNousV2AntiCheatTrainingPlan throws NousV2AntiCheatTrainingPlanError when any classifier, dataset, event topic, or label class (clean plus at least one non-clean) is missing. The plan's five stages run event-bus-ingest → feature-extraction → offline-training → threshold-calibration → human-review-shadow; mayInfluenceRollback is fixed false.

17. Safety and Governance (@nous/safety)#

@nous/safety hosts safety metadata and governance records for deployed AI systems. It is the package that makes a Nous-served model accountable — every model deployed in the V2 project has a Model Card hosted here that documents its risk class, evaluation results, disclosure obligations, and fallback behaviour. The package also ships a broad set of classifiers and enforcement modules that consuming domains (Kuanyin, Themis) use to build safety policies around.

The boundary is explicit: @nous/safety does not own player settings, product rollback logic, or enforcement decisions. It produces records and classifier outputs; downstream domains decide what to do with them.

Source: libs/nous/safety/src/, 64 modules.

Module Groups#

The 64 modules span six safety and governance areas:

• Content classification / moderation: content-classifier, toxicity-detection, hate-speech-detection, harassment-detection, violence-detection, sexual-content-detection, self-harm-detection, spam-detection, abuse-detection, bot-detection, coordinated-behavior-detection, influence-operation-detection, misinformation-detection, manipulation-detection, synthetic-content-detection, deepfake-detection, watermark-detection, content-fingerprinting.
• Injection / input safety: adversarial-input-detection, prompt-injection-detection, jailbreak-detection, code-injection-detection, sql-injection-detection, xss-detection, secrets-detection, input-validation, output-validation.
• PII: pii-detection, pii-redaction.
• Output enforcement: format-enforcement, length-enforcement, style-enforcement, topic-enforcement, safety-filtering, relevance-filtering.
• Uncertainty / verification: abstention, confidence-estimation, calibration-analysis, uncertainty-quantification, selective-prediction, consistency-checking, factuality-checking, hallucination-detection, error-detection, external-verification, self-correction, explanation-generation.
• Interpretability: interpretability-tools, attribution-analysis, attribution-extraction, goal-inference, intent-detection, preference-modeling, value-alignment-system.
• Oversight / process: human-in-the-loop, oversight-mechanisms, approval-workflows, escalation-system, audit-trails, provenance-tracking.
• Formal verification: kalika-formal-verification — imports @kalika/core (DerivationStep, Expr, ProvenResult) and @kalika/formal-verification (verifyKalikaComputationWithLean, LeanReplSession, …) and wraps ExternalVerifier to produce typed safety invariant results (KalikaFormalSafetyInvariantKind ∈ safety-constraint | invariant-preservation | policy-compliance | output-contract).

V2 Model Card Hosting#

The V2 Model Card system records governance metadata for each AI system deployed in the V2 project. buildV2AIModelCard is the generic builder; the three pre-built wrappers (buildV2AdaptiveAIModelCard, buildV2AICommentaryModelCard, buildV2GenerationPipelinesModelCard) produce cards for specific V2 AI systems. All three ship with riskClass: 'limited', concrete sourcePackages, EU Article 50 disclosure language in transparencyDisclosure, and evaluationResults that reference the benchmarks backing the system. buildV2AIModelCard throws V2ModelCardHostingError on any missing required field, invalid risk class, or empty source/evaluation lists — incomplete Model Cards are never created silently.

Source: libs/nous/safety/src/v2-model-card-hosting.ts.

const NOUS_SAFETY_PACKAGE_NAME = '@nous/safety';
const V2_ADAPTIVE_AI_MODEL_CARD_ID = 'v2.adaptive-ai-director.model-card';
const V2_ADAPTIVE_AI_MODEL_CARD_PATH =
  'V2/docs/ai/model-cards/adaptive-ai-director.md';
const V2_AI_COMMENTARY_MODEL_CARD_ID = 'v2.ai-commentary.model-card';
const V2_AI_COMMENTARY_MODEL_CARD_PATH =
  'V2/docs/ai/model-cards/ai-commentary.md';
const V2_GENERATION_PIPELINES_MODEL_CARD_ID =
  'v2.generation-pipelines.model-card';
const V2_GENERATION_PIPELINES_MODEL_CARD_PATH =
  'V2/docs/ai/model-cards/generation-pipelines.md';

type V2AIActRiskClass = 'minimal' | 'limited' | 'high' | 'prohibited';

interface V2AIModelCardEvaluationResult {
  readonly metric: string;
  readonly value: string;
  readonly threshold: string;
  readonly status: 'pass' | 'watch' | 'fail';
}

interface V2AIModelCard {
  readonly schemaVersion: 1;
  readonly sourcePackageName: '@nous/safety';
  readonly modelCardId: string;
  readonly serviceName: string;
  readonly owner: string;
  readonly riskClass: V2AIActRiskClass;
  readonly purpose: string;
  readonly sourcePackages: readonly string[];
  readonly transparencyDisclosure: string;
  readonly optOutMechanism: string;
  readonly fallbackBehavior: string;
  readonly regionRestrictions: readonly string[];
  readonly evaluationResults: readonly V2AIModelCardEvaluationResult[];
  readonly hostedPath: string;
  readonly hostedBy: '@nous/safety';
  readonly generatedAtUnixMs: number;
  readonly publicationStatus: 'hosted';
}

function buildV2AIModelCard(input: V2AIModelCardInput): V2AIModelCard;
function buildV2AdaptiveAIModelCard(
  overrides?: Partial<V2AIModelCardInput>
): V2AIModelCard;
function buildV2AICommentaryModelCard(
  overrides?: Partial<V2AIModelCardInput>
): V2AIModelCard;
function buildV2GenerationPipelinesModelCard(
  overrides?: Partial<V2AIModelCardInput>
): V2AIModelCard;
class V2ModelCardHostingError extends Error {
  readonly validationErrors: readonly string[];
}

The three pre-built cards (Adaptive AI Director, AI Commentary, Generation Pipelines) are all riskClass: 'limited', carry concrete sourcePackages, transparencyDisclosure, optOutMechanism, fallbackBehavior, regionRestrictions (EU Article 50 disclosure), and evaluationResults. buildV2AIModelCard throws V2ModelCardHostingError on missing required fields, an invalid riskClass, or empty sourcePackages/evaluationResults.

V2 Anti-Cheat Classifier Model Card#

Source: libs/nous/safety/src/v2-anti-cheat-classifier-model-card.ts.

const V2_ANTI_CHEAT_CLASSIFIER_MODEL_CARD_ID =
  'v2.anti-cheat-classifier-suite.model-card';
const V2_ANTI_CHEAT_CLASSIFIER_MODEL_CARD_PATH = /* V2/docs/ai/model-cards/... */;
const V2_ANTI_CHEAT_MODEL_CARD_CLASSIFIERS = /* classifier id list */;

type V2AntiCheatModelCardClassifierId = …;
interface V2AntiCheatClassifierModelCard extends V2AIModelCard { … }
function buildV2AntiCheatClassifierModelCard(...): V2AntiCheatClassifierModelCard;
class V2AntiCheatClassifierModelCardError extends Error { … }

This pairs the @nous/training anti-cheat classifier suite with a hosted, limited-risk Model Card whose evaluation surface enforces review-only, appealable, off-rollback operation.

18. Generative Media Packages#

Eight specialised packages extend the generative-media surface beyond what @nous/vision provides. Each is an implemented TypeScript package with per-module *.spec.ts tests and a typed-class API. The packages are independent of each other — a consuming domain can import only the ones it needs. Module names below are taken directly from each src/index.ts barrel file.

@nous/image-control (50 modules)#

Conditioned image-generation control: controlnet-plus-plus-cycle-consistency, anydoor-reference-object-insertion, attend-and-excite-semantic-binding, boxdiff-layout-control, concept-sliders-lora-directions, attention-based-region-isolation, attention-heat-map-visualization, batch-instruction-editing-pipeline, combined-editing-pipeline, control-backend-comparison-harness, control-condition-preprocessor-registry, control-condition-quality-scorer, and related modules.

@nous/video-control (43 modules)#

Camera/subject control for video diffusion: anyv2v-video-editing-pipeline, camera-trajectory-preset-library, camera-trajectory-visualization-workbench, depthdirector-depth-conditioned-camera-control, hunyuancustom-* (dual-stream processing, multi-subject/multimodal generation, subject-aware temporal coherence, subject-identity preservation), keyframe-based-video-editing-workflow, motion-prompting-camera-control-pipeline, and related modules.

@nous/video-removal (48 modules)#

Object removal / video inpainting (package.json depends on bullmq for job queuing): cogvideox-fun-base-model, diffueraser-integration, avid-integration, grey-mask-generation, edge-artifact-detector, background-reconstruction-quality, automated-quality-gate, human-evaluation-pipeline, gemini-physics-consequence-analyzer, humoto-synthetic-data-pipeline, fine-tuning-quality-validation, and related modules.

@nous/video-to-audio (21 modules)#

Video/image-conditioned audio + Foley: any2audio-* (audio-editing, image-to-audio, text-to-audio, multimodal-conditioning, model-loader, quality-metrics pipelines), av-link-* (inference pipeline, model loader), foley-* (intensity control, library augmentation, A/B comparison), audio-video-sync-quality-validation-pipeline, and related modules.

@nous/advanced-speech (31 modules)#

Advanced TTS / voice cloning / music: f5-tts-model-loader, cosyvoice-integration, fish-audio-s2-pro-integration, cross-tts-voice-cloning, accent-conversion, emotion-prosody-control-api, multi-language-tts-routing, music-generation-evaluation-suite, music-generation-midi-control, music-to-music-style-transfer, music-continuation-style-transition, and related modules.

@nous/generative-relighting (23 modules)#

Image/video/3DGS relighting: ic-light-* (foreground relighting, background generation, model loader), genlit-video-relighting-pipeline, gaussctrl-3dgs-editing-pipeline, ctrl-d-dynamic-3dgs-editing-pipeline, intergsedit-interactive-3dgs-editing-api, intrinsic-edit-pipeline, environment-map-conditioning, lightlab-light-source-control-pipeline, lumigauss-material-decomposition-pipeline, and related modules.

@nous/portrait-animation (23 modules)#

Audio/video-driven portrait animation and dubbing: liveportrait-* (expression-transfer, stitching-animation), hallo3-* (portrait-animation inference, video-DiT model loader), echomimic-audio-to-facial-motion-pipeline, chatanyone-realtime-portrait-pipeline, im-portrait-3d-aware-video-diffusion, multi-person-portrait-animation-pipeline, cross-language-dubbing-pipeline, dubbing-batch-processor, dubbing-quality-validation-pipeline, and related modules.

@nous/joint-av-generation (34 modules)#

Joint audio-video generation with physics awareness: joint-av-generation-library, joint-av-generation-manifest, joint-denoising-pipeline, dual-branch-mmdit-architecture, denoising-temporal-alignment-validator, camera-language-planner, and physics-awareness modules (collision-contact-physics, fabric-material-dynamics, fluid-dynamics-awareness, gravity-simulation-awareness, inertia-momentum-preservation).

19. Concordia Cooperative-Intelligence Packages#

Four packages provide the computational layer for multi-party bargaining and dispute resolution used by the Concordia domain. The boundary is important: these packages own algorithms (preference model fitting, multi-objective search, privacy primitives) but not policy (legal boundaries, settlement authority, product UX). Concordia provides the contracts (@concordia/contracts) that define parties, cases, and outcomes; Nous provides the math and privacy machinery that operates on those contracts.

All four packages have implemented source but no *.spec.ts files yet. All three bargaining packages depend on @concordia/contracts (except @nous/concordia-sealed-memory, which depends only on zod). All use zod for schema validation. Their src/index.ts doc comments reference phase numbers §179.x.

@nous/preference-inference (24 modules)#

Source: libs/nous/preference-inference/src/. Pairwise preference elicitation and five calibrated utility-model estimators (per index.ts doc comment): Bradley-Terry, Thurstone-Mosteller, Plackett-Luce, Gaussian-process preference (Chu-Ghahramani 2005), and a neural utility ranker. Exported surfaces:

• Elicitation: buildElicitationPlan, computeTargetPairCount, meetsStabilityThresholds, PairwiseElicitationPlanSchema, StabilityThresholdsSchema, InferenceModelFamilySchema.
• Comparison prompts: buildPairwiseComparisonPrompt, buildPairwisePromptFromStatements, parsePairwiseComparisonResponse, isDecisive, with ComparisonChoiceSchema, AbstentionReasonSchema, CitationSchema, PairwiseComparisonResponseSchema.
• Estimators: fitBradleyTerry, fitThurstoneMosteller, fitPlackettLuce, fitGaussianProcessPreference, fitNeuralRanker, fitUtilityModel, plus toContractUtilityModel, buildCredibleIntervals.
• Calibration / stability: calibrationMetrics, predictiveCalibration, aggregateStability, runStabilityProbes, prompt-perturbation builders (buildParaphrasePerturbation, buildOrderSwapPerturbation, buildAdversarialFramingPerturbation, …).
• Active learning: computeParetoFrontier, selectActiveLearningPairs, selectStabilityProbes.
• Utility scoring / gating: buildUtilityScores, gatePairwiseComparison, observationsForOptimizer, assertOptimizerSafe, summarizeGatedComparisons.
• Multi-attribute utility: computeMultiAttributeUtility, scoreCandidates, inferIssueWeights.
• Hard constraints: checkFeasibility, filterFeasibleCandidates, guardedUtility, reservationPointConstraint, authorityCeilingConstraint, redlineConstraint.
• BATNA: analyzeBATNA, evaluateAcceptance, computeBatnaPlausibility.
• Fairness: nashProduct, utilitarianSum, maxMinUtility, egalitarianWelfare, kalaiSmorodinskyDistance, envy, regret, inequality, burdenSymmetry, proceduralDignity, computeAllFairnessMetrics, plus BUILTIN_PROFILES, selectFairnessProfile, applyFairnessProfile, detectProtectedClassViolations.

@nous/agreement-search (22 modules)#

Source: libs/nous/agreement-search/src/. Search-kernel registry plus seven implemented search kernels over a candidate-agreement space:

• Registry: SEARCH_KERNEL_REGISTRY, getKernelDescriptor, rankKernelsForCaseProfile, with SearchKernelDescriptorSchema, SearchKernelCapabilitiesSchema, SearchKernelCostProfileSchema.
• Search kernels: runNashGeneticSearch (Nash-product genetic search), runNsgaIISearch (NSGA-II, with fastNonDominatedSort, crowdingDistance), runMapElitesSearch, runMctsSearch (MCTS/LATS), runCpSatSearch (CP-SAT), runBayesianOptimization (with expectedImprovement, upperConfidenceBound, probabilityOfImprovement), runPsroSearch (PSRO opponent modeling), and runCoalitionStabilitySearch (Shapley values exact and Monte-Carlo, least-core allocation, bargaining set).
• Candidate generation / mutation: generateSeedCandidates, ten domain playbooks (procurementPlaybook, creativeRoyaltyPlaybook, daoGovernancePlaybook, platformModerationAppealPlaybook, marketplaceDisputePlaybook, cofounderEquityPlaybook, supplierServiceLevelPlaybook, collaborativeProductionPlaybook, multiplayerGuildGovernancePlaybook, agentToAgentContractPlaybook), ten clause mutators (numericPerturbation, deadlineShift, proportionalSplit, installmentSchedule, equityVesting, royaltyWaterfall, scopeNarrowing, escalationLadder, auditRight, reversibleTrialPeriod), and proposal decomposition/recombination (decomposeProposal, recombineProposals, crossoverPlan).
• GA operators: seeded RNG (mulberry32), crossoverCandidates, applyRandomMutations, produceChild.
• Audits / explanations: auditSpecGaming, auditCoerciveChoice, filterCandidates, renderSharedNeutralExplanation, renderPartyPrivateBriefing, renderParetoFrontierExplanation, computeSteeringBias / createControlRegistry (workbench controls).

@nous/cooperative-bargaining (2 modules)#

Source: libs/nous/cooperative-bargaining/src/. A session-scoped bargaining substrate bridging @concordia/contracts and the agreement-search kernels.

const BargainingPhaseSchema = z.enum([
  'intake',
  'preference_inference',
  'generation',
  'scoring',
  'frontier_stable',
  'review_pending',
  'awaiting_acceptance',
  'closed',
]);
const BargainingClosureReasonSchema = z.enum([
  'accepted',
  'withdrawn_by_party',
  'escalated_to_reviewer',
  'boundary_triggered',
  'budget_exhausted',
  'cancelled_by_mediator',
]);
const BargainingBudgetSchema = z.object({
  maxIterations: z.number().int().positive(),
  maxWallClockSeconds: z.number().int().positive(),
  maxModelSpendMicros: z.string().regex(/^\d+$/), // USD micros, string-encoded
});

StakeholderSchema projects @concordia/contracts Party to the hot-path fields kernels need (partyId, displayName, role, authorityVerificationState ∈ unverified | self_attested | reviewer_verified | documentary_verified | revoked | expired, canAccept, canBindOrganization). Exported functions: buildBargainingSession, advancePhase, closeSession, recordIteration, allStakeholdersCanAccept, stakeholdersBlockingAcceptance, toStakeholder.

@nous/concordia-sealed-memory (5 modules)#

Source: libs/nous/concordia-sealed-memory/src/. Concordia privacy primitives:

• Sealed-memory store (sealed-memory-store.ts): envelope-encrypted store addressed by reference, with a KmsProvider abstraction (createInMemoryKmsProvider), per-operation authorization (AuthorizeFn), and an audit sink. SealedMemoryOperation, SealedMemoryRole, AccessActor, AuditRecord, AuditOutcome are exported types.
• Zero-retention routing (zero-retention-mode.ts): chooseEndpoint, requiredRetentionMode, useCaseRequiresZeroRetention, generateRetentionAttestation, verifyAttestation, createHmacAttestationSigner, with RetentionMode, ZeroRetentionRoutingDecision, LocalModelEndpoint types.
• Confidential compute (confidential-compute.ts): createConfidentialScoringSession, runConfidentialScoring, signTeeAttestation, verifyTeeAttestation, with IsolationTier, TeeAttestation, TeeAttestationPolicy, ConfidentialScoringDecision types.

20. Configuration and Environment#

Configuration Objects#

Nous has no single domain-wide NousConfig object. This is a deliberate consequence of the composable-primitive design principle: a consumer that only needs embedding generation should not pay for configuration schema validation across inference, batching, agent, and safety modules it does not use. Each module is configured through its own *Options/*Config argument, documented in the sections above (e.g. OnnxInferenceEngineOptions, OnnxSessionManagerOptions, DynamicBatchingOptions, KvCacheManagerOptions, CpuExecutionProviderConfig, QuantizationConfig, EmbeddingGeneratorOptions, CompletionAPIOptions, AgentArchitectureOptions). The consumer constructs and wires only the objects it needs.

Environment Variables#

Nous reads environment variables only for hardware-capability detection in the execution-provider modules — there are no NOUS_DEFAULT_MODEL, NOUS_KV_CACHE_MAX_TOKENS, or similar runtime-behaviour overrides. The NOUS_SAFETY_PACKAGE_NAME and NOUS_TRAINING_PACKAGE_NAME identifiers are exported TypeScript constants, not environment variables. The complete list of NOUS_* variables referenced in source is:

detectExecutionProviderEnvironment (Section 4) additionally reads standard toolchain variables — CUDA_PATH, CUDA_HOME, CUDA_VISIBLE_DEVICES, NVIDIA_VISIBLE_DEVICES, ROCM_HOME, HIP_PATH, VULKAN_SDK, ANDROID_ROOT, QNN_SDK_ROOT, and LD_LIBRARY_PATH.

NOUS_SAFETY_PACKAGE_NAME and NOUS_TRAINING_PACKAGE_NAME are exported constants (the package name strings), not environment variables.

There are no NOUS_DEFAULT_MODEL, NOUS_PREFERRED_PROVIDERS, NOUS_KV_CACHE_MAX_TOKENS, NOUS_MODEL_REGISTRY_PATH, or NOUS_LOG_LEVEL variables in the codebase.

21. Integration Points#

This section summarises the declared integration boundaries: what Nous depends on, and how consuming domains wire into it. Engineers adding a new cross-domain dependency should update the table in §21.2 and justify the dependency boundary.

Optional Runtime Dependency#

onnxruntime-node (^1.20.0) is declared as an optionalDependency of @nous/core only. loadOnnxRuntime (Section 3) loads it lazily and throws OnnxRuntimeNotAvailableError when it is absent, so any consumer can compile @nous/core without the native runtime installed. A custom runtime module can be supplied through OnnxRuntimeLoaderOptions.load or OnnxInferenceEngineOptions.runtime.

Cross-Domain Dependencies#

The following workspace dependencies are the only cross-domain package.json entries across all nineteen implemented packages. Any other external symbols visible in a @nous/* package are brought in through devDependencies (e.g. @types/node) and do not create a runtime dependency. The reason for each boundary is explained in architecture.md §12.

These are the only declared cross-domain library dependencies in any libs/nous/*/package.json. @concordia/contracts and @kalika/* belong to the Concordia and Kalika domains respectively.

V2 Composition Boundary#

The V2 contracts in @nous/training and @nous/safety are designed for composition by V2-layer packages. The training plan fixes modelCardOwnerPackageName: '@nous/safety' and mayInfluenceRollback: false; the model cards fix hostedBy: '@nous/safety'. Nous owns the training loop and the Model Card record; product rollback/enforcement behaviour is owned outside Nous.

21a. V2 Cross-Domain Service Contracts#

The V2 fighting-game project composes the Nous libraries described above into shipping services under V2/services/. Each service is a thin, deterministic composition layer: it imports the Nous primitives, fixes the cross-domain ownership fields, and exposes a single validated surface object. None of these services may touch the rollback netcode path — every surface fixes mayInfluenceRollback: false, and the rollback simulation never opens an RPC to a classifier, a Model Card host, or a dispute substrate. This subsection documents the four cross-domain boundaries that consume Nous from V2.

V2 Anti-Cheat Classifier Contract (@v2/nous-anti-cheat-classifiers)#

The V2 Anti-Cheat Classifier Contract is the composition surface that pairs the @nous/training classifier suite (§16) with its hosted @nous/safety Model Card (§17). The service package @v2/nous-anti-cheat-classifiers depends on both @nous/training and @nous/safety at workspace:* and exposes buildV2NousAntiCheatClassifierSuite, which builds the training plan and the Model Card together, then cross-validates them before returning a single V2NousAntiCheatClassifierSuite.

The four classifiers are fixed and domain-specific: smurf, win-trading, coordinated-throw, and geographic-anomaly. The suite trains them off-rollback from event-bus labeled data (usesEventBusLabeledData: true, v2PublishesLabeledTrainingData: true) and surfaces inference only as a review aid — never as automatic enforcement. The contract pins humanReviewRequired: true, appealsRequired: true, automatedDisciplineAllowed: false, and deprecatedMlPipelineAbsent: true, so the deprecated @nous/ml-pipeline path can never be reintroduced. Inference output feeds first-line moderation as evidence; a human reviewer makes every disciplinary decision, and every decision is appealable.

buildV2NousAntiCheatClassifierSuite throws V2NousAntiCheatClassifierSuiteError (with a populated validationErrors array) when a classifier is missing from either the training plan or the Model Card, when the training loop is not owned by @nous/training, or when the Model Card is not hosted by @nous/safety. The suite also emits an anti_cheat audit-publication request (see "Audit Publication" below) so that every review event lands in the retained Oshun audit ledger.

Source: apps/v2/nous-anti-cheat-classifiers/src/nous-anti-cheat-classifiers.ts.

V2 Adaptive AI Director and EU AI Act Surface#

Nous is one of three domains behind the V2 Adaptive AI Director EU AI Act conformity surface (@v2/eu-ai-act-surface). Nous owns the Model Card leg: @nous/safety hosts the limited-risk cards for every shipping V2 AI system — the Adaptive AI Director (adaptive-ai-director.md), AI commentary (ai-commentary.md), the generation pipelines (generation-pipelines.md), and the anti-cheat classifier suite. buildV2AdaptiveAIModelCard, buildV2AICommentaryModelCard, buildV2GenerationPipelinesModelCard, and buildV2AntiCheatClassifierModelCard (§17) all set hostedBy: '@nous/safety' and publicationStatus: 'hosted'.

The other two legs sit outside Nous: @psyche/action-safety produces the player-facing classifier transparency notice and opt-out, and @themis/accountability registers each Model Card as an AI-system-of-record and owns the regulator-facing audit export. The composition service binds the three together (modelCardHostedBy: '@nous/safety', auditExportOwnedBy: @themis/accountability) while keeping the whole surface off the deterministic rollback path. The anti-cheat Model Card additionally links @themis/accountability and @themis/dispute-resolution so that appeals are routed to a real dispute workflow rather than handled implicitly.

V2 Concordia Cooperative-Intelligence Substrate (@v2/concordia-substrate)#

The Concordia cooperative-intelligence packages (§19) are composed by @v2/concordia-substrate into the dispute-resolution substrate that powers anti-cheat appeals, tournament-result disputes, and crew conflicts. The service wires all four Nous primitives — @nous/cooperative-bargaining (the bargaining session and acceptance ledger), @nous/preference-inference (Bradley–Terry elicitation over disputed terms), @nous/agreement-search (Nash genetic search over the candidate-agreement space), and @nous/concordia-sealed-memory (the party-isolated, KMS-sealed intake store) — alongside @iris/concordia-assistant for appellant/arbiter dialogue, @oshun/concordia-integration for event routing, and @kuanyin/concordia-restorative for the safety-gated restorative branch. The whole substrate is flag-gated behind ENABLE_V2_CONCORDIA_SUBSTRATE and fixes offRollback: true / mayInfluenceRollback: false. Sealed-memory keeps each party's intake private and exposes only sealed references, so an arbiter never sees the opposing party's context. The Nous side of this boundary is unchanged from §19: Nous ships reusable preference/search/bargaining/privacy primitives, and Concordia (through the V2 substrate) owns the dispute policy and settlement authority.

Source: apps/v2/concordia-substrate/src/concordia-substrate.ts.

Audit Publication through @oshun/audit-platform#

The anti-cheat service does not retain its own audit log. Instead, V2 publishes canonical audit events through @oshun/audit-platform, and Oshun retains and exports them. The shared @oshun/audit-platform publisher accepts three V2 publication kinds — moderation, anti-cheat, and DSR (data-subject request) — and @v2/nous-anti-cheat-classifiers emits the anti_cheat kind via buildV2AuditPublicationRequest({ kind: 'anti_cheat', … }) on the canonical v2.anti_cheat.review_published action. The publication is tagged with the v2-anti-cheat-audit-retention policy and carries v2Publishes: true / oshunRetainsAndExports: true, so a single Oshun-owned ledger holds the retained record and serves the regulator/investigation export. This is the same routing used by Kuanyin first-line moderation and the Themis privacy DSR service; the cross-domain contract is documented in V2/docs/integration/v2-audit-platform-routing.md.

22. Acceptance Criteria#

These criteria apply to any pull request that touches libs/nous/. They encode the architectural invariants described in this document and in architecture.md. A reviewer should check each criterion independently — passing the test suite alone does not satisfy all of them.

A change to the Nous domain is acceptable when:

1. Build and typecheck pass. tsc --noEmit -p libs/nous/<pkg>/tsconfig.lib.json succeeds for every touched package, and the @nx/esbuild:esbuild build target produces ESM output.
2. Tests pass. Every implemented module in core, llm, vision, audio, training, safety, and the eight generative-media packages has a sibling *.spec.ts; vitest must pass for touched packages. New modules in those packages must add a sibling spec.
3. Types are exhaustive. New domain objects use discriminated literal unions (not loose strings) for state/kind fields, mirroring OnnxExecutionProviderName, AgentLifecycleState, AgentTaskState, BargainingPhaseSchema, and NousV2AntiCheatLabel.
4. Provider abstraction holds. Inference code routes through resolveExecutionProviders / OnnxSessionManager; no module hard-codes a single backend or bypasses provider fallback.
5. The runtime stays optional. @nous/core continues to compile and import without onnxruntime-node; the runtime is reached only via loadOnnxRuntime.
6. V2 contracts validate strictly. buildNousV2AntiCheatTrainingPlan and buildV2AIModelCard (and the wrappers) reject incomplete input by throwing their typed error classes with a populated validationErrors array.
7. Concordia boundary is preserved. @nous/preference-inference, @nous/agreement-search, @nous/cooperative-bargaining, and @nous/concordia-sealed-memory expose only reusable preference/search/bargaining/privacy primitives; domain policy and settlement authority remain in the Concordia domain.
8. Barrel exports stay complete. New modules are re-exported from the owning package's src/index.ts.

Document Provenance#

This specification was written against the source under libs/nous/ at the state captured in the working tree. Type signatures, enum members, class methods, constants, and identifiers are quoted from: core/src/types.ts, core/src/runtime-loader.ts, core/src/execution-providers.ts, core/src/cpu-execution-provider.ts, core/src/session-manager.ts, core/src/inference-engine.ts, core/src/model-loader.ts, core/src/model-registry.ts, core/src/dynamic-batching.ts, core/src/kv-cache.ts, core/src/quantization.ts, core/src/embedding-generation.ts, core/src/similarity-search.ts, core/src/{speculative-decoding,multi-lora-serving}.ts, llm/src/{completion-api,chat-completion-api,prompt-template-system,agent-architecture,document-ingestion}.ts, vision/src/{object-detection,stable-diffusion}.ts, training/src/v2-anti-cheat-classifier-training.ts, safety/src/{v2-model-card-hosting,v2-anti-cheat-classifier-model-card,kalika-formal-verification}.ts, cooperative-bargaining/src/bargaining-session.ts, and the src/index.ts barrels of all nineteen implemented packages, plus every package.json and project.json under libs/nous/. Module-name lists are taken directly from the corresponding src/index.ts barrel files.