# nous-interp-core

Mechanistic-interpretability foundations (Phase 177.1), implemented in PyTorch
and runnable end to end on CPU (forward **and** backward). Provides the hooking,
capture, statistics, cross-model adapter, and attribution primitives that the
SAE (`@nous/sae`), circuits (`@nous/circuits`), and probing (`@nous/probes`)
tracks build on.

## Components

- **Hook machinery** (`hooks.py`) — a TransformerLens-compatible `HookPoint` /
  `HookedRootModule` model (`run_with_cache`, `run_with_hooks`) plus a generic
  `HookedModel` wrapper that installs `nn.Module` forward/pre hooks on any
  externally-served checkpoint, exposing streaming hooks at every residual-stream
  layer, MLP pre/post, attention Q/K/V/O, and logits. (177.1.1.1)
- **Reference transformer** (`transformer.py`) — a real GPT-2-style
  `HookedTransformer` (learned embeddings, pre-LN blocks, multi-head causal
  attention with per-head Q/K/V/pattern/Z hooks, GELU MLP, unembed) that
  overfits a toy sequence in tests, so every tool runs against a genuine model.
- **Activation capture** (`capture.py`) — a format-correct Zarr-v2 uncompressed
  `ChunkedStore` (per-array `.zarray`, one raw C-order chunk file per occupied
  chunk) and an `ActivationCapture` recorder keyed by
  `(model_rev, dataset, sample_id, layer, point)` with per-sample metadata and
  deterministic, bit-for-bit replay (`verify_replay`). (177.1.1.2)
- **Online statistics** (`stats.py`) — `ActivationStats`: Welford mean/variance,
  Chan-et-al. parallel covariance co-moments (associative `merge`), diagonal
  empirical Fisher from per-sample gradients, and p100 activation magnitudes —
  the inputs to SAE initialisation and Fisher-based EWC. (177.1.1.3)
- **Cross-model adapters** (`adapters.py`) — `ModelAdapter` specs mapping the
  real HF submodule naming of Llama-4 MoE, Qwen3 dense + MoE, Gemma-4,
  Mixtral / DeepSeek-V3 MoE routers, and Nous fine-tunes onto the canonical hook
  names, resolving loudly (missing path → error) against a live module.
  (177.1.1.4)
- **Causal patching** (`patching.py`) — exact activation patching (per-node
  forward intervention, denoising, normalised recovered-metric fraction) and
  first-order attribution patching (single corrupt forward+backward, O(1) per
  prompt) whose summed effect matches the directional derivative of the exact
  patch. (177.1.2.1, 177.1.2.3)
- **Attribution backends** (`attribution.py`) — Integrated Gradients (passes the
  completeness axiom exactly), SmoothGrad (± input), and LIME (locally-weighted
  ridge surrogate over token on/off perturbations) over the embedding input.
  (177.1.2.2)

## Tests

`python3 -m pytest tests/ -q` — 24 tests asserting domain correctness: causal +
normalised attention patterns, overfit convergence, chunk-boundary store
round-trips, deterministic capture replay, moment/covariance agreement with a
NumPy reference and merge associativity, adapter path resolution against real HF
naming (dense + MoE router), full-clean patch recovery, the attribution/directional-
derivative identity, and the IG completeness axiom.