Eve library

nous-interp-core

Other

Revision 1

On this page

Mechanistic-interpretability foundations (Phase 177.1), implemented in PyTorch and runnable end to end on CPU (forward and backward). Provides the hooking, capture, statistics, cross-model adapter, and attribution primitives that the SAE (@nous/sae), circuits (@nous/circuits), and probing (@nous/probes) tracks build on.

Components#

  • Hook machinery (hooks.py) — a TransformerLens-compatible HookPoint / HookedRootModule model (run_with_cache, run_with_hooks) plus a generic HookedModel wrapper that installs nn.Module forward/pre hooks on any externally-served checkpoint, exposing streaming hooks at every residual-stream layer, MLP pre/post, attention Q/K/V/O, and logits. (177.1.1.1)
  • Reference transformer (transformer.py) — a real GPT-2-style HookedTransformer (learned embeddings, pre-LN blocks, multi-head causal attention with per-head Q/K/V/pattern/Z hooks, GELU MLP, unembed) that overfits a toy sequence in tests, so every tool runs against a genuine model.
  • Activation capture (capture.py) — a format-correct Zarr-v2 uncompressed ChunkedStore (per-array .zarray, one raw C-order chunk file per occupied chunk) and an ActivationCapture recorder keyed by (model_rev, dataset, sample_id, layer, point) with per-sample metadata and deterministic, bit-for-bit replay (verify_replay). (177.1.1.2)
  • Online statistics (stats.py) — ActivationStats: Welford mean/variance, Chan-et-al. parallel covariance co-moments (associative merge), diagonal empirical Fisher from per-sample gradients, and p100 activation magnitudes — the inputs to SAE initialisation and Fisher-based EWC. (177.1.1.3)
  • Cross-model adapters (adapters.py) — ModelAdapter specs mapping the real HF submodule naming of Llama-4 MoE, Qwen3 dense + MoE, Gemma-4, Mixtral / DeepSeek-V3 MoE routers, and Nous fine-tunes onto the canonical hook names, resolving loudly (missing path → error) against a live module. (177.1.1.4)
  • Causal patching (patching.py) — exact activation patching (per-node forward intervention, denoising, normalised recovered-metric fraction) and first-order attribution patching (single corrupt forward+backward, O(1) per prompt) whose summed effect matches the directional derivative of the exact patch. (177.1.2.1, 177.1.2.3)
  • Attribution backends (attribution.py) — Integrated Gradients (passes the completeness axiom exactly), SmoothGrad (± input), and LIME (locally-weighted ridge surrogate over token on/off perturbations) over the embedding input. (177.1.2.2)

Tests#

python3 -m pytest tests/ -q — 24 tests asserting domain correctness: causal + normalised attention patterns, overfit convergence, chunk-boundary store round-trips, deterministic capture replay, moment/covariance agreement with a NumPy reference and merge associativity, adapter path resolution against real HF naming (dense + MoE router), full-clean patch recovery, the attribution/directional- derivative identity, and the IG completeness axiom.