Mechanistic-interpretability foundations (Phase 177.1), implemented in PyTorch
and runnable end to end on CPU (forward and backward). Provides the hooking,
capture, statistics, cross-model adapter, and attribution primitives that the
SAE (@nous/sae), circuits (@nous/circuits), and probing (@nous/probes)
tracks build on.
Components#
- Hook machinery (
hooks.py) — a TransformerLens-compatibleHookPoint/HookedRootModulemodel (run_with_cache,run_with_hooks) plus a genericHookedModelwrapper that installsnn.Moduleforward/pre hooks on any externally-served checkpoint, exposing streaming hooks at every residual-stream layer, MLP pre/post, attention Q/K/V/O, and logits. (177.1.1.1) - Reference transformer (
transformer.py) — a real GPT-2-styleHookedTransformer(learned embeddings, pre-LN blocks, multi-head causal attention with per-head Q/K/V/pattern/Z hooks, GELU MLP, unembed) that overfits a toy sequence in tests, so every tool runs against a genuine model. - Activation capture (
capture.py) — a format-correct Zarr-v2 uncompressedChunkedStore(per-array.zarray, one raw C-order chunk file per occupied chunk) and anActivationCapturerecorder keyed by(model_rev, dataset, sample_id, layer, point)with per-sample metadata and deterministic, bit-for-bit replay (verify_replay). (177.1.1.2) - Online statistics (
stats.py) —ActivationStats: Welford mean/variance, Chan-et-al. parallel covariance co-moments (associativemerge), diagonal empirical Fisher from per-sample gradients, and p100 activation magnitudes — the inputs to SAE initialisation and Fisher-based EWC. (177.1.1.3) - Cross-model adapters (
adapters.py) —ModelAdapterspecs mapping the real HF submodule naming of Llama-4 MoE, Qwen3 dense + MoE, Gemma-4, Mixtral / DeepSeek-V3 MoE routers, and Nous fine-tunes onto the canonical hook names, resolving loudly (missing path → error) against a live module. (177.1.1.4) - Causal patching (
patching.py) — exact activation patching (per-node forward intervention, denoising, normalised recovered-metric fraction) and first-order attribution patching (single corrupt forward+backward, O(1) per prompt) whose summed effect matches the directional derivative of the exact patch. (177.1.2.1, 177.1.2.3) - Attribution backends (
attribution.py) — Integrated Gradients (passes the completeness axiom exactly), SmoothGrad (± input), and LIME (locally-weighted ridge surrogate over token on/off perturbations) over the embedding input. (177.1.2.2)
Tests#
python3 -m pytest tests/ -q — 24 tests asserting domain correctness: causal +
normalised attention patterns, overfit convergence, chunk-boundary store
round-trips, deterministic capture replay, moment/covariance agreement with a
NumPy reference and merge associativity, adapter path resolution against real HF
naming (dense + MoE router), full-clean patch recovery, the attribution/directional-
derivative identity, and the IG completeness axiom.