Eve library

nous-probes

Other

Revision 1

On this page

Probing & representation analysis for interpretability (Phase 177.4), in PyTorch, CPU-runnable. Operates on activation tensors from @nous/interp-core.

Components#

  • Linear probing (linear.py, 177.4.1.1) — multinomial L2-regularised logistic regression trained to convergence with L-BFGS, K-fold cross-validation, percentile bootstrap CIs on accuracy, and per-layer / per-position sweeps to locate where a property becomes linearly decodable.
  • Representation similarity (similarity.py, 177.4.1.2) — RSA (RDMs + Spearman correlation of their upper triangles) and CKA (linear + RBF-kernel, HSIC-normalised) for comparing checkpoints, model families, and model↔brain (NSD / Algonauts) representations.
  • TCAV (tcav.py, 177.4.1.3) — concept activation vectors (unit normal of a concept-vs-random linear classifier), directional-derivative sensitivity, the TCAV score, statistical TCAV against random directions, and the causal concept effect (inject α·CAV, measure the real output change).
  • Emergent-capability probes (emergent.py, 177.4.1.4) — track a probe's cross-validated accuracy across training checkpoints, flag the emergence checkpoint (threshold crossing) and the largest accuracy jump, for scaling-law capability-onset analysis.

Tests#

python3 -m pytest tests/ -q — 16 tests: probe recovers separable classes / stays at chance on noise under strong L2, bootstrap CI brackets accuracy, layer/position sweeps prefer the informative layer/position; RSA identity + ordering, CKA orthogonal/scale invariance and low value for independent reps; CAV recovers the concept axis, TCAV score high/low/chance by target alignment, causal effect matches the analytic linear head; and emergence detection at the decodability transition.