Continual-learning foundations (Phase 177.5), in PyTorch, CPU-runnable.
Components#
- Task stream (
stream.py, 177.5.1.1) —ContinualTaskStreamofTasks (train/test split, label set,eval_fn), supporting disjoint task-incremental and blurry class-incremental regimes, the growing class space, and blurry- overlap detection. - Forgetting metrics (
metrics.py, 177.5.1.2) — a GEMAccuracyMatrixR[i,j]and the derived ACC, BWT (backward transfer / forgetting), FWT (forward transfer vs a random-init baseline), per-task learning curves and per-task forgetting; aForgettingMetricdriver records a row after each task. - Benchmark suite (
benchmarks.py, 177.5.1.3) — Permuted MNIST (fixed pixel permutations), Split CIFAR-100 (disjoint class groups), CLEAR-style temporal streams (chronological period buckets), and a real Oshun synthetic stream (Iris intent + Isis style + Kuan-Yin harms). Continual World / LORA-CL fail loud (external envs/checkpoints) rather than fabricate data. - Oshun domain stream (
oshun_stream.py, 177.5.1.4) — build a stream from ordered period datasets (Iris/week, Kuan-Yin/month, Galatea/fleet-cohort, Veritas/cycle) and aRetrogradeSamplerimplementing the Phase-85 flywheel's recency-weighted (decay**k) replay over history.
Regularisation-based CL (Phase 177.7)#
- EWC / Online EWC (
ewc.py, 177.7.1.1/2) — diagonal empirical Fisher, the quadratic anchor penaltyΣ (λ/2)·F·(θ−θ*)², multi-task anchors or a running Fisher (F ← γF + F_new). Verified to cut forgetting on a conflicting-task pair (0.85 vs 0.04 task-0 retention) — the Permuted-MNIST BWT ≥ −2% target on a synthetic proxy. - Synaptic Intelligence (
importance.py, 177.7.1.3) — online path-integral importanceω = −Σ g·Δθ, normalised by parameter movement; no extra backward. - Memory-Aware Synapses (
importance.py, 177.7.1.4) — unsupervised output-sensitivity importanceE[|∂½‖f(x)‖²/∂θ|]. - Automatic λ-tuning (
lambda_tuning.py, 177.7.1.5) — a 1-D GP (RBF kernel) + Expected-Improvement Bayesian optimiser overlog10(λ), walking the held-out BWT/ACC frontier without exhaustive sweeps.
Tests#
python3 -m pytest tests/ -q — 29 tests: default-accuracy eval + class inference,
class-incremental growth + blurry detection; ACC/BWT/FWT against hand-computed
values, running metrics, learning curves + forgetting, FWT baseline guard;
Permuted-MNIST label preservation + distinct permutations, Split-CIFAR disjoint
classes, temporal partition, Oshun three-domain disjoint labels, external
fail-loud; domain stream ordering, retrograde recency weighting + sampling.