Replay-based continual learning (Phase 177.8), in PyTorch, CPU-runnable.
Components#
- CLEAR (
clear.py, 177.8.1.1) — Rolnick et al. Mixed on/off-policy replay with V-trace off-policy correction (clipped importance weights, recursive value targets) plus policy- and value-cloning terms.vtracereduces to the discounted Monte-Carlo return on-policy (verified exactly). - GEM & A-GEM (
gem.py, 177.8.1.2) — Lopez-Paz & Ranzato / Chaudhry et al. GEM projects the update off any conflicting past-task gradient via a dual QP solved with projected gradient (no SciPy); A-GEM uses the single-reference closed form. Episodic memory computes the reference gradients. - DER++ (
der.py, 177.8.1.3) — Buzzega et al. Reservoir memory of inputs + stored logits; replay loss = α logit-distillation + β label-replay. Verified to cut forgetting vs naive fine-tuning. - Generative replay (
generative_replay.py, 177.8.1.4) — Shin et al. A small VAE (ELBO = reconstruction + KL) generates prior-task pseudo-samples, labelled by a frozen solver snapshot (scholar pair), so no original example is stored. - Privacy-preserving replay (
privacy.py, 177.8.1.5) — row clipping, the Gaussian DP mechanism (σ = √(2ln(1.25/δ))·Δ/ε), greedy k-anonymisation (centroid groups of ≥ k), and SHA-256 counter-mode keystream encryption at rest.
Tests#
python3 -m pytest tests/ -q — 22 tests: A-GEM/GEM constraint satisfaction +
no-op-when-safe, V-trace = MC return on-policy + clipping + monotonicity, CLEAR
loss assembly, reservoir capacity/coverage + DER++ forgetting reduction, VAE ELBO
- sample-distribution match + scholar labelling, and DP σ formula + k-anonymity group sizes + encrypt/decrypt round-trip + wrong-key failure.