Eve library

nous-replay

Other

Revision 1

On this page

Replay-based continual learning (Phase 177.8), in PyTorch, CPU-runnable.

Components#

  • CLEAR (clear.py, 177.8.1.1) — Rolnick et al. Mixed on/off-policy replay with V-trace off-policy correction (clipped importance weights, recursive value targets) plus policy- and value-cloning terms. vtrace reduces to the discounted Monte-Carlo return on-policy (verified exactly).
  • GEM & A-GEM (gem.py, 177.8.1.2) — Lopez-Paz & Ranzato / Chaudhry et al. GEM projects the update off any conflicting past-task gradient via a dual QP solved with projected gradient (no SciPy); A-GEM uses the single-reference closed form. Episodic memory computes the reference gradients.
  • DER++ (der.py, 177.8.1.3) — Buzzega et al. Reservoir memory of inputs + stored logits; replay loss = α logit-distillation + β label-replay. Verified to cut forgetting vs naive fine-tuning.
  • Generative replay (generative_replay.py, 177.8.1.4) — Shin et al. A small VAE (ELBO = reconstruction + KL) generates prior-task pseudo-samples, labelled by a frozen solver snapshot (scholar pair), so no original example is stored.
  • Privacy-preserving replay (privacy.py, 177.8.1.5) — row clipping, the Gaussian DP mechanism (σ = √(2ln(1.25/δ))·Δ/ε), greedy k-anonymisation (centroid groups of ≥ k), and SHA-256 counter-mode keystream encryption at rest.

Tests#

python3 -m pytest tests/ -q — 22 tests: A-GEM/GEM constraint satisfaction + no-op-when-safe, V-trace = MC return on-policy + clipping + monotonicity, CLEAR loss assembly, reservoir capacity/coverage + DER++ forgetting reduction, VAE ELBO

  • sample-distribution match + scholar labelling, and DP σ formula + k-anonymity group sizes + encrypt/decrypt round-trip + wrong-key failure.