# nous-progressive-nets

Progressive networks & parameter-expansion continual learning (Phase 177.6), in
PyTorch, CPU-runnable.

## Components

- **Progressive Networks** (`progressive.py`, 177.6.1.1) — Rusu & Hadsell 2016.
  A growing stack of columns; adding a column freezes all prior columns, and each
  column reads lateral adapters from prior columns' previous-layer activations.
  The backward pass only touches the new column, so earlier tasks are preserved
  **bit-for-bit** (verified: task-0 predictions are identical after training
  column 1).
- **Column-pruning distillation** (`column_pruning.py`, 177.6.1.3) — distil
  rarely-used frozen columns into a compact shared trunk with per-task heads
  (logit KD + label CE), gated on measured skill retention.
- **Dynamically Expandable Networks** (`den.py`, 177.6.2.1) — Yoon et al. 2018.
  Selective retraining with L1, then loss-triggered width expansion training only
  the new units, old units frozen.
- **PackNet** (`packnet.py`, 177.6.2.2) — Mallya & Lazebnik 2018. One backbone,
  iterative magnitude pruning, per-task weight ownership + per-task biases, and
  the two-phase train→prune→re-train schedule. Earlier tasks preserved exactly.
- **Supermasks** (`supermask.py`, 177.6.2.3) — Zhou et al. / edge-popup. Frozen
  random weights, per-task learnable scores, top-k binary masks with a
  straight-through estimator; different tasks route through different masks with
  zero weight mutation.

## Tests

`python3 -m pytest tests/ -q` — 15 tests: progressive no-forgetting + frozen
priors + lateral presence, distillation retention + compression, DEN expansion
on hard tasks / no growth when easy, PackNet exact preservation + disjoint masks
+ both-tasks-learn, and edge-popup fitting frozen weights + distinct per-task
masks + STE gradient routing.

## Not locally actionable

177.6.1.2 (integrate progressive columns with the Galatea VLA, Phase 176.7.1)
requires that VLA policy and is left unchecked; the column mechanism it would use
is complete here.