Policy distillation & kickstarting (Phase 177.9), in PyTorch, CPU-runnable.
Components#
- Policy distillation (
policy_distillation.py, 177.9.1.1) — Rusu et al. Temperature-scaled KL distillation (Hinton,·T²) of a teacher policy into a student over shared rollouts, and joint distillation of several teachers / progressive columns into one deployable student (each on its own input region). - Kickstarting (
kickstarting.py, 177.9.1.2) — Schmitt et al. RL objective plus a decayed teacher-distillation auxiliary loss (λ_texponential/linear schedule with a floor) and a mixture-of-experts teacher (per-state gating over teacher policies) for bootstrapping a new robot from fleet teachers. - Data-free distillation (
data_free.py, 177.9.1.3) — DistBelief-style adversarial ZSKD: a generator maximises teacher/student disagreement (with an information-entropy diversity term and a known-input-normalisation moment prior to stay on-manifold) while the student minimises it, so the student matches the teacher on real held-out inputs it never saw (0.43 → 0.91 agreement). - Quality scorecard (
scorecard.py, 177.9.1.4) — per-task retention (student vs teacher accuracy), mean/min retention, and a ≥ 97 %-mean promotion gate.
Tests#
python3 -m pytest tests/ -q — 15 tests: zero-loss on identical logits +
student matching the teacher after distillation + multi-column joint
distillation; schedule decay (exp/linear/floor) + convex mixture teacher +
kickstart loss/train decay; data-free transfer to real held-out data + KL
trajectory; and scorecard retention/promotion pass & fail.