Eve library

phase-89.md

Tracker

Revision 1

On this page

Historical task source. The Eve task board holds current task state.

Phase 89: ML Sovereignty — Training & Fine-Tuning Infrastructure (SOTA April 2026)#

Integrate the latest training frameworks, PEFT methods, alignment techniques, optimizers, and scaling approaches as of April 2026.


89.1 Training Frameworks Integration#

89.1.1 Framework Setup#

  • 89.1.1.1 Integrate TRL v1.0 as reference post-training library (75+ methods: DPO, SimPO, KTO, ORPO, CPO, GRPO, DAPO, RLOO, REINFORCE++, GKD — stable API)
  • 89.1.1.2 Integrate Unsloth for single-GPU training (12x faster MoE, 7x longer context RL, 70% less VRAM, 150+ model support)
  • 89.1.1.3 Integrate Axolotl v0.8+ for production-scale training (multimodal fine-tuning, QAT, sequence parallelism, GRPO, YAML-driven config)
  • 89.1.1.4 Integrate torchtune for PyTorch-native training (DoRA support, QAT, PPO, PyTorch 2.5 compile for 20-24% speedup)
  • 89.1.1.5 Integrate LLaMA-Factory v0.9+ for accessible training (LlamaBoard web UI, OFT support, Megatron-LM integration)
  • 89.1.1.6 Build unified training launcher CLI (oshun-train --framework unsloth/axolotl/torchtune/trl --method lora/dpo/grpo --model --dataset → full lifecycle)

89.2 Advanced PEFT Methods#

89.2.1 Beyond LoRA#

  • 89.2.1.1 Implement DoRA training support (weight decomposition into magnitude + direction — no inference overhead, better convergence)
  • 89.2.1.2 Implement VeRA training support (shared frozen random matrices + learnable scaling vectors — dramatically fewer params)
  • 89.2.1.3 Implement Spectrum layer selection (SNR-based selection of highest signal-to-noise layers — compute-efficient)
  • 89.2.1.4 Implement QAT pipeline via Axolotl (5x perplexity improvement vs post-training quantization — critical for edge)
  • 89.2.1.5 Implement AdaLoRA with adaptive rank allocation per layer (SVD-based, automatic rank selection)
  • 89.2.1.6 Implement multi-LoRA composition training (LoRA Soups concatenation + optimal weighting — 43% improvement over merge baselines)

89.3 RLVR & Advanced Alignment Training#

89.3.1 Reinforcement Learning with Verifiable Rewards#

  • 89.3.1.1 Implement GRPO training pipeline (Group Relative PO: sample 8-64 responses → compute group advantages → policy gradient without critic — DeepSeek-R1 method)
  • 89.3.1.2 Implement DAPO training pipeline (Clip-Higher + Dynamic Sampling + Token-level PG Loss + Overlong Reward Shaping — 50% fewer steps than GRPO)
  • 89.3.1.3 Implement RLVR infrastructure (verifiable reward functions: math checker, code sandbox, proof verifier, format validator — replacing human labels for reasoning)
  • 89.3.1.4 Implement SimPO training (reference-free preference optimization using average log probability — +6.4 AlpacaEval 2 vs DPO)
  • 89.3.1.5 Implement Process Reward Model training (stepwise reward at every reasoning step — ThinkPRM verbalized verification chains)
  • 89.3.1.6 Implement self-play training pipeline (SPIRAL: zero-sum game self-play → reasoning transfer, +10% on 8 benchmarks)
  • 89.3.1.7 Implement online DPO/iterative DPO (generate new responses during training → train beyond offline dataset quality)
  • 89.3.1.8 Implement RainbowPO training (7 orthogonal DPO improvements combined — +28.74% LC Win Rate on AlpacaEval2)
  • 89.3.1.9 Implement KTO training for binary feedback (thumbs-up/down without pairwise comparisons, models loss aversion)
  • 89.3.1.10 Implement multimodal alignment training via MM-DPO (120K multimodal preference pairs, critique-based reward model)
  • 89.3.1.11 Integrate NeMo-RL as scalable RL training backend (NVIDIA's post-training library: Megatron-Core backend, native GRPO + DAPO support, multi-GPU/multi-node, NeMo Gym for multi-environment RL, FP8 training, HuggingFace compatibility)
  • 89.3.1.12 Build standalone reward model training pipeline (Bradley-Terry preference model from pairwise data, multi-dimensional reward models a la Nemotron, evaluation on RewardBench, deployment as reward API for synthetic data filtering and online RLHF)
  • 89.3.1.13 Build outcome reward model and process reward model training (ORM: final answer correctness scoring, PRM: per-step reasoning verification — both needed for RLVR, training from stepwise annotation data from 87.4.1.15)

89.4 Optimizers & Training Efficiency#

89.4.1 Modern Optimizers & Memory Reduction#

  • 89.4.1.1 Integrate Muon optimizer (Newton-Schulz orthogonalization — 2x efficiency vs AdamW, 50% cost reduction, 33% memory savings)
  • 89.4.1.2 Integrate GaLore 2 for memory-efficient training (gradient projection — 82.5% optimizer memory reduction, 63.3% total memory reduction)
  • 89.4.1.3 Integrate Liger Kernel (fused RMSNorm, RoPE, SwiGLU, CrossEntropy — 20% throughput increase, 60% memory reduction)
  • 89.4.1.4 Integrate FSDP2 via TorchTitan (per-parameter dim-0 sharding, 7% lower memory, deterministic, BF16/FP32 mixed precision)
  • 89.4.1.5 Integrate DeepSpeed ZeRO++ (4x communication reduction, 50%+ training time reduction at scale)
  • 89.4.1.6 Implement NeMo Megatron Bridge for bidirectional HuggingFace/Megatron format conversion
  • 89.4.1.7 Build training memory profiler (predict peak VRAM for model + batch + sequence + method → right-size GPU)
  • 89.4.1.8 Implement curriculum learning for training data scheduling (easy-to-hard progression, competence-aware adaptive scheduling, length-based online scheduling, configurable difficulty metrics per domain)
  • 89.4.1.9 Build parallelism strategy auto-configurator (given model size + cluster topology + memory budget → select optimal DDP/FSDP/FSDP2/DeepSpeed + tensor/pipeline parallelism degrees, with hybrid parallelism and communication-computation overlap tuning)
  • 89.4.1.10 Build mixed-precision training configuration system (BF16/FP32 default with FP8 for Hopper/Blackwell, Transformer Engine integration for automatic precision selection, loss scaling, master weight management, per-layer precision overrides)

89.4.2 Advanced Training Capabilities#

  • 89.4.2.1 Build continual pre-training pipeline for domain-adaptive pre-training (instruction synthesizer for raw corpora, catastrophic forgetting mitigation via replay buffers + elastic weight consolidation, configurable domain data mixing, checkpoint evaluation for forgetting detection)
  • 89.4.2.2 Build mid-training / knowledge injection pipeline (inject new domain knowledge between pre-training and fine-tuning: long-form document ingestion, knowledge distillation from domain corpora, evaluation-guided schedule)
  • 89.4.2.3 Build long-context training pipeline (extend model context window: YaRN RoPE extension, LongRoPE2 evolutionary search, RoPE base frequency adjustment, synthetic long-context data generation, progressive extension schedule)

89.5 MoE Training & Scaling#

89.5.1 Mixture of Experts#

  • 89.5.1.1 Implement MoE fine-tuning pipeline (fine-tune individual experts or routing for Llama 4 Scout/Maverick 16/128-expert architectures)
  • 89.5.1.2 Implement MoE expert parallelism via NeMo Megatron Core (distribute experts across GPUs with FP8 training)
  • 89.5.1.3 Build MoErges/FrankenMoE pipeline (upcycle dense fine-tuned models into MoE using mergekit-moe with learned router)
  • 89.5.1.4 Implement selective expert fine-tuning (freeze base experts, fine-tune only domain-relevant experts)

89.6 Model Merging & Composition#

89.6.1 Merge Methods#

  • 89.6.1.1 Integrate mergekit for automated model merging (SLERP, TIES, DARE, Task Arithmetic, Passthrough, MoErges — yaml-driven)
  • 89.6.1.2 Build automated merge quality evaluation (merge → benchmark → compare against sources → accept/reject)
  • 89.6.1.3 Implement multi-LoRA routing system (LoRA-Mixer: serial attention routing → domain-specific adapters per request)
  • 89.6.1.4 Build model merge search (try strategies × weight ratios → evaluate → select Pareto-optimal on quality/size)

89.7 Multi-Modal Fine-Tuning#

89.7.1 VLM & Multi-Modal Training#

  • 89.7.1.1 Implement multi-modal PEFT for VLM training (vision-language LoRA targeting both language and vision encoder layers, visual adapter tuning, frozen vision encoder + trainable connector patterns, multi-modal QLoRA for memory-efficient VLM fine-tuning)
  • 89.7.1.2 Build VLM instruction tuning pipeline (image-text instruction following, visual QA fine-tuning, chart/document understanding training, multi-image reasoning — supporting Qwen2.5-VL, LLaVA, InternVL architectures)

89.8 Scaling-Law Discipline & Compute-Optimal Training#

No checkpoint should be trained without first proving, on a small-scale IsoFLOP sweep, that its compute allocation lies on the compute-optimal frontier. These tasks make scaling-law experimentation a first-class part of every training run — not a post-hoc rationalisation.

89.8.1 Parametrisation & Transfer#

  • 89.8.1.1 Implement μTransfer (μP, Yang & Hu) in the training stack — maximum-update parametrisation for width and depth, verified by shape-invariant optimal learning-rate sweeps on 40M → 400M → 2B width scans; optimal HPs found at 40M must transfer zero-shot to 2B (loss within 1% of independently tuned baseline)
  • 89.8.1.2 Implement depth-μP extensions for deep Transformers (residual-block scaling) and verify across 12L → 48L → 96L sweeps
  • 89.8.1.3 Build scaling-law sweep harness — launch IsoFLOP grids across (N_params, D_tokens, batch, LR) parameterised by compute budget C = 6ND, fit parametric loss law L(N,D) = E + A/N^α + B/D^β, report compute-optimal (N*, D*) per architecture family with bootstrap CIs
  • 89.8.1.4 Reproduce Chinchilla-class optimum on a dense Transformer (expect D*/N* ≈ 20 tokens/param) and DeepSeek-V2-class MoE optimum (separate sparse-active vs total-params law) — publish sovereign scaling-law report per architecture

89.8.2 Loss Prediction & Emergence Bench#

  • 89.8.2.1 Build loss-prediction CI gate — every production training run must ship a fitted loss curve from ≤1/16 final compute predicting final loss within ±2%; runs that drift outside the band are auto-halted and escalated (prevents silent scaling-law violation à la GPT-4-early)
  • 89.8.2.2 Build downstream-capability emergence tracker — at each Chinchilla decile, probe a fixed eval suite (MMLU, GSM8K, HumanEval, MGSM, RewardBench, RULER long-context, SWE-Bench-Verified subset) and record the compute threshold at which each capability crosses chance / crosses SOTA, producing per-capability emergence curves
  • 89.8.2.3 Build data-quality scaling law — pair scaling sweeps with dataset-quality variants (web-only, web+code, web+code+math, distilled, deduplicated) to quantify quality vs quantity exchange rate (token-efficiency multipliers per recipe)
  • 89.8.2.4 Build MoE scaling-law experiments — joint scan over (active params, total params, n_experts, routing granularity) with RouterLM ablations (top-k, expert-choice, soft-MoE) producing compute-optimal (N_active, N_total) frontier
  • 89.8.2.5 Build multimodal scaling-law experiments extending N, D, and modality-mix ratio (text:image:audio:video) — guides Phase 96.1 model selection and our own future Oshun-native foundation-model plans

89.8.3 Scaling-Law Governance#

  • 89.8.3.1 Store every scaling-law experiment (configs, checkpoints, loss traces, fitted parameters) in the Phase 92 experiment store with immutable hashes — make scaling-law data reusable across domain training recipes
  • 89.8.3.2 Require every Phase 89 training recipe to declare target (N, D, C) with a pointer to the scaling-law study that justifies it — recipe linter fails CI if the declaration is missing or inconsistent with published laws
  • 89.8.3.3 Publish quarterly sovereign scaling-law bulletin (open artefact) so Oshun-internal recipes stay calibrated against the evolving external frontier (Chinchilla → Hoffmann → Llama-3 / DeepSeek V3 refits)

Completion evidence#

All 60 locally actionable items are implemented in libs/nous/training/src/phase-89-*.ts and the oshun-train CLI. phase-89-capability-catalog.ts maintains an exact one-to-one mapping from every checklist ID to its implementation and automated verification; its test reads this checklist and fails on missing, duplicate, or unexpected evidence.

Verification: nx test nous-training, nx lint nous-training, nx build nous-training, nx typecheck nous-training, and the hash-resolving nx recipe-lint nous-training CI gate.