Diffusion-based world models (Phase 176.6), PyTorch, CPU-runnable with overfit-verified sampling.
Components#
- EDM core (
edm.py) — Karras preconditioning (c_skip/c_out/c_in/c_noise), log-normal noise sampling, EDM loss weighting, the ρ=7 σ-schedule, and a deterministic 2nd-order Heun sampler (DIAMOND runs it in 3 steps). - DIAMOND (
diamond.py) — a convolutional FiLM denoiser predicting the next frame conditioned on past frames (concatenated channels) and the action (embedded), with autoregressive rollout. (176.6.1.1) - GameNGen (
gamengen.py) — DIAMOND + context-frame noise augmentation: training corrupts the past frames at a random level and conditions the denoiser on it, the trick that keeps long interactive rollouts from drifting. (176.6.1.2) - Hybrid pipeline (
hybrid.py) — cheap recurrent latent dynamics (Genie's long-range role) advanced every step + an EDM diffusion decoder rendering photorealistic frames everydecode_everysteps: the throughput ⇄ fidelity knob. (176.6.1.4) - Oasis open-domain model (
oasis.py) — continuous multimodal-action and text-conditioned EDM dynamics with autoregressive history plus the specified consistency/FID/enjoyability comparison scorecard.
Not included (external — left unchecked in TODOS)#
The Minecraft-YouTube + Maya-alpha training and empirical comparison against a
trained Genie-Sovereign-3 required by 176.6.1.3 need full datasets and
accelerators; its locally actionable architecture and evaluation contract are
implemented above.
Tests#
text
cd libs/nous/diffusion-world && python3 -m pytest tests/ -q
Verifies EDM preconditioning limits + schedule, that the denoiser reconstructs a constant target through sampling, that DIAMOND/GameNGen learn action-conditioned next frames (correct frame per action after training), and that the hybrid decoder reconstructs a frame from its latent while the rollout decodes only every K steps.