Eve library

nous-diffusion-world

Other

Revision 1

On this page

Diffusion-based world models (Phase 176.6), PyTorch, CPU-runnable with overfit-verified sampling.

Components#

  • EDM core (edm.py) — Karras preconditioning (c_skip/c_out/c_in/c_noise), log-normal noise sampling, EDM loss weighting, the ρ=7 σ-schedule, and a deterministic 2nd-order Heun sampler (DIAMOND runs it in 3 steps).
  • DIAMOND (diamond.py) — a convolutional FiLM denoiser predicting the next frame conditioned on past frames (concatenated channels) and the action (embedded), with autoregressive rollout. (176.6.1.1)
  • GameNGen (gamengen.py) — DIAMOND + context-frame noise augmentation: training corrupts the past frames at a random level and conditions the denoiser on it, the trick that keeps long interactive rollouts from drifting. (176.6.1.2)
  • Hybrid pipeline (hybrid.py) — cheap recurrent latent dynamics (Genie's long-range role) advanced every step + an EDM diffusion decoder rendering photorealistic frames every decode_every steps: the throughput ⇄ fidelity knob. (176.6.1.4)
  • Oasis open-domain model (oasis.py) — continuous multimodal-action and text-conditioned EDM dynamics with autoregressive history plus the specified consistency/FID/enjoyability comparison scorecard.

Not included (external — left unchecked in TODOS)#

The Minecraft-YouTube + Maya-alpha training and empirical comparison against a trained Genie-Sovereign-3 required by 176.6.1.3 need full datasets and accelerators; its locally actionable architecture and evaluation contract are implemented above.

Tests#

text
cd libs/nous/diffusion-world && python3 -m pytest tests/ -q

Verifies EDM preconditioning limits + schedule, that the denoiser reconstructs a constant target through sampling, that DIAMOND/GameNGen learn action-conditioned next frames (correct frame per action after training), and that the hybrid decoder reconstructs a frame from its latent while the rollout decodes only every K steps.