Pith. sign in

REVIEW 16 cited by

Diffusion for World Modeling: Visual Details Matter in Atari

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.12399 v2 pith:XOY5FLSK submitted 2024-05-20 cs.LG cs.AIcs.CV

Eloi Alonso , Adam Jelley , Vincent Micheli , Anssi Kanervisto , Amos Storkey , Tim Pearce , François Fleuret This is my paper · ORCID
classification cs.LGcs.AIcs.CV
keywords worlddiffusionmodelmodelingmodelsagentsdetailsdiamond
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

World models constitute a promising approach for training reinforcement learning agents in a safe and sample-efficient manner. Recent world models predominantly operate on sequences of discrete latent variables to model environment dynamics. However, this compression into a compact discrete representation may ignore visual details that are important for reinforcement learning. Concurrently, diffusion models have become a dominant approach for image generation, challenging well-established methods modeling discrete latents. Motivated by this paradigm shift, we introduce DIAMOND (DIffusion As a Model Of eNvironment Dreams), a reinforcement learning agent trained in a diffusion world model. We analyze the key design choices that are required to make diffusion suitable for world modeling, and demonstrate how improved visual details can lead to improved agent performance. DIAMOND achieves a mean human normalized score of 1.46 on the competitive Atari 100k benchmark; a new best for agents trained entirely within a world model. We further demonstrate that DIAMOND's diffusion world model can stand alone as an interactive neural game engine by training on static Counter-Strike: Global Offensive gameplay. To foster future research on diffusion for world modeling, we release our code, agents, videos and playable world models at https://diamond-wm.github.io.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 16 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Heuristically Adaptive Diffusion-Model Evolutionary Strategy

    cs.NE 2024-11 conditional novelty 7.0 of 10

    An evolutionary algorithm that uses an online-trained diffusion model as its offspring generator can adapt to changing fitness landscapes and condition the search toward target traits without altering the fitness function.

  2. Population-Scalable Multi-Agent World Modeling

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Khora decouples world-state evolution from visual rendering through a shared STBoard and fixed-dimensional per-view renderers, enabling inference-time addition and removal of agents without retraining.

  3. Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills

    cs.RO 2026-08 conditional novelty 6.0 of 10

    A taxonomy of robot learning on a weights-versus-skills axis, with a five-rung self-improvement ladder whose top cell (feedback plus memory plus search) holds only a few recent systems.

  4. EchoWorld: Learning Motion-Aware World Models for Echocardiography Probe Guidance

    cs.CV 2025-04 conditional novelty 6.0 of 10

    A motion-aware world model pretraining approach reduces echocardiography probe guidance error relative to existing visual backbones and guidance frameworks on a private clinical dataset.

  5. Behavior-Regularized Diffusion Policy Optimization for Offline Reinforcement Learning

    cs.LG 2025-02 conditional novelty 6.0 of 10

    BDPO computes the behavior-regularization penalty for diffusion policies as a sum of per-denoising-step KL divergences and optimizes with a two-time-scale actor-critic, achieving strong D4RL performance.

  6. Learning Real-World Action-Video Dynamics with Heterogeneous Masked Autoregression

    cs.RO 2025-02 conditional novelty 6.0 of 10

    HMA is a masked autoregressive transformer that predicts future video and actions across many robot embodiments, running up to 15x faster than prior diffusion-based video simulators while matching or improving visual ...

  7. GLAM: Global-Local Variation Awareness in Mamba-based World Model

    cs.LG 2025-01 conditional novelty 6.0 of 10

    GLAM improves world model prediction in model-based RL by feeding state differences into two parallel Mamba modules and training agents on imagined variation-aware trajectories.

  8. The Matrix: Infinite-Horizon World Generation with Real-Time Moving Control

    cs.AI 2024-12 reject novelty 6.0 of 10

    A 2.7B parameter diffusion model trained on game and internet footage generates control-responsive 720p video streams, but the paper's 'infinite, real-time, zero-shot' claims are not backed by public benchmarks or rel...

  9. IQA-Adapter: Exploring Knowledge Transfer from Image Quality Assessment to Diffusion-based Generative Models

    cs.CV 2024-12 conditional novelty 6.0 of 10

    An adapter module trained on IQA/IAA scores gives SDXL controllable quality-aware generation, improving perceived quality and enabling reference-based distortion transfer.

  10. Quo Vadis, World Modeling?

    cs.CV 2026-08 conditional novelty 5.0 of 10

    An agent-centric reframing of world modeling, replacing physical state prediction with 'information transitions' organized into six proxy functions and three empowerment levels.

  11. Value Flows

    cs.LG 2025-10 reject novelty 5.0 of 10

    Value Flows fits the full return distribution in RL with a flow-matching critic and reweights its learning objective by estimated return variance; the central theoretical guarantee does not follow from the stated equations.

  12. Exploratory Diffusion Model for Unsupervised Reinforcement Learning

    cs.LG 2025-02 conditional novelty 5.0 of 10

    A diffusion-model denoising loss serves as an intrinsic reward to guide unsupervised RL exploration, plus an alternating fine-tuning scheme for diffusion policies.

  13. Pre-Trained Video Generative Models as World Simulators

    cs.CV 2025-02 conditional novelty 5.0 of 10

    A lightweight action-conditioning module and a motion-reinforced loss convert pre-trained video generators into action-following world simulators that also speed up model-based reinforcement learning.

  14. Humans Coexist, So Must Embodied Artificial Agents

    cs.LG 2025-02 conditional novelty 5.0 of 10

    Coexistence, defined as sustained meaningful and reciprocal interaction among an agent, humans, and environment, is presented as a necessary design goal for embodied AI.

  15. SimuDICE: Offline Policy Optimization Through World Model Updates and DICE Estimation

    cs.LG 2024-12 conditional novelty 5.0 of 10

    SimuDICE uses DualDICE weights and model confidence to re-sample synthetic transitions in a tabular world model, improving offline Dyna-Q style policy optimization in small discrete environments.

  16. Enhancing Decision Transformer with Diffusion-Based Trajectory Branch Generation

    cs.LG 2024-11 conditional novelty 5.0 of 10

    BG uses a value-guided diffusion model to generate trajectory branches that augment offline datasets, and the authors show this substantially improves Decision Transformer on maze and antmaze D4RL tasks.

Pith tools