Pith. sign in

REVIEW 3 major objections 6 minor 69 references

EDELINE: Enhancing Memory in Diffusion-based World Models via Linear-Time Sequence Modeling

T0 review · 3 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read EDELINE claims that a Mamba state-space recurrent embedding removes the fixed-context memory limit of diffusion world models, giving 1.87x human-normalized Atari 100k scores, 11.5 average return on Crafter, and superior ViZDoom visual…

desk verdict Strong architecture and results, but the long-memory claim is not actually tested—training and imagination both cap the effective context at roughly 19 steps. read the letter →

arxiv 2502.00466 v2 pith:QJ5CJU6B submitted 2025-02-01 cs.LG

Jia-Hua Lee , Bor-Jiun Lin , Wei-Fang Sun , Chun-Yi Lee This is my paper · ORCID
classification cs.LG
keywords worldmodelsdiffusionstatespaceMambamodel-basedreinforcementlearninglong-termmemoryAtari100ksampleefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the fixed-context memory limit of diffusion-based world models can be removed by feeding observation-action histories through a Mamba state space model before diffusing the next frame. If true, a single unified architecture could deliver both the visual fidelity of diffusion and long-horizon memory, which matters for partially observable environments where agents must recall events far in the past. EDELINE reports a 1.87x human-normalized mean score on Atari 100k, the best among model-based methods without look-ahead search, an 11.5 average return on Crafter, and strong visual consistency in ViZDoom. A sympathetic reader would therefore take the paper's central claim to be that memory, not just generative quality, is what unlocks sample-efficient model-based reinforcement learning.

What carries the argument

The load-bearing object is the Recurrent Embedding Module (REM), a Mamba selective state space model that maps the unbounded sequence of frames and actions to a hidden state $h_t = f_\phi(h_{t-1}, o_t, a_t)$ in linear time. That hidden state is the single conditioning signal passed to a U-Net-like Next-Frame Predictor through adaptive group normalization and cross-attention, and to the reward and termination predictors implemented as linear heads. The predictor is trained with a denoising objective $L_{obs}(\phi) = \|\hat{o}_j^0 - o_j^0\|^2$ at one randomly sampled target timestep per trajectory segment, using Mamba's parallel scan, while reward and termination losses are averaged over the segment; the total loss is $L(\phi) = w_o L_{obs}(\phi) + w_r L_{rew}(\phi) + L_{end}(\phi) + \log(w_o^{-1}) + \log(w_r^{-1})$, with harmonizers $w_o, w_r$ learned from HarmonyDream.

What would settle it

Run EDELINE on Crafter or MiniGrid-MemoryS9 with a training variant that carries the hidden state across full episodes instead of resetting it per segment; if the reset-trained version already matches the carried-state version, the memory claim is supported, but if the carried-state version scores meaningfully higher, the reported gains rest on an untested training loophole. A second check is to measure imagination prediction error, such as pixel MSE, as a function of imagined horizon length beyond the segment length used in training.

Watch

Extended reading notes

Core claim

EDELINE's central claim is that conditioning a diffusion next-frame predictor on a recurrent embedding computed by Mamba, rather than on a fixed window of four frames, lets a world model keep and use long-term context, and that this memory is the main driver of its performance. The architecture computes a hidden state $h_t = f_\phi(h_{t-1}, o_t, a_t)$ over the full observation-action history, conditions observation generation, reward prediction, and termination prediction all on this single shared representation, and balances the observation and reward losses with learnable harmonizers. On the Atari 100k benchmark the method reports a mean human-normalized score of 1.87, a median of 0.82, and an IQM of 0.94, surpassing all model-based baselines that do not use look-ahead search; on Crafter it reports 11.5 average return, a 4.1x improvement over DIAMOND at comparable parameter count. The paper's own argument is that these gains come from the removal of the fixed-context bottleneck while keeping diffusion's visual fidelity.

Load-bearing premise

The world model is trained on short random segments with the Mamba memory reset at the start of each segment, while at imagination time the memory is carried across the whole episode, and the paper does not test whether this mismatch weakens long-term memory.

Editorial extensions

If this is right

  • Diffusion world models no longer need a fixed observation window: any history length is representable in the Mamba hidden state, so long-horizon imagination can stay consistent.
  • Reward, termination, and observation prediction share one representation, which the paper argues improves learning efficiency and lets the model track reward-relevant visual details.
  • Training cost stays comparable to DIAMOND because Mamba's parallel scan and single-timestep observation loss keep the per-update cost low, despite the recurrent memory.
  • On memory-demanding benchmarks (Crafter, MiniGrid Memory, ViZDoom) the margin over DIAMOND grows with the need for long-term context, suggesting memory is the binding constraint being removed.
  • The reported Atari 100k numbers put EDELINE ahead of prior model-based methods that do not use look-ahead search, on mean, median, and IQM human-normalized scores.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The training-inference gap is the natural place to probe the claim: Algorithm 1 initializes the Mamba hidden state at the start of each sampled segment (presumably to zero), yet imagination carries state continuously; a test comparing this against carried-over states on Crafter or MiniGrid would settle whether the memory claim holds.
  • The same integration pattern, an SSM encoder feeding a diffusion decoder, could transfer to other generative sequence tasks like video prediction or world models for autonomous driving, where long context and visual fidelity are both needed.
  • The memory gain is attributed to Mamba specifically, but the architecture only requires a linear-time recurrent encoder; other selective SSMs or linear attention variants might replicate the result, which the paper does not test.
  • The 4.1x Crafter improvement over DIAMOND at comparable parameter count suggests that memory, not scale, is the main differentiator; a small memory-enhanced model may rival much larger latent models, which would matter for deployed agents with limited compute.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces EDELINE, a diffusion-based world model that replaces DIAMOND's fixed four-frame conditioning context with a Mamba state space model as a recurrent embedding module (REM). The world model also unifies reward and termination prediction on top of the REM hidden state and adopts HarmonyDream-style dynamic loss harmonization. The authors report a 1.87 human-normalized mean score on Atari 100k, a 4.1x improvement over DIAMOND on Crafter with a comparable parameter count, and improved generation consistency on ViZDoom and MiniGrid memory tasks. The central claim is that integrating an SSM into a diffusion world model provides long-term memory beyond fixed-context limitations while preserving visual fidelity.

Significance. If the claims hold, this is a strong empirical contribution: EDELINE is evaluated on standard benchmarks (Atari 100k, Crafter, ViZDoom, MiniGrid) with multiple ablations, bootstrap confidence intervals for aggregate Atari metrics, per-environment training curves, and a computational cost comparison against DIAMOND. The reported gains over strong baselines like DreamerV3 and DIAMOND are substantial, and the parameter efficiency on Crafter is notable. The main risk is that the signature 'memory enhancement' claim is not isolated experimentally: the training procedure and the imagination loop both cap the effective context at roughly 19 steps, so the unbounded-memory mechanism is untested, and the only full-benchmark ablation removes the harmonizer rather than the memory module.

major comments (3)
  1. [§5.1.4, Algorithm 1] The training procedure in update_world_model samples random length-T segments (T=19, Table 9) and initializes the Mamba hidden state h_{t-1} without any specification of a value or carry-over from a full episode; the observation reconstruction loss is evaluated at a single target j in {t+B, ..., t+T-1}, so gradients never train the recurrent state to encode information older than about T-1 = 18 steps. In update_actor_critic, the imagination loop samples a B=4-frame buffer, burns in Mamba state from that buffer, and rolls out H=15 steps, giving an effective context of B+H = 19 steps and discarding all earlier episode context. Consequently the paper's central claims that EDELINE 'processes unbounded observation-action sequences' (Abstract), 'overcomes the fixed context limitations' (§1), and 'effectively integrates information from the entire history' (§6.3) are not supported by any training signal or evaluation regime that exercises longer histories; the reported gains over DIAMOND could be driven by a 19-step recurrence plus loss harmonization rather than by long-term memory. Please add an experiment that trains with hidden state carried over contiguous full-episode segments or otherwise provides a gradient path for memory beyond T, and/or restrict the claims to 'longer context'; also state explicitly what h_{t-1} is initialized to in Algorithm 1.
  2. [Table 7] The only full-benchmark ablation removes the harmonizers (EDELINE w/o Harmonizers mean HNS 1.674 vs 1.866 with harmonizers), not the memory module; the REM architecture ablation in §D.9.1 compares Mamba to GRU and LSTM but does not compare against a non-recurrent fixed-window conditioning model, and Table 8 compares Mamba to a Transformer-based REM rather than to DIAMOND's fixed context. To support the attribution of the 1.87 HNS and the 4.1x Crafter improvement to 'memory enhancement', please add an ablation that removes or truncates the REM (e.g., replacing Mamba with DIAMOND-style last-four-frames conditioning while keeping the harmonizer and unified predictors) or that varies T and H. Without such an ablation, the contribution of the memory mechanism to the headline results is not isolated.
  3. [Appendix D.6, Table 3] The linear probing comparison uses EDELINE's Mamba hidden state, which is the direct conditioning input to the diffusion next-frame predictor, and DIAMOND's LSTM hidden state from its auxiliary reward/termination network, which is not used to condition observation prediction. The reported 57.3% reduction in observation loss may therefore reflect that only EDELINE's probed representation was trained by the observation reconstruction loss, rather than a genuine superiority of the unified hidden representation. Please probe a matched representation (e.g., DIAMOND's four-frame stack, or an LSTM state trained with an observation reconstruction loss) or re-frame the representation-sharing claim accordingly.
minor comments (6)
  1. [Section 5] The text says 'as illustrated in Fig. 5' but the architecture figure is Figure 2; there is no Figure 5 in the main text.
  2. [Appendix B] The heading 'Boarder Impact' should be 'Broader Impact'.
  3. [Algorithm 1] The line 'Initialize h_{t-1}' should specify that it is set to zero (or describe any other reset scheme), because the initialization is important for evaluating the train/inference mismatch discussed above.
  4. [Throughout] The spelling of the environment and model names is inconsistent: 'ViZDoom' and 'VizDoom', 'MAMBA' and 'Mamba' are used interchangeably; please standardize.
  5. [Table 1] Per-game scores are reported as means over three seeds without error bars; the bootstrap confidence intervals in Figure 3 are welcome, but game-specific variance would improve the table.
  6. [Section 6.3] The ViZDoom comparison includes only DIAMOND as a baseline; adding DreamerV3 or a model-free method would make the claim of superior performance in 3D first-person environments more robust.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: EDELINE's claims are externally benchmarked and no fitted parameter is relabeled as a prediction.

full rationale

EDELINE is an empirical systems paper whose load-bearing claims are measured against external environments (Atari 100k, Crafter, ViZDoom, MiniGrid) rather than derived from fitted constants or from the paper's own assumptions. The only author self-citation appears as part of a background list on score-based generative models in Appendix C.1 ([21, 61, 62], with [62] sharing two authors); it is not used to justify any architectural choice, benchmark result, or so-called uniqueness claim, so it is not load-bearing. The harmonizers w_o and w_r in Eq. (5) are adaptive weights learned through HarmonyDream's variational objective, not parameters tuned to test scores, and no equation in Section 5 or Algorithm 1 reduces a reported result to a fitted input. The train/inference memory mismatch noted in the reader's take (zero-initialized fixed-length segments during world-model training versus continuous hidden states in imagination) is a substantive validity and robustness concern, but it is not circularity: the paper does not define memory performance in terms of Eq. (4) or Algorithm 1. Appendix A's explicit limitation concerns diffusion computational overhead rather than any circular dependency. No specific circular step can be exhibited by quoting an equation that equals its own input, so the appropriate finding is no significant circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim depends on standard MBRL and diffusion assumptions, several hand-chosen hyperparameters, and one ad hoc training choice (zero-initialized Mamba state on random segments) that is not validated. No new physical entities are introduced.

free parameters (5)
  • Sequence length T = 19
    Sampled trajectory segment length for world model training; balances memory and compute.
  • Burn-in length B = 4
    Steps skipped before sampling the target observation loss, chosen to let the Mamba state warm up.
  • Number of denoising steps = 3
    Diffusion sampling steps during imagination; directly affects world model fidelity and speed.
  • Mamba state dimension d_state = 16
    State size of the selective SSM; determines memory capacity.
  • Number of conditioning observations L = 4
    Number of raw frames given to the diffusion model; inherited from DIAMOND.
assumptions (5)
  • domain assumption The RL problem is a POMDP with image observations, as formulated in Section 3.1.
    The paper models the environment as a Partially Observable Markov Decision Process, which is standard for image-based RL.
  • domain assumption The Mamba selective SSM is a sufficiently expressive sequence model to encode the full history into a hidden state.
    Section 3.2 and 5.1.1 rely on Mamba's capacity for long-term dependencies; this is central to the memory claim.
  • domain assumption The diffusion model can learn the conditional next-frame distribution p(o_{t+1}|h_t, last L frames).
    Section 5.1.2 relies on score matching and EDM preconditioning to model this conditional distribution.
  • domain assumption Imagination with the learned world model transfers to the real environment for policy learning.
    Standard model-based RL assumption, invoked in Section 5.2 and Algorithm 1.
  • ad hoc to paper Zero-initialized Mamba hidden states on random training segments are sufficient for learning long-term memory.
    Algorithm 1 initializes h_{t-1} for a random segment without specifying the true previous state; this train/inference mismatch is not validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EDELINE: Enhancing Memory in Diffusion-based World Models via Linear-Time Sequence Modeling." pith.science (2026). https://pith.science/paper/QJ5CJU6B

@misc{pith2026250200466,
  author       = {Pith},
  title        = {Pith review of: EDELINE: Enhancing Memory in Diffusion-based World Models via Linear-Time Sequence Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QJ5CJU6B}},
  note         = {Machine review of arXiv:2502.00466}
}
read the original abstract

World models represent a promising approach for training reinforcement learning agents with significantly improved sample efficiency. While most world model methods primarily rely on sequences of discrete latent variables to model environment dynamics, this compression often neglects critical visual details essential for reinforcement learning. Recent diffusion-based world models condition generation on a fixed context length of frames to predict the next observation, using separate recurrent neural networks to model rewards and termination signals. Although this architecture effectively enhances visual fidelity, the fixed context length approach inherently limits memory capacity. In this paper, we introduce EDELINE, a unified world model architecture that integrates state space models with diffusion models. Our approach outperforms existing baselines across visually challenging Atari 100k tasks, memory-demanding Crafter benchmark, and 3D first-person ViZDoom environments, demonstrating superior performance in all these diverse challenges.

Figures

Figures reproduced from arXiv: 2502.00466 by the authors.

Figure 1
Figure 1. Motivational examples for both qualita￾tive and quantitative evidences to demonstrate that DIAMOND face difficulties in imagining accurate future under memorization tasks. To substantiate the memory limitations of the DIAMOND model, we conducted experiments using the MiniGrid MemoryS7 and MemoryS9 environments [16]. These experiments evaluate memory consistency and temporal prediction capabilities in world models [… view at source ↗
Figure 2
Figure 2. The EDELINE world model includes three principal components: (1) A U-Net-like [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. Training curves comparing EDELINE (blue) and DIAMOND (green) across five ViZDoom [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figures from the paper (9 more)
Figure 5
Figure 5. Figure 5: Qualitative comparison on the DeadlyCorridor scenario. Each row shows ground truth, [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Training curves of EDELINE on the Atari100k benchmark for individual games (400K [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparison of world model predictions on three Atari games. Each panel [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Additional performance analyses. (a) Performance profiles showing fraction of runs achiev [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Comparison of state-of-the-art model-based RL methods without using look-ahead search techniques on the Atari 100k benchmark. EDELINE outperforms all existing model-based approaches on the Atari 100k benchmark. Previous methods can be categorized by their world model a…
Figure 10
Figure 10. Figure 10: Performance comparison of different linear-time sequence models as REM architecture [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]
Figure 11
Figure 11. Figure 11: Ablation study comparing EDELINE with cross-attention blocks ( [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]
Figure 12
Figure 12. Figure 12: Effect of harmonizers on world model predictions. (a) Without harmonizers, EDELINE [PITH_FULL_IMAGE:figures/full_fig_p024_12.png]
Figure 13
Figure 13. Figure 13: Qualitative comparison of world model predictions in Crafter. We compare GT (Ground [PITH_FULL_IMAGE:figures/full_fig_p027_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 55 canonical work pages

  1. [1]

    Recurrent world models facilitate policy evolution

    David Ha and Jürgen Schmidhuber. Recurrent world models facilitate policy evolution. InProc. Conf. on Neural Information Processing Systems (NeurIPS), 2018

    work page 2018

  2. [2]

    Mastering diverse domains through world models

    Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv:2301.04104, 2024

    arXiv 2024

  3. [3]

    Mastering atari, go, chess and shogi by planning with a learned model.Nature, 2020

    Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, and et al. Mastering atari, go, chess and shogi by planning with a learned model.Nature, 2020

    work page 2020

  4. [4]

    Mastering atari games with limited data

    Weirui Ye, Shaohuai Liu, Thanard Kurutach, Pieter Abbeel, and Yang Gao. Mastering atari games with limited data. InProc. Conf. on Neural Information Processing Systems (NeurIPS), 2021

    work page 2021

  5. [5]

    Day- dreamer: World models for physical robot learning

    Philipp Wu, Alejandro Escontrela, Danijar Hafner, Pieter Abbeel, and Ken Goldberg. Day- dreamer: World models for physical robot learning. InProc. Conf. on Annual Conference on Robot Learning (CoRL), 2022

    work page 2022

  6. [6]

    Dream to control: Learning behaviors by latent imagination

    Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination. InInternational Conference on Learning Representa- tions (ICLR), 2020

    2020

  7. [7]

    Mastering atari with discrete world models

    Danijar Hafner, Timothy P Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models. InInternational Conference on Learning Representations (ICLR), 2021

    work page 2021

  8. [8]

    Diffu- sion for world modeling: Visual details matter in atari

    Eloi Alonso, Adam Jelley, Vincent Micheli, Anssi Kanervisto, Amos Storkey, and et al. Diffu- sion for world modeling: Visual details matter in atari. InProc. Conf. on Neural Information Processing Systems (NeurIPS), 2024

    work page 2024

Show all 69 references
  1. [9]

    Efficiently modeling long sequences with structured state spaces

    Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. InInt. Conf. on Learning Representations (ICLR), 2022

    2022

  2. [10]

    On the parameterization and initialization of diagonal state space models

    Albert Gu, Ankit Gupta, Karan Goel, and Christopher Ré. On the parameterization and initialization of diagonal state space models. InProc. Conf. on Neural Information Processing Systems (NeurIPS), 2022

    2022

  3. [11]

    Smith, Andrew Warrington, and Scott Linderman

    Jimmy T.H. Smith, Andrew Warrington, and Scott Linderman. Simplified state space layers for sequence modeling. InInt. Conf. on Learning Representations (ICLR), 2023

    2023

  4. [12]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. In Proc. Int. Conf. on Language Modeling (CoLM), 2024

    2024

  5. [13]

    Transformers are ssms: Generalized models and efficient algorithms through structured state space duality

    Albert Gu and Tri Dao. Transformers are ssms: Generalized models and efficient algorithms through structured state space duality. InProc. Int. Conf. on Machine Learning (ICML), 2024

    2024

  6. [14]

    Mas- tering memory tasks with world models

    Mohammad Reza Samsami, Artem Zholus, Janarthanan Rajendran, and Sarath Chandar. Mas- tering memory tasks with world models. InThe Twelfth International Conference on Learning Representations (ICLR), 2024

    2024

  7. [15]

    Model based reinforcement learning for atari

    Łukasz Kaiser, Mohammad Babaeizadeh, Piotr Miłos, Bła˙zej Osi´nski, Roy H Campbell, and et al. Model based reinforcement learning for atari. InInternational Conference on Learning Representations (ICLR), 2020

    2020

  8. [16]

    Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks

    Maxime Chevalier-Boisvert, Bolun Dai, Mark Towers, Rodrigo De Lazcano Perez-Vicente, Lucas Willems, and et al. Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks. InProc. Conf. on Neural Information Processing Systems (Neur...

    2023

  9. [17]

    Benchmarking the spectrum of agent capabilities

    Danijar Hafner. Benchmarking the spectrum of agent capabilities. InInternational Conference on Learning Representations, 2022

    2022

  10. [18]

    Viz- doom: A doom-based ai research platform for visual reinforcement learning

    Michał Kempka, Marek Wydmuch, Grzegorz Runc, Jakub Toczek, and Wojciech Ja´skowski. Viz- doom: A doom-based ai research platform for visual reinforcement learning. arXiv:1605.02097, 2016. 10

    2016 arXiv

  11. [19]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  12. [20]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations (ICLR), 2021

    2021

  13. [21]

    Generative modeling by estimating gradients of the data distribution

    Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. InProc. Conf. on Neural Information Processing Systems (NeurIPS), 2019

    2019

  14. [22]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and et al. Score-based generative modeling through stochastic differential equations. InInternational Conference on Learning Representations (ICLR), 2021

    2021

  15. [23]

    Tenenbaum, Sander Dieleman, and et al

    Yilun Du, Conor Durkan, Robin Strudel, Joshua B. Tenenbaum, Sander Dieleman, and et al. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and mcmc. InProc. Int. Conf. on Machine Learning (ICML), 2023

    2023

  16. [24]

    Diffusion policies as an expressive policy class for offline reinforcement learning

    Wang Zhendong, Hunt Jonathan J, and Zhou Mingyuan. Diffusion policies as an expressive policy class for offline reinforcement learning. InInt. Conf. on Learning Representations (ICLR), 2023

    2023

  17. [25]

    Tenenbaum Joshua, S

    Ajay Anurag, Du Yilun, Gupta Abhi, B. Tenenbaum Joshua, S. Jaakkola Tommi, and et al. Is conditional generative modeling all you need for decision making? InInt. Conf. on Learning Representations (ICLR), 2023

    2023

  18. [26]

    Imitating human behaviour with diffusion models

    Pearce Tim, Rashid Tabish, Kanervisto Anssi, Bignell Dave, Sun Mingfei, and et al. Imitating human behaviour with diffusion models. InInt. Conf. on Learning Representations (ICLR), 2023

    2023

  19. [27]

    Tenenbaum Joshua, and Levine Sergey

    Janner Michael, Du Yilun, B. Tenenbaum Joshua, and Levine Sergey. Planning with diffusion for flexible behavior synthesis. InProc. Int. Conf. on Machine Learning (ICML), 2023

    2023

  20. [28]

    Adaptd- iffuser: Diffusion models as adaptive self-evolving planners

    Liang Zhixuan, Mu Yao, Ding Mingyu, Ni Fei, Tomizuka Masayoshi, and Luo Ping. Adaptd- iffuser: Diffusion models as adaptive self-evolving planners. InProc. Int. Conf. on Machine Learning (ICML), 2023

    2023

  21. [29]

    Extracting reward functions from diffusion models

    Nuti Felipe Pinto Coelho, Franzmeyer Tim, and Henriques Joao F. Extracting reward functions from diffusion models. InProc. Conf. on Neural Information Processing Systems (NeurIPS), 2023

    2023

  22. [30]

    Metadiffuser: Diffusion model as conditional planner for offline meta-rl

    Fei Ni, Jianye Hao, Yao Mu, Yifu Yuan, Yan Zheng, and et al. Metadiffuser: Diffusion model as conditional planner for offline meta-rl. InProc. Int. Conf. on Machine Learning (ICML), 2023

    2023

  23. [31]

    Synthetic experience replay

    Lu Cong, Ball Philip J., Teh Yee Whye, and Parker-Holder Jack. Synthetic experience replay. InProc. Conf. on Neural Information Processing Systems (NeurIPS), 2023

    2023

  24. [32]

    Transformer-based world models are happy with 100k interactions

    Jan Robine, Marc Höftmann, Tobias Uelwer, and Stefan Harmeling. Transformer-based world models are happy with 100k interactions. InThe Eleventh International Conference on Learning Representations (ICLR), 2023

    2023

  25. [33]

    STORM: Efficient stochas- tic transformer based world models for reinforcement learning

    Weipu Zhang, Gang Wang, Jian Sun, Yetian Yuan, and Gao Huang. STORM: Efficient stochas- tic transformer based world models for reinforcement learning. InProc. Conf. on Neural Information Processing Systems (NeurIPS), 2023

    2023

  26. [34]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. V on Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors,Advances in Neural ...

    2017

  27. [35]

    Transformers are sample-efficient world models

    Vincent Micheli, Eloi Alonso, and François Fleuret. Transformers are sample-efficient world models. InThe Eleventh International Conference on Learning Representations (ICLR), 2023

    2023

  28. [36]

    Learning to Simulate Dynamic Environments with GameGAN

    Seung Wook Kim, Yuhao Zhou, Jonah Philion, Antonio Torralba, and Sanja Fidler. Learning to Simulate Dynamic Environments with GameGAN. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 11

    2020

  29. [37]

    Chan, Nicolas Heess, Lucy Gonzalez, Simon Osindero, Sherjil Ozair, Scott Reed, Jingwei Zhang, Konrad Zolna, Jeff Clune, Nando de Freitas, Satinder Singh, and Tim Rocktäschel

    Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Maria Elis- abeth Bechtle, Feryal Behbahani, Stephanie C.Y . Chan, Nicolas Heess, Lucy Gonzalez, Simon O...

    2024

  30. [38]

    Diffusion models are real-time game engines

    Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. Diffusion models are real-time game engines. arXiv:2408.14837, 2024

    2024 arXiv

  31. [39]

    Gaia-1: A generative world model for autonomous driving, 2023

    Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and Gianluca Corrado. Gaia-1: A generative world model for autonomous driving, 2023

    2023

  32. [40]

    Gaia-2: A controllable multi-view generative world model for autonomous driving, 2025

    Lloyd Russell, Anthony Hu, Lorenzo Bertoni, George Fedoseev, Jamie Shotton, Elahe Arani, and Gianluca Corrado. Gaia-2: A controllable multi-view generative world model for autonomous driving, 2025

    2025

  33. [41]

    Xing, and Zhiting Hu

    Jiannan Xiang, Guangyi Liu, Yi Gu, Qiyue Gao, Yuting Ning, Yuheng Zha, Zeyu Feng, Tianhua Tao, Shibo Hao, Yemin Shi, Zhengzhong Liu, Eric P. Xing, and Zhiting Hu. Pandora: Towards general world model with natural language actions and video states, 2024

    2024

  34. [42]

    Hippo: Recurrent memory with optimal polynomial projections

    Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christopher Ré. Hippo: Recurrent memory with optimal polynomial projections. InProc. Conf. on Neural Information Processing Systems (NeurIPS), 2020

    2020

  35. [43]

    Diagonal state spaces are as effective as structured state spaces

    Ankit Gupta, Albert Gu, and Jonathan Berant. Diagonal state spaces are as effective as structured state spaces. InProc. Conf. on Neural Information Processing Systems (NeurIPS), 2022

    2022

  36. [44]

    Liquid structural state-space models

    Hasani Ramin, Lechner Mathias, Wang Tsun-Hsuan, Chahine Makram, Amini Alexander, and et al. Liquid structural state-space models. InInt. Conf. on Learning Representations (ICLR), 2023

    2023

  37. [45]

    Structured state space models for in-context reinforcement learning

    Lu Chris, Schroecker Yannick, Gu Albert, Parisotto Emilio, Nicolaus Foerster Jakob, and et al. Structured state space models for in-context reinforcement learning. InProc. Conf. on Neural Information Processing Systems (NeurIPS), 2023

    2023

  38. [46]

    Decision mamba: Reinforcement learning via hybrid selective sequence modeling

    Sili Huang, Jifeng Hu, Zhejian Yang, Liwei Yang, Tao Luo, and et al. Decision mamba: Reinforcement learning via hybrid selective sequence modeling. InProc. Conf. on Neural Information Processing Systems (NeurIPS), 2024

    2024

  39. [47]

    Decision trans- former: Reinforcement learning via sequence modeling

    Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, and et al. Decision trans- former: Reinforcement learning via sequence modeling. InProc. Conf. on Neural Information Processing Systems (NeurIPS), 2021

    2021

  40. [48]

    Drama: Mamba-enabled model-based reinforcement learning is sample and parameter efficient

    Wang Wenlong, Dusparic Ivana, Shi Yucheng, Zhang Ke, and Cahill Vinny. Drama: Mamba-enabled model-based reinforcement learning is sample and parameter efficient. arXiv:2410.08893, 2024

    2024 arXiv

  41. [49]

    Optimal control of markov processes with incomplete state information

    Karl Johan Åström. Optimal control of markov processes with incomplete state information. Journal of Mathematical Analysis and Applications, 10:174–205, 1965

    1965

  42. [50]

    Richard S. Sutton. Learning to predict by the methods of temporal differences.Machine Learning, 3:9–44, 1988

    1988

  43. [51]

    Improving the closed-loop tracking performance using the first-order hold sensing technique with experiments

    Yang Chifu, Gao Shuang, and Xue Zhu. Improving the closed-loop tracking performance using the first-order hold sensing technique with experiments. arXiv:1801.01263, 2018

    2018 arXiv

  44. [52]

    Hochreiter and J

    S. Hochreiter and J. Schmidhuber. Long short-term memory.Neural Computation, 9:1735–1780, 1997

    1997

  45. [53]

    Chung, C

    J. Chung, C. Gulcehre, K. Cho, and Y . Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. InProc. Conf. on Neural Information Processing Systems Workshop (NeurIPSW), 2014. 12

    2014

  46. [54]

    Learning semantic- aware normalization for generative adversarial networks

    Heliang Zheng, Jianlong Fu, Yanhong Zeng, Jiebo Luo, and Zheng-Jun Zha. Learning semantic- aware normalization for generative adversarial networks. InAdvances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  47. [55]

    Harmonydream: Task harmonization inside world models

    Haoyu Ma, Jialong Wu, Ningya Feng, Chenjun Xiao, Dong Li, Jianye HAO, Jianmin Wang, and Mingsheng Long. Harmonydream: Task harmonization inside world models. InForty-first International Conference on Machine Learning (ICML), 2024

    2024

  48. [56]

    Dueling network architectures for deep reinforcement learning

    Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Hasselt, Marc Lanctot, and Nando Freitas. Dueling network architectures for deep reinforcement learning. InProceedings of The 33rd International Conference on Machine Learning (ICML), 2016

    2016

  49. [57]

    Deep reinforcement learning at the edge of the statistical precipice.Advances in Neural Information Processing Systems (NeurIPS), 2021

    Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron C Courville, and Marc Belle- mare. Deep reinforcement learning at the edge of the statistical precipice.Advances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  50. [58]

    Towards efficient world models

    Eloi Alonso, Vincent Micheli, and François Fleuret. Towards efficient world models. In Workshop on Efficient Systems for Foundation Models @ ICML2023, 2023

    2023

  51. [59]

    Deep unsuper- vised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. InInternational conference on machine learning, pages 2256–2265. PMLR, 2015

    2015

  52. [60]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021

    2021

  53. [61]

    Improved techniques for training score-based generative models

    Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. Advances in neural information processing systems, 33:12438–12448, 2020

    2020

  54. [62]

    Denoising likelihood score matching for con- ditional score-based data generation

    Chen-Hao Chao, Wei-Fang Sun, Bo-Wun Cheng, Yi-Chen Lo, Chia-Che Chang, Yu-Lun Liu, Yu-Lin Chang, Chia-Ping Chen, and Chun-Yi Lee. Denoising likelihood score matching for con- ditional score-based data generation. InInternational Conference on Learning Representations, 2022

    2022

  55. [63]

    Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35:26565–26577, 2022

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35:26565–26577, 2022

    2022

  56. [64]

    Multitask learning.Machine Learning, 28:41–75, 1997

    Rich Caruana. Multitask learning.Machine Learning, 28:41–75, 1997

    1997

  57. [65]

    Improving token-based world models with parallel observation prediction

    Lior Cohen, Kaixin Wang, Bingyi Kang, and Shie Mannor. Improving token-based world models with parallel observation prediction. InForty-first International Conference on Machine Learning (ICML), 2024

    2024

  58. [66]

    Learning transformer-based world models with contrastive predictive coding

    Anonymous. Learning transformer-based world models with contrastive predictive coding

  59. [67]

    Parallelizing model-based reinforce- ment learning over the sequence length

    ZiRui Wang, DENG Yue, Junfeng Long, and Yin Zhang. Parallelizing model-based reinforce- ment learning over the sequence length. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems (NeurIPS), 2024

    2024

  60. [68]

    Sutton and Andrew G

    Richard S. Sutton and Andrew G. Barto.Reinforcement Learning: An Introduction. The MIT Press, 2018. 13 A Limitation Although EDELINE maintains computational efficiency equivalent to that of DIAMOND and si- multaneously addresses its memory limitations, the fundamental computat...

    2018

  61. [2025]

    Under review at the International Conference on Learning Representations

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.