Pith. sign in

REVIEW 1 major objections 1 minor 39 references

Training robot policies inside diffusion world models learned from real interaction data, using a decoupled gradient scheme, yields zero-shot transfer to real robots.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 03:59 UTC pith:MOHDSZFS

load-bearing objection Worth refereeing: a real attempt to train image-space RL policies inside diffusion world models, with encouraging real-robot results but a load-bearing assumption about local-model Jacobians that isn't tested. the 1 major comments →

arxiv 2602.06219 v2 pith:MOHDSZFS submitted 2026-02-05 cs.RO cs.AI

Coupled Local and Global World Models for Efficient First Order RL

Joseph Amigo , Rooholla Khorrambakht , Nicolas Mansard , Ludovic Righetti This is my paper
classification cs.RO cs.AI
keywords world modelsmodel-based reinforcement learningdiffusion modelsfirst-order gradientsdecoupled optimizationreal-robot learningmanipulationzero-shot transfer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that reinforcement learning can be done entirely inside world models learned from real robot interaction data, bypassing hand-built physics simulators. Its method, DMO-SAPO, couples a large diffusion-based world model that generates accurate forward image trajectories with a lightweight latent-space model that provides the gradients for policy updates. The authors report that this decoupled first-order gradient approach dramatically outperforms PPO in sample and time efficiency on two real-robot manipulation tasks, and transfers zero-shot to the real robot with high success.

Core claim

The central claim is that the bottleneck to first-order model-based RL with expressive image-based models—backpropagating through a large diffusion model—can be removed by evaluating the policy gradient with a local surrogate. Forward rollouts are produced by the global diffusion world model; the local RSSM, operating in latent space, supplies the Jacobians needed to differentiate the return. Because the local model only needs single-step accuracy at states generated by the global model, it can be kept small and fast. The paper demonstrates this on Push-T (9/10 real-robot successes vs 1/10 for PPO) and ego-centric Push Cube, where the learned policy exhibits emergent active search behavior a

What carries the argument

The decoupling of forward and backward models. The global model is a diffusion world model (DIAMOND for Push-T, a DreamerV4-style transformer latent diffusion model for Push Cube) that produces realistic, long-horizon image rollouts. The backward model is a Recurrent State-Space Model (RSSM) from DreamerV3, trained in a compact latent space, whose Jacobians are evaluated at the encodings of the forward-simulated images rather than at its own next states. This keeps gradient computation cheap and avoids differentiating through diffusion; the SAPO objective adds entropy regularization for stable policy updates.

Load-bearing premise

The backward RSSM is required to be accurate only one step ahead, because its Jacobians are evaluated at the exact forward-simulated images, but the policy gradient is a product of those Jacobians along H steps, so single-step accuracy does not guarantee the accumulated gradients match the true dynamics.

What would settle it

Roll out a DMO-trained policy inside the global diffusion world model, then compute the policy gradient two ways: once through the local RSSM Jacobians and once via finite differences or direct backpropagation through the diffusion model. If the two gradient estimates diverge significantly along states the policy actually visits, the decoupling's core premise fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim holds, reinforcement learning for hard-to-model manipulation tasks no longer requires hand-crafted physics simulators; world models trained on real play data can serve as the training environment.
  • The decoupled gradient scheme would make large diffusion world models practical for first-order model-based RL, removing the computational barrier that previously limited them to zeroth-order, sample-inefficient optimization.
  • Policies trained this way can transfer zero-shot to real robots, and in the reported Push Cube experiments they prove robust to changes in the low-level locomotion controller without retraining the world models.
  • Compared to behavior cloning, the RL objective inside the world model yields behaviors that go beyond the demonstration distribution, such as pushing the object fully into the goal and actively searching when the object leaves the field of view.
  • The intent-label reward mechanism (a user toggling a button to mark demonstration-like behavior) offers a practical replacement for hand-engineered reward functions in pixel-space tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The division of labor between a heavy generative forward model and a small analytic backward model may generalize beyond diffusion: any expensive differentiable simulator could be paired with a lightweight local surrogate for gradient computation.
  • A testable extension is to apply this decoupling to longer-horizon or contact-rich tasks with sparse rewards; the current evidence is limited to two manipulation tasks with a small number (10 and 3) of real-robot trials.
  • The emergent active-search behavior suggests that RL inside learned world models can discover strategies that are absent from the demonstration data, which could be studied more directly by varying the proportion of play versus demonstration data.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 1 minor

Summary. The paper proposes DMO-SAPO, a model-based RL method that trains policies inside learned diffusion world models from real-robot image data. A global diffusion model generates forward rollouts in pixel space, while a lightweight RSSM local model supplies first-order policy gradients through a decoupled backpropagation scheme. The authors evaluate on a table-top Push-T task with a Flexiv arm and an ego-centric Push Cube task with a Unitree Go2 quadruped, reporting superior sample and time efficiency against PPO, a 9/10 vs 1/10 real-robot success rate on Push-T, and qualitative improvements over behavior cloning. The central claim is that the decoupling makes first-order gradient RL feasible with high-fidelity diffusion world models, enabling zero-shot transfer to real robots without a physics simulator.

Significance. If the results are reproducible, the paper is a meaningful step toward simulator-free RL with diffusion world models. The decoupling of a high-fidelity global forward model from a lightweight local gradient model is a conceptually clean solution to the differentiability bottleneck of diffusion models, and the real-robot experiments, though small, go beyond simulation-only benchmarks. The comparison against ACT and the robustness experiment under a changed low-level controller are useful additions. However, the central algorithmic premise—that the local model only needs single-step accuracy—is not backed by a Jacobian-fidelity analysis, and the evaluation statistics are too sparse to fully support the broad efficiency and robustness claims.

major comments (1)
  1. [IV.E, Fig. 7] The robustness claim that 'both DMO and BC maintained their performance' under a changed low-level controller is not supported by quantitative evidence. No success counts, episode numbers, or trajectory metrics are reported for this condition. Since the paper uses this experiment to argue that pretrained world models may generalize across robot configurations, please provide the same quantitative metrics as in Table II (or at least success counts and N) for the changed-controller condition.
minor comments (1)
  1. [Fig. 2 caption] Figure 2(b) says both models are 'initialized with zero context,' while Section IV.B states that the DreamerV4 model for the Push Cube task uses a 96-frame context. Please clarify whether the figure shows a separate zero-context diagnostic or an inconsistency in the experimental setting.

Circularity Check

0 steps flagged

No significant circularity: the central derivation is empirically validated on real robots, with only minor self-citation that is not load-bearing.

full rationale

The paper's central claim is not circular in a derivation sense. The policy is optimized inside a diffusion world model learned from real robot interactions, with a local RSSM surrogate providing gradients, and the final success is evaluated on the real robot (Push-T 9/10, Push Cube with additional trajectory metrics). None of the reported success criteria are used to fit the core world models or the policy; the global model is trained on play data, the local model is pretrained on that data and fine-tuned on global-model rollouts, and the policy is then deployed zero-shot. The main self-citation is to the authors' prior DMO work [2], which supplies the decoupled forward/backward modeling idea. This is an architectural origin rather than an external uniqueness theorem or a fitted parameter: the paper does not invoke [2] as proof of correctness, and the experimental validation is independent. The Section III.E statement that the backward model 'requires only single-step accuracy' is a genuine correctness risk, because the H-step policy gradient is a product of learned Jacobians and single-step predictive accuracy does not control Jacobian fidelity; however, that is a modeling/soundness concern, not circularity. No equation reduces to its input by construction, and no fitted value is renamed as a prediction. Overall score 1 reflects the minor, disclosed self-citation that is not itself verified in this paper but is not load-bearing for the empirical central claim.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

No new physical entities or conservation laws are invented. The paper's contribution is algorithmic; its load-bearing assumptions concern the fidelity of learned models and the sufficiency of surrogate gradients. Several hyperparameters (α, λ, H, diffusion steps, context lengths) are chosen by hand and not fully reported, making the central result dependent on unstated tuning.

free parameters (6)
  • SAPO temperature α
    Entropy-regularization coefficient in Eq. 4; chosen by hand or tuned, value not reported.
  • λ (critic return mixing)
    Mixing coefficient for λ-returns in Eq. 6; value not reported.
  • horizon H = 64 (DMO); 128 (PPO baseline)
    Unroll horizon in Algorithm 1; set per task and per baseline, affecting the gradient path length.
  • Diffusion steps (DIAMOND) = 3
    Number of denoising steps for the forward model (Section III.D.1), chosen as a speed/accuracy tradeoff.
  • Context length L_init = 32 (Push-T), 96 (Push Cube)
    History frames pre-filled into KV cache (Section III.F), chosen per task to handle occlusions.
  • Replay buffer / model mini-epochs
    Training schedule hyperparameters in Algorithm 1 not specified.
axioms (5)
  • domain assumption The diffusion-based global world model produces rollouts faithful enough to serve as a training environment over H steps.
    Section III.D.1 and Fig. 2 show qualitative unrolls but no quantitative error; the entire policy training relies on this fidelity.
  • domain assumption The local RSSM's Jacobians, evaluated at forward-model states, are a valid proxy for true dynamics Jacobians over the H-step backward unroll.
    Section III.E asserts only single-step accuracy is needed; this is the paper's key methodological premise and is unproven.
  • domain assumption The learned reward models (contrastive for Push-T, binary-label token for Push Cube) encode the intended task objective.
    Section III.D.2; reward mis-specification would mislead the policy, and no reward-model accuracy is reported.
  • domain assumption The collected play data (4h for Push-T, 12h for Push Cube) covers the state-action distribution needed for the tasks.
    Section III.C; the paper does not analyze data coverage.
  • standard math Standard MDP, policy gradient, and stochastic approximation assumptions hold (smoothness, stationarity, differentiability).
    Section III.A; Eq. 1-2 assume differentiability and interchange of gradient and expectation.

pith-pipeline@v1.3.0-alltime-deepseek · 10652 in / 18493 out tokens · 172706 ms · 2026-08-03T03:59:13.069911+00:00 · methodology

0 comments
read the original abstract

World models offer a promising avenue for more faithfully capturing complex dynamics, including contacts and non-rigidity, as well as complex sensory information, such as visual perception, in situations where standard simulators struggle. However, these models are computationally complex to evaluate, posing a challenge for popular RL approaches that have been successfully used with simulators to solve complex locomotion tasks but yet struggle with manipulation. This paper introduces a method that bypasses simulators entirely, training RL policies inside world models learned from robots' interactions with real environments. At its core, our approach enables policy training with large-scale diffusion models via a novel decoupled first-order gradient (FoG) method: a full-scale world model generates accurate forward trajectories, while a lightweight latent-space surrogate approximates its local dynamics for efficient gradient computation. This coupling of a local and global world model ensures high-fidelity unrolling alongside computationally tractable differentiation. We demonstrate the efficacy of our method on the Push-T manipulation task, where it significantly outperforms PPO in sample efficiency. We further evaluate our approach through an ego-centric object manipulation task with a quadruped. Together, these results demonstrate that learning inside data-driven world models is a promising pathway for solving hard-to-model RL tasks in image space without reliance on hand-crafted physics simulators.

Figures

Figures reproduced from arXiv: 2602.06219 by Joseph Amigo, Ludovic Righetti, Nicolas Mansard, Rooholla Khorrambakht.

Figure 1
Figure 1. Figure 1: Overview of the proposed approach. Global world/reward models are learned from play/demonstration data (Steps 1 and [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: (a) Unrolling of real and model-predicted trajectories [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 5
Figure 5. Figure 5: Task completion comparison. Top: The Behavior Cloning (ACT) policy successfully approaches the cube but stops pushing before the object enters the goal, reflecting the sub-optimal demonstration distribution. Bottom: Our RL Pol￾icy (DMO) generalizes beyond the demonstrations, learning to push the cube fully into the net to maximize the task reward. iment ( [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 3
Figure 3. Figure 3: (a) Efficiency comparison on the Push-T task: sample [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Three real-robot Push-T trajectories executed by the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 7
Figure 7. Figure 7: Real-world execution of the DMO-learned policy for [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 8 linked inside Pith

  1. [1]

    Diffusion for world modeling: Visual details matter in atari.Advances in Neural Information Processing Sys- tems, 37, 2024

    Eloi Alonso, Adam Jelley, Vincent Micheli, Anssi Kan- ervisto, Amos Storkey, Tim Pearce, and Franc ¸ois Fleuret. Diffusion for world modeling: Visual details matter in atari.Advances in Neural Information Processing Sys- tems, 37, 2024

    2024

  2. [2]

    First order model-based rl through decoupled backpropagation

    Joseph Amigo, Rooholla Khorrambakht, Elliot Chane- Sane, Nicolas Mansard, and Ludovic Righetti. First order model-based rl through decoupled backpropagation. In Conference on Robot Learning (CoRL), 2025

    2025

  3. [3]

    Sample-efficient reinforce- ment learning with stochastic ensemble value expansion

    Jacob Buckman, Danijar Hafner, George Tucker, Eugene Brevdo, and Honglak Lee. Sample-efficient reinforce- ment learning with stochastic ensemble value expansion. Advances in neural information processing systems, 31, 2018

    2018

  4. [4]

    Soloparkour: Constrained reinforcement learning for visual locomotion from privileged experience

    Elliot Chane-Sane, Joseph Amigo, Thomas Flayols, Lu- dovic Righetti, and Nicolas Mansard. Soloparkour: Constrained reinforcement learning for visual locomotion from privileged experience. InConference on Robot Learning (CoRL), 2024

    2024

  5. [5]

    Cat: Constraints as terminations for legged locomotion reinforcement learning

    Elliot Chane-Sane, Pierre-Alexandre Leziart, Thomas Flayols, Olivier Stasse, Philippe Sou `eres, and Nicolas Mansard. Cat: Constraints as terminations for legged locomotion reinforcement learning. In2024 IEEE/RSJ International Conference on Intelligent Robots and Sys- tems (IROS), pages 13303–13310, 2024

    2024

  6. [6]

    Diffu- sion forcing: Next-token prediction meets full-sequence diffusion.Advances in Neural Information Processing Systems, 37, 2024

    Boyuan Chen, Diego Mart ´ı Mons ´o, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffu- sion forcing: Next-token prediction meets full-sequence diffusion.Advances in Neural Information Processing Systems, 37, 2024

    2024

  7. [7]

    Extreme parkour with legged robots

    Xuxin Cheng, Kexin Shi, Ananye Agarwal, and Deepak Pathak. Extreme parkour with legged robots. In2024 IEEE International Conference on Robotics and Automa- tion (ICRA), pages 11443–11450, 2024

    2024

  8. [8]

    Deep reinforcement learning in a handful of trials using probabilistic dynamics models

    Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. Advances in neural information processing systems, 31, 2018

    2018

  9. [9]

    Model- augmented actor-critic: Backpropagating through paths

    Ignasi Clavera, Violet Fu, and Pieter Abbeel. Model- augmented actor-critic: Backpropagating through paths. InInternational Conference on Learning Representations (ICLR), 2020

    2020

  10. [10]

    One step diffusion via shortcut models.arXiv preprint arXiv:2410.12557, 2024

    Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models.arXiv preprint arXiv:2410.12557, 2024

    Pith/arXiv arXiv 2024

  11. [11]

    Pwm: Policy learning with multi-task world models.arXiv preprint arXiv:2407.02466, 2024

    Ignat Georgiev, Varun Giridhar, Nicklas Hansen, and Animesh Garg. Pwm: Policy learning with multi-task world models.arXiv preprint arXiv:2407.02466, 2024

    Pith/arXiv arXiv 2024

  12. [12]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. InInternational conference on machine learning (ICML), 2018

    2018

  13. [13]

    Soft actor-critic algorithms and applications

    Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, and Sergey Levine. Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905, 2019

    Pith/arXiv arXiv 2019

  14. [14]

    Mastering atari with discrete world models

    Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models. InInternational Conference on Learning Rep- resentations (ICLR), 2021

    2021

  15. [15]

    Mastering diverse control tasks through world models.Nature, 640(8059):647–653, 2025

    Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Tim- othy Lillicrap. Mastering diverse control tasks through world models.Nature, 640(8059):647–653, 2025

    2025

  16. [17]

    Training agents inside of scalable world models.arXiv preprint arXiv:2509.24527, 2025

    Danijar Hafner, Wilson Yan, and Timothy Lillicrap. Training agents inside of scalable world models.arXiv preprint arXiv:2509.24527, 2025

    Pith/arXiv arXiv 2025

  17. [18]

    Temporal difference learning for model predictive control

    Nicklas Hansen, Xiaolong Wang, and Hao Su. Temporal difference learning for model predictive control. In International Conference on Machine Learning (ICML), 2022

    2022

  18. [19]

    Td-mpc2: Scalable, robust world models for continuous control

    Nicklas Hansen, Hao Su, and Xiaolong Wang. Td-mpc2: Scalable, robust world models for continuous control. InInternational Conference on Learning Representations (ICLR), 2024

    2024

  19. [20]

    Axial attention in multidimensional transformers

    Jonathan Ho, Nal Kalchbrenner, Dirk Weissenborn, and Tim Salimans. Axial attention in multidimensional transformers. InInternational Conference on Learning Representations (ICLR), 2020

    2020

  20. [21]

    Anymal parkour: Learning agile navigation for quadrupedal robots.Science Robotics, 9(88), 2024

    David Hoeller, Nikita Rudin, Dhionis Sako, and Marco Hutter. Anymal parkour: Learning agile navigation for quadrupedal robots.Science Robotics, 9(88), 2024

    2024

  21. [22]

    When to trust your model: Model-based policy optimization.Advances in neural information processing systems, 32, 2019

    Michael Janner, Justin Fu, Marvin Zhang, and Sergey Levine. When to trust your model: Model-based policy optimization.Advances in neural information processing systems, 32, 2019

    2019

  22. [23]

    An introduction to zero-order op- timization techniques for robotics.arXiv preprint arXiv:2506.22087, 2025

    Armand Jordana, Jianghan Zhang, Joseph Amigo, and Ludovic Righetti. An introduction to zero-order op- timization techniques for robotics.arXiv preprint arXiv:2506.22087, 2025

    arXiv 2025

  23. [24]

    Elucidating the design space of diffusion-based genera- tive models.Advances in neural information processing systems, 35, 2022

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based genera- tive models.Advances in neural information processing systems, 35, 2022

    2022

  24. [25]

    Worldplanner: Monte carlo tree search and mpc with action-conditioned visual world models, 2025

    Rooholla Khorrambakht, Joaquim Ortiz-Haro, Joseph Amigo, Omar Mostafa, Daniel Dugas, Franziska Meier, and Ludovic Righetti. Worldplanner: Monte carlo tree search and mpc with action-conditioned visual world models, 2025

    2025

  25. [26]

    Auto-encoding variational bayes

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes. InInternational Conference on Learn- ing Representations (ICLR), 2014

    2014

  26. [27]

    Model-ensemble trust-region policy optimization

    Thanard Kurutach, Ignasi Clavera, Yan Duan, Aviv Tamar, and Pieter Abbeel. Model-ensemble trust-region policy optimization. InInternational Conference on Learning Representations (ICLR), 2018

    2018

  27. [28]

    Investigating compounding prediction errors in learned dynamics models.arXiv preprint arXiv:2203.09637, 2022

    Nathan Lambert, Kristofer Pister, and Roberto Calandra. Investigating compounding prediction errors in learned dynamics models.arXiv preprint arXiv:2203.09637, 2022

    Pith/arXiv arXiv 2022

  28. [29]

    Uncertainty-aware robotic world model makes offline model-based reinforcement learning work on real robots, 2025

    Chenhao Li, Andreas Krause, and Marco Hutter. Uncertainty-aware robotic world model makes offline model-based reinforcement learning work on real robots, 2025

    2025

  29. [30]

    Lightzero: A unified benchmark for monte carlo tree search in general sequential decision scenarios.Advances in Neural Information Processing Systems, 36, 2024

    Yazhe Niu, Yuan Pu, Zhenjie Yang, Xueyan Li, Tong Zhou, Jiyuan Ren, Shuai Hu, Hongsheng Li, and Yu Liu. Lightzero: A unified benchmark for monte carlo tree search in general sequential decision scenarios.Advances in Neural Information Processing Systems, 36, 2024

    2024

  30. [31]

    Unizero: Generalized and effi- cient planning with scalable latent world models.arXiv preprint arXiv:2406.10667, 2024

    Yuan Pu, Yazhe Niu, Jiyuan Ren, Zhenjie Yang, Hong- sheng Li, and Yu Liu. Unizero: Generalized and effi- cient planning with scalable latent world models.arXiv preprint arXiv:2406.10667, 2024

    Pith/arXiv arXiv 2024

  31. [32]

    Mastering atari, go, chess and shogi by planning with a learned model.Nature, 588(7839):604– 609, 2020

    Julian Schrittwieser, Ioannis Antonoglou, Thomas Hu- bert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al. Mastering atari, go, chess and shogi by planning with a learned model.Nature, 588(7839):604– 609, 2020

    2020

  32. [33]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  33. [34]

    Daydreamer: World models for physical robot learning

    Philipp Wu, Alejandro Escontrela, Danijar Hafner, Pieter Abbeel, and Ken Goldberg. Daydreamer: World models for physical robot learning. InConference on robot learning, pages 2226–2240. PMLR, 2023

    2023

  34. [35]

    Learning to combat compounding- error in model-based reinforcement learning, 2019

    Chenjun Xiao, Yifan Wu, Chen Ma, Dale Schuurmans, and Martin M ¨uller. Learning to combat compounding- error in model-based reinforcement learning, 2019

    2019

  35. [36]

    Stabilizing rein- forcement learning in differentiable multiphysics simula- tion.International Conference on Learning Representa- tions (ICLR), 2025

    Eliot Xing, Vernon Luk, and Jean Oh. Stabilizing rein- forcement learning in differentiable multiphysics simula- tion.International Conference on Learning Representa- tions (ICLR), 2025

    2025

  36. [37]

    Rank2reward: Learning shaped reward functions from passive video

    Daniel Yang, Davin Tjia, Jacob Berg, Dima Damen, Pulkit Agrawal, and Abhishek Gupta. Rank2reward: Learning shaped reward functions from passive video. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 2806–2813. IEEE, 2024

    2024

  37. [38]

    Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn

    Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manip- ulation with low-cost hardware, 2023

    2023

  38. [39]

    Sim-to-real transfer in deep reinforcement learning for robotics: a survey

    Wenshuai Zhao, Jorge Pe ˜na Queralta, and Tomi Wester- lund. Sim-to-real transfer in deep reinforcement learning for robotics: a survey. In2020 IEEE symposium series on computational intelligence (SSCI), pages 737–744. IEEE, 2020

    2020

  39. [40]

    Robot parkour learning.arXiv preprint arXiv:2309.05665, 2023

    Ziwen Zhuang, Zipeng Fu, Jianren Wang, Christo- pher Atkeson, Soeren Schwertfeger, Chelsea Finn, and Hang Zhao. Robot parkour learning.arXiv preprint arXiv:2309.05665, 2023

    Pith/arXiv arXiv 2023