Pith. sign in

REVIEW 4 major objections 5 minor 57 references

MPAIL2 learns real-world pick-and-place from video alone, reaching 82% success in under 40 minutes, where RL and behavior cloning with extra supervision reach 0% and 12%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 20:01 UTC pith:DPNI3R4M

load-bearing objection Genuinely novel real-world IRLfO demo, but the headline numbers are overstated and need cleanup before I'd trust them. the 4 major comments →

arxiv 2602.24121 v2 pith:DPNI3R4M submitted 2026-02-27 cs.RO

Online World Modeling Enables Real-World Inverse Reinforcement Learning from Observation

Tyler Han , Bat Nemekhbold , Siyang Shen , Rohan Baijal , Richard Ebock , Harine Ravichandiran , Sanghun Jung , Kevin Huang
show 1 more author
Byron Boots
This is my paper
classification cs.RO
keywords inverse reinforcement learning from observationworld modelsmodel predictive path integrallearning from demonstrationreal-world robot learningmanipulationonline transferadversarial imitation learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to make Inverse Reinforcement Learning from Observation (IRLfO) work on real robots, not just in simulation. It claims that MPAIL2, a planning-based algorithm, can learn visual manipulation from scratch using only video demonstrations of a human and its own online experience, with no hand-designed reward, no action labels, and no simulator. In under 40 minutes it reaches 82% success on real pick-and-place, and the authors claim it is the first IRLfO method demonstrated end-to-end in the real world. If true, this would remove two major bottlenecks in robot learning—reward design and teleoperation—and make video-only teaching practical.

Core claim

The paper's central claim is that adversarial IRL, previously too sample-inefficient for real robots, becomes practical when the reward is trained off-policy and the agent plans over a learned latent world model rather than executing a policy directly. The world model predicts how the scene evolves in a learned latent space, the reward distinguishes expert video transitions from the agent's own, and MPPI uses these to choose actions, with a multi-step policy used mainly to seed planning and support value learning. The empirical claim is that on real pick-and-place and block pushing, MPAIL2 succeeds in under an hour where RLPD (with dense reward and action labels) and Diffusion Policy (with a

What carries the argument

The load-bearing component is the self-supervised latent dynamics model fψ(z,a), trained jointly with an encoder by predicting future latents, plus the adversarially trained transition reward rθ(z,z') with a gradient penalty, an off-policy Q-value function, and a multi-step policy used to seed MPPI planning. The dynamics model is deliberately independent of task rewards so that it can transfer across tasks; planning over predicted latent trajectories is what lets the agent recover from mistakes and continue improving when the policy alone would fail.

Load-bearing premise

The load-bearing assumption is that the success metrics reflect true task mastery: push success is credited if the block crosses the target line at any point during the episode (Appendix B.1), and pick-and-place success is hand-labeled by the operators; if these metrics are lenient, the reported success rates exaggerate what the robot actually learned.

What would settle it

Re-run Real: Block Push evaluations with success defined as the block remaining beyond the target line at the end of the episode, and re-run Real: Pick-and-Place with an independent evaluator who was not involved in training and who labels the four stages from recorded video. If success rates drop substantially below the reported 62-100% push and 68-82% pick-and-place, the headline claims overstate the policy's genuine capability.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • IRLfO becomes a viable paradigm for real-world robot learning: a user only needs to record a few videos of the task.
  • The same algorithm, with identical hyperparameters, works on two different robot arms and on pushing and pick-and-place, suggesting generality across embodiments.
  • Transfer learning between manipulation tasks is possible online from scratch, and transferring only the encoder and dynamics preserves most of the benefit.
  • Baselines that require more supervision (dense reward, action labels) can be less sample-efficient in the real world than a method that requires none, at least in this setting.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the encoder/dynamics transfer result holds beyond these two tasks, MPAIL2 could be paired with pre-trained visual encoders or foundation models to cut real-world interaction needs further; the paper does not test this.
  • The success metrics are permissive: push credit is given for any crossing of the target line and pick-and-place stages are hand-labeled by the operators, so advertised success rates could overstate true task completion; an independent strict-metric evaluation would clarify.
  • The planner's central role suggests that iterative, implicit policies (like diffusion) and model-based planners may be converging on the same principle; the paper draws this connection as an interpretation, not a proof.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MPAIL2, a planning-based inverse reinforcement learning from observation (IRLfO) algorithm that learns real-world visual manipulation tasks from demonstration observations and online interaction, without access to rewards or demonstration actions. The method combines a latent encoder, a learned dynamics model, an adversarially trained reward, a value function, a multi-step policy, and MPPI planning. Experiments are reported in simulation and on real robots for block pushing and pick-and-place, alongside ablations and a transfer-learning study. The central claims are that MPAIL2 is the first real-world IRLfO method trained from scratch, that it reaches 82% pick-and-place success in under 40 minutes, and that it outperforms RLPD (0%) and BC (12%) under comparable data budgets.

Significance. If the headline claims survive scrutiny, this is a meaningful advance: it would demonstrate that a reward-free, action-free observational learning paradigm can succeed on real-world manipulation in under an hour, and that learned world models can transfer online between tasks. The paper is accompanied by detailed algorithm pseudocode, hyperparameter tables, ablations (over demonstration count, planning horizon, and GP coefficient), and a project page with videos. These elements increase reproducibility and give the work value beyond the headline result. However, the load-bearing evaluation choices — particularly the permissive and partly subjective success definitions — currently undercut the strength of the claims, and the abstract's 'under 40 minutes' is internally inconsistent with the reported 70–90 minute training times.

major comments (4)
  1. [Abstract vs. Fig. 5 caption] The abstract states that 'In under 40 minutes, MPAIL2 learns pick-and-place from scratch to 82% success,' but Fig. 5's caption reports that complete training time is approximately 90 minutes for Block Push and 70 minutes for Pick and Place. Section I's 'consistent success within 40 minutes or less' appears to refer to time until first success, which is a different claim. The 82% is the Best checkpoint in Table IV, not necessarily achieved in the first 40 minutes. This inconsistency must be corrected — either change the abstract's time claim or define precisely what occurs at 40 minutes (e.g., first success, not full training).
  2. [Appendix B.1] The push success definition is permissive: 'Success of an episode or evaluation is credited if any y_B < y_goal at some point in the trajectory.' A transient crossing of the target line that is not maintained counts as success. Since Real: Block Push (Table IV) and the Transfer Push results rely on this metric, the reported 100% Best and 62% Last success rates may materially overstate the actual ability to push the block beyond the line and keep it there. A terminal-condition success metric (e.g., y_B < y_goal at the final time step) should be reported, or the current metric should be explicitly justified as the intended task definition.
  3. [Appendix B.2] Pick-and-place success is determined by completing four stages 'at any time in order,' where a stage such as 'placed' can be credited at any point in the trajectory, not necessarily at the end. In the real-world setup, 'stages are hand-labeled after training for evaluation metrics.' This makes the headline 82% Best / 68% Last success rates non-verifiable and potentially inflated by a transient grasp-and-place or a lenient human label. An objective, terminal-state criterion (e.g., final block pose with gripper open) should be used, or in the absence of full automation, the hand-labeling protocol should be described in enough detail to permit independent replication and inter-rater reliability.
  4. [Table IV / Abstract] Real-world results in Tables IV and VI are reported as a single number for each of Best and Last checkpoints, with no variance or per-seed breakdown, even though the caption states evaluations are over 50 trials per checkpoint across 3 seeds. The abstract quotes the Best checkpoint (82%) for pick-and-place without qualification, while Last is 68%. Without confidence intervals or per-seed data, the headline comparison against BC (12%) and RLPD (0%) is not statistically grounded. Please report per-seed results and clearly label the quoted 82% as the best checkpoint rather than the typical performance.
minor comments (5)
  1. [III.C, Eq. (13)] The notation E_d[r] in Eq. (13) is unclear; it likely means expectation over expert demonstrations. Use E_D[r] for consistency with Eq. (16).
  2. [IV.B.2] In the real-world Pick-and-Place setup, the text states '64×64RG images'; this should be '64×64 RGB images.'
  3. [Algorithm 2, line 19] The terminal value term Q_zeta(hat z_{t+H}, a_{t+H}) uses a_{t+H}, which is outside the length-H action plan a_{t:t+H-1}. This is likely a typo; please clarify whether it should be a_{t+H-1} or the last action of the plan.
  4. [III.E] The multi-step policy notation is inconsistent: it is introduced as π(a_{t:t+H−1}|z_t) but Eq. (18) writes E_{hat a_t∼π(·|z_t)}. Please make the action-sequence indexing consistent throughout.
  5. [Project page] The paper provides a project page with videos but does not state whether code will be released. For reproducibility of a real-world robot learning method, a code release (or at least trained model checkpoints) would be valuable.

Circularity Check

0 steps flagged

No significant circularity; one minor non-load-bearing self-citation noted.

full rationale

The derivation chain of MPAIL2 is self-contained. The inferred reward (Eq. 13/16) is an adversarial objective separating expert observations from learner replay; the encoder/dynamics (Eq. 12) is a self-supervised latent prediction loss; the value and policy are trained against model-based returns (Eqs. 14-19). None of these losses is defined in terms of the reported success metrics. Success is evaluated externally: for Real: Block Push, an AprilTag-derived block position crossing the target line at any point (Appendix B.1); for Real: Pick-and-Place, four stages hand-labeled after training (Appendix B.2). Those success definitions are permissive and partly subjective, which is a correctness and evaluation-reliability concern, not a circularity concern; the 82% pick-and-place success is itself an externally measured quantity, not a fitted parameter or a re-expression of the adversarial reward. The only self-citation is the related-work claim that the authors' prior MPAIL is 'the only IRLfO algorithm demonstrating sufficient robustness for real-world deployment without assumptions upon the inferred reward' (Section II, [16]). This citation is not load-bearing for the paper's empirical claims, which are benchmarked against independent baselines (RLPD, Diffusion Policy, simulator tasks, and gymnasium environments). Thus no load-bearing step reduces by construction to its input, and the circularity score is low.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 0 invented entities

No new physical entities are introduced. The central assumptions are about the adequacy of deterministic latent dynamics, the representativeness of the replay buffer for the adversarial reward objective, and the quality of human demonstrations. Two hyperparameters (β, H) are tuned via ablation on the target task.

free parameters (2)
  • Gradient penalty coefficient β = 0.1
    Selected via ablation on Sim: Block Push (Figure 8, GP coefficients), i.e., tuned on the target task. Affects reward training stability.
  • Planning horizon H = 7
    Selected via ablation on Sim: Block Push (Figure 8, Horizons). Affects value and policy learning.
axioms (5)
  • domain assumption Latent state is Markovian and deterministic dynamics f is sufficient to predict future latent states (Section III-B: 'As the dynamics model is deterministic...').
    If real-world dynamics require stochastic or recurrent state, the planner's rollouts are biased; the paper itself discusses recurrent dynamics as a limitation.
  • domain assumption The replay buffer B approximates the distribution of 'other policies' well enough for the adversarial reward objective (Eq. 16).
    The reward is defined as expert advantage over B; if B is unrepresentative, the learned reward is miscalibrated for states outside the buffer.
  • domain assumption Human demonstrations collected via space-mouse/keyboard are sufficiently consistent and informative to learn a reward (Section IV-B).
    The method's performance depends on the quality of human demos; the paper uses only human demos and cautions against synthetic demos.
  • domain assumption MPPI with random and policy-seeded sampling can adequately optimize the latent model-based return (Algorithm 2).
    If the planning landscape is too rugged, MPPI may not find good actions; paper hypothesizes other planners could improve reward focus.
  • domain assumption The encoder/dynamics model, trained only by self-prediction (Eq. 12), preserves task-relevant information such as object position and contact.
    The reward and value depend on latent states; if the encoder discards object state, the reward cannot distinguish expert and learner behavior.

pith-pipeline@v1.3.0-alltime-deepseek · 25602 in / 16010 out tokens · 137697 ms · 2026-08-02T20:01:55.394960+00:00 · methodology

0 comments
read the original abstract

Current methods in robot learning are fundamentally bottlenecked by one or more of: hand-designed rewards, simulation modeling, or action supervision (e.g. teleoperation) each requiring significant domain expertise, engineering effort, and robot-operator labor. Towards eliminating these bottlenecks, this work pursues observational learning via Inverse Reinforcement Learning from Observation (IRLfO) in which only access to task observations (e.g. video) is assumed. Due to the challenging setting and limitations of RL methods, IRLfO has thus far remained impractical for real-world robot learning. Here, we present the first IRL method to learn visual manipulation in the real world from scratch, and the first real-world demonstration of positive online transfer across visual manipulation tasks from scratch. In under 40 minutes, MPAIL2 learns pick-and-place from scratch to 82% success, where RL and BC with equal interaction and demonstration budgets reach only 0% and 12% despite their reward and action supervision. Interactive project page with training videos: https://uwrobotlearning.github.io/mpail2/

Figures

Figures reproduced from arXiv: 2602.24121 by Bat Nemekhbold, Byron Boots, Harine Ravichandiran, Kevin Huang, Richard Ebock, Rohan Baijal, Sanghun Jung, Siyang Shen, Tyler Han.

Figure 1
Figure 1. Figure 1: Overview of MPAIL2. (0) The learner observes a task demonstration and stores the observations before training. (1) The learner observes the world, and encodes the observation into its current latent state, z0 = e(o). The learner’s policy π suggests reactive, possibly suboptimal actions (purple dotted lines). Before executing any actions, the learner predicts and evaluates the world at future latent states … view at source ↗
Figure 2
Figure 2. Figure 2: Top. Simulation of MPAIL2 on the Block Push (state) task at three time steps along an episode. Bottom. Predicted plans in the XY plane (top￾down) for the next one second at each time step. End-effector plans are drawn in green. Block plans are drawn in orange. Block trajectories, static in the leftmost frame, become dynamic as the robot approaches and makes contact. These predictions show how the agent gra… view at source ↗
Figure 3
Figure 3. Figure 3: Overview of evaluation tasks. We evaluate our method and baselines on 4 tasks - 2 in sim and 2 in real. The Pick and Place tasks (a,c) involve [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Cumulative successes in simulated experiments. Offline results of BC are shown in Table III. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Cumulative successes in real-world experiments. Offline results of BC [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Cumulative successes in real-world transfer experiment. Methods [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Results in gymnasium environments [47]. All methods are provided 50 synthetic demonstration observations. RLPD is additionally provided dense reward and demonstration actions. Reward is normalized such that 1.0 reflects the average reward across the demonstrations. 0 50 100 150 200 250 300 0 50 100 150 Cumulative Success Count Demonstration Quantities 1 Demos 5 Demos 10 Demos 20 Demos 27 Demos 0 50 100 150… view at source ↗
Figure 8
Figure 8. Figure 8: MPAIL2 ablations over number of demonstrations (left), horizon lengths (middle), and gradient penalty coefficients (right) in Sim: Block Push. 16 cm reset region. The minimum distance between the reset region and the target placement line is 18 cm. Actions are defined in end-effector space and include Carte￾sian position commands in x,y, and z, together with a gripper command. All actions are constrained t… view at source ↗
Figure 9
Figure 9. Figure 9: Time efficiency of MPAIL2 compared to other methods. We plot talgo(c)/tMPAIL2(c) where t(c) is the learning iteration at which the algorithm achieves a cumulative success percentage. With respect to a given baseline, the value reflects how many times faster MPAIL2 achieves a given success percentage. An endpoint indicates where the corresponding baseline achieves its maximum success percentage. 0 100 200 3… view at source ↗
Figure 10
Figure 10. Figure 10: Success rate stability. We plot the average success rate of each method over a moving window of 25 episodes. A value of 1 indicates success on all episodes across 25 updates. This graph captures a method’s stability across updates. In Sim: Block Push (State) and Sim: Pick and Place, MPAIL2 is able to maintain a consistent success rate of roughly 0.8 and 0.6 respectively while the decreasing curve in Sim: … view at source ↗
Figure 11
Figure 11. Figure 11: Cumulative successes in real-world transfer experiment in Pick and Place. New task demonstrations invert the directionality of the initial demonstrations’ pick and place procedure by inverting the start and end regions for the block. Model evaluations for this experiment are shown in Table VI. 0 100 200 300 400 Number of Environment Steps (x10 2 ) 0.0 0.2 0.4 0.6 0.8 Episode Policy Influence [PITH_FULL_I… view at source ↗
Figure 12
Figure 12. Figure 12: Policy influence on planning. “Influence” of the policy on planning is calculated by taking the proportion of the total score contributed by the sampled policy plans during the final iteration of MPPI at one timestep (see Algorithm 2). Averaging over all timesteps in the episode gives the Episode Policy Influence on the y-axis of this plot. be achieved by depending on the demonstrations. BC’s high relativ… view at source ↗
Figure 14
Figure 14. Figure 14: PyTorch [36] 7.1M Model Architecture for observation space with two RGB images and proprioception (i.e. Push Block and Pick-and-Place). 19 [PITH_FULL_IMAGE:figures/full_fig_p019_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 19 linked inside Pith

  1. [1]

    Pieter Abbeel and Andrew Y . Ng. Apprenticeship learn- ing via inverse reinforcement learning. InTwenty-first international conference on Machine learning - ICML ’04, page 1, Banff, Alberta, Canada, 2004. ACM Press. doi: 10.1145/1015330.1015430. URL http://portal.acm. org/citation.cfm?doid=1015330.1015430

    arXiv 2004

  2. [2]

    Wasserstein Generative Adversarial Networks

    Martin Arjovsky, Soumith Chintala, and Léon Bot- tou. Wasserstein Generative Adversarial Networks. In Proceedings of the 34th International Conference on Machine Learning, pages 214–223. PMLR, July 2017. URL https://proceedings.mlr.press/v70/arjovsky17a.html. ISSN: 2640-3498

    2017

  3. [3]

    Ball, Laura Smith, Ilya Kostrikov, and Sergey Levine

    Philip J. Ball, Laura Smith, Ilya Kostrikov, and Sergey Levine. Efficient Online Reinforcement Learning with Offline Data, May 2023. URL http://arxiv.org/abs/2302. 02948. arXiv:2302.02948 [cs]

    Pith/arXiv arXiv 2023

  4. [4]

    Blending MPC & Value Function Approximation for Efficient Reinforcement Learning

    Mohak Bhardwaj, Sanjiban Choudhury, and Byron Boots. Blending MPC & Value Function Approximation for Efficient Reinforcement Learning. October 2020. URL https://openreview.net/forum?id=RqCC_00Bg7V

    2020

  5. [5]

    Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Robert Equi, Chelsea Finn, Niccolo Fusai, Manuel Y . Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Allen Z. Ren,...

  6. [6]

    Xinyue Chen, Che Wang, Zijian Zhou, and Keith W. Ross. Randomized Ensembled Double Q-Learning: Learning Fast Without a Model. October 2020. URL https://openreview.net/forum?id=AY8zfZm0tDd

    2020

  7. [7]

    Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, 44(10-11):1684–1704, September 2025

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, 44(10-11):1684–1704, September 2025. ISSN 0278-3649. doi: 10.1177/02783649241273668. URL https://doi.org/10.1177/02783649241273668...

    doi:10.1177/02783649241273668 2025

  8. [8]

    From Play to Policy: Condi- tional Behavior Generation from Uncurated Robot Data, December 2022

    Zichen Jeff Cui, Yibin Wang, Nur Muhammad Mahi Shafiullah, and Lerrel Pinto. From Play to Policy: Condi- tional Behavior Generation from Uncurated Robot Data, December 2022. URL http://arxiv.org/abs/2210.10047. arXiv:2210.10047 [cs]

    Pith/arXiv arXiv 2022

  9. [9]

    Model-Based Inverse Reinforcement Learning from Visual Demonstra- tions

    Neha Das, Sarah Bechtle, Todor Davchev, Dinesh Jayara- man, Akshara Rai, and Franziska Meier. Model-Based Inverse Reinforcement Learning from Visual Demonstra- tions. InProceedings of the 2020 Conference on Robot Learning, pages 1930–1942. PMLR, October 2021. URL https://proceedings.mlr.press/v155/das21a.html

    2020

  10. [10]

    Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization

    Chelsea Finn, Sergey Levine, and Pieter Abbeel. Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization. InProceedings of The 33rd Interna- tional Conference on Machine Learning, pages 49–58. PMLR, June 2016. URL https://proceedings.mlr.press/ v48/finn16.html

    2016

  11. [11]

    Learning Robust Rewards with Adverserial Inverse Reinforcement Learning

    Justin Fu, Katie Luo, and Sergey Levine. Learning Robust Rewards with Adverserial Inverse Reinforcement Learning. February 2018. URL https://openreview.net/ forum?id=rkHywl-A-

    2018

  12. [12]

    IQ-Learn: Inverse soft-Q Learning for Imitation

    Divyansh Garg, Shuvam Chakraborty, Chris Cundy, Jiaming Song, and Stefano Ermon. IQ-Learn: Inverse soft-Q Learning for Imitation. InAdvances in Neural Information Processing Systems, volume 34, pages 4028–4039. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper/2021/hash/ 210f760a89db30aa72ca258a3483cc7f-Abstract.html

    2021

  13. [13]

    Improved Training of Wasserstein GANs

    Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved Training of Wasserstein GANs. InAdvances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https: //proceedings.neurips.cc/paper_files/paper/2017/hash/ 892c3b1c6dccd52936e27cbd0ff683d6-Abstract.html

    2017

  14. [14]

    Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochas- tic Actor, August 2018

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochas- tic Actor, August 2018. URL http://arxiv.org/abs/1801. 01290. arXiv:1801.01290 [cs]

    Pith/arXiv arXiv 2018

  15. [15]

    Mastering diverse control tasks through world models.Nature, 640(8059):647–653, April 2025

    Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse control tasks through world models.Nature, 640(8059):647–653, April 2025. ISSN 1476-4687. doi: 10.1038/ s41586-025-08744-2. URL https://www.nature.com/ articles/s41586-025-08744-2

    2025

  16. [16]

    Model Predictive Adversarial Imitation Learn- ing for Planning from Observation, July 2025

    Tyler Han, Yanda Bao, Bhaumik Mehta, Gabriel Guo, Anubhav Vishwakarma, Emily Kang, Sanghun Jung, Rosario Scalise, Jason Zhou, Bryan Xu, and Byron Boots. Model Predictive Adversarial Imitation Learn- ing for Planning from Observation, July 2025. URL http://arxiv.org/abs/2507.21533. arXiv:2507.21533 [cs]

    arXiv 2025

  17. [17]

    TD- MPC2: Scalable, Robust World Models for Continuous 10 Control

    Nicklas Hansen, Hao Su, and Xiaolong Wang. TD- MPC2: Scalable, Robust World Models for Continuous 10 Control. October 2023. URL https://openreview.net/ forum?id=Oxh5CstDJU

    2023

  18. [18]

    Learn- ing Massively Multitask World Models for Continuous Control, December 2025

    Nicklas Hansen, Hao Su, and Xiaolong Wang. Learn- ing Massively Multitask World Models for Continuous Control, December 2025. URL http://arxiv.org/abs/2511. 19584. arXiv:2511.19584 [cs]

    arXiv 2025

  19. [19]

    Tactile Beyond Pixels: Multisensory Touch Representations for Robot Manipulation, June 2025

    Carolina Higuera, Akash Sharma, Taosha Fan, Chaithanya Krishna Bodduluri, Byron Boots, Michael Kaess, Mike Lambeta, Tingfan Wu, Zixi Liu, Francois Robert Hogan, and Mustafa Mukadam. Tactile Beyond Pixels: Multisensory Touch Representations for Robot Manipulation, June 2025. URL http://arxiv.org/abs/2506.14754. arXiv:2506.14754 [cs]

    Pith/arXiv arXiv 2025

  20. [20]

    Generative Adversarial Imitation Learning

    Jonathan Ho and Stefano Ermon. Generative Adversarial Imitation Learning. InAdvances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL https: //proceedings.neurips.cc/paper_files/paper/2016/hash/ cc7e2b878868cbae992d1fb743995d8f-Abstract.html

    2016

  21. [21]

    A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to Search

    Arnav Kumar Jain, Vibhakar Mohta, Subin Kim, Atiksh Bhardwaj, Juntao Ren, Yunhai Feng, Sanjiban Choud- hury, and Gokul Swamy. A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to Search. October 2025. URL https://openreview.net/ forum?id=qN5hmLkBtC

    2025

  22. [22]

    Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learning

    Ryan Julian, Benjamin Swanson, Gaurav Sukhatme, Sergey Levine, Chelsea Finn, and Karol Hausman. Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learning. InProceedings of the 2020 Conference on Robot Learning, pages 2120–

    2020

  23. [23]

    Discriminator-Actor-Critic: Addressing Sample Ineffi- ciency and Reward Bias in Adversarial Imitation Learn- ing

    Ilya Kostrikov, Kumar Krishna Agrawal, Debidatta Dwibedi, Sergey Levine, and Jonathan Tompson. Discriminator-Actor-Critic: Addressing Sample Ineffi- ciency and Reward Bias in Adversarial Imitation Learn- ing. September 2018. URL https://openreview.net/forum? id=Hk4fpoA5Km

    2018

  24. [24]

    MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation, May 2024

    Patrick Lancaster, Nicklas Hansen, Aravind Rajeswaran, and Vikash Kumar. MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation, May 2024. URL http://arxiv.org/abs/2309.14236. arXiv:2309.14236 [cs]

    Pith/arXiv arXiv 2024

  25. [25]

    Unifying Model Predic- tive Path Integral Control, Reinforcement Learning, and Diffusion Models for Optimal Control and Planning, February 2025

    Yankai Li and Mo Chen. Unifying Model Predic- tive Path Integral Control, Reinforcement Learning, and Diffusion Models for Optimal Control and Planning, February 2025. URL http://arxiv.org/abs/2502.20476. arXiv:2502.20476 [cs] version: 1

    Pith/arXiv arXiv 2025

  26. [26]

    Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation

    YuXuan Liu, Abhishek Gupta, Pieter Abbeel, and Sergey Levine. Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 1118–1125, May 2018. doi: 10.1109/ICRA.2018.8462901. URL https://ieeexplore. ieee.org/document/8462901/

    arXiv 2018

  27. [27]

    SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learn- ing, March 2025

    Jianlan Luo, Zheyuan Hu, Charles Xu, You Liang Tan, Jacob Berg, Archit Sharma, Stefan Schaal, Chelsea Finn, Abhishek Gupta, and Sergey Levine. SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learn- ing, March 2025. URL http://arxiv.org/abs/2401.16013. arXiv:2401.16013 [cs]

    Pith/arXiv arXiv 2025

  28. [28]

    Under- standing and Preventing Capacity Loss in Reinforcement Learning, May 2022

    Clare Lyle, Mark Rowland, and Will Dabney. Under- standing and Preventing Capacity Loss in Reinforcement Learning, May 2022. URL http://arxiv.org/abs/2204. 09560. arXiv:2204.09560 [cs]

    Pith/arXiv arXiv 2022

  29. [29]

    Fearing, Pieter Abbeel, Sergey Levine, and Chelsea Finn

    Anusha Nagabandi, Ignasi Clavera, Simin Liu, Ronald S. Fearing, Pieter Abbeel, Sergey Levine, and Chelsea Finn. Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning, February 2019. URL http://arxiv.org/abs/1803.11347. arXiv:1803.11347 [cs]

    Pith/arXiv arXiv 2019

  30. [30]

    Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL, January 2019

    Anusha Nagabandi, Chelsea Finn, and Sergey Levine. Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL, January 2019. URL http://arxiv.org/abs/1812.07671. arXiv:1812.07671 [cs]

    Pith/arXiv arXiv 2019

  31. [31]

    Cosmos World Foundation Model Platform for Physical AI, July 2025

    NVIDIA, Niket Agarwal, Arslan Ali, Maciej Bala, Yo- gesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chat- topadhyay, Yongxin Chen, Yin Cui, Yifan Ding, Daniel Dworakowski, Jiaojiao Fan, Michele Fenzi, Francesco Ferroni, Sanja Fidler, Dieter Fox, Songwei Ge, Yunhao Ge, Jinwei Gu, Siddharth Gururani, Ethan He, Jiahui Huang, Jacob Huffman, Pooya Jannaty, J...

    Pith/arXiv arXiv 2025

  32. [32]

    Carlson, Ji Yuan Feng, Animesh Garg, Renato Gasoto, Lionel Gulich, Yijie Guo, M

    NVIDIA, Mayank Mittal, Pascal Roth, James Tigue, Antoine Richard, Octi Zhang, Peter Du, Antonio Serrano- Muñoz, Xinjie Yao, René Zurbrügg, Nikita Rudin, Lukasz Wawrzyniak, Milad Rakhsha, Alain Denzler, Eric Heiden, Ales Borovicka, Ossama Ahmed, Ireti- ayo Akinola, Abrar Anwar, Mark T. Carlson, Ji Yuan Feng, Animesh Garg, Renato Gasoto, Lionel Gulich, Yiji...

    Pith/arXiv arXiv 2025

  33. [33]

    What Matters for Adversarial Imitation Learning? InAdvances in Neural Information Processing Systems, volume 34, pages 14656– 14668

    Manu Orsini, Anton Raichuk, Leonard Hussenot, Damien Vincent, Robert Dadashi, Sertan Girgin, Matthieu Geist, Olivier Bachem, Olivier Pietquin, and Marcin Andrychowicz. What Matters for Adversarial Imitation Learning? InAdvances in Neural Information Processing Systems, volume 34, pages 14656– 14668. Curran Associates, Inc., 2021. URL https: //proceedings....

    2021

  34. [34]

    Abby O’Neill, Abdul Rehman, Abhiram Maddukuri, Ab- hishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, Albert Tung, Alex Bewley, Alex Herzog, Alex Irpan, Alexander Khazatsky, Anant Rai, Anchit Gupta, Andrew Wang, Anikait Singh, Animesh Garg, Aniruddha Kemb- havi, Annie Xie, Anthony Brohan, Antonin Raffin,...

  35. [35]

    Much Ado About Noising: Dispelling the Myths of Generative Robotic Control, December 2025

    Chaoyi Pan, Giri Anantharaman, Nai-Chieh Huang, Claire Jin, Daniel Pfrommer, Chenyang Yuan, Frank Permenter, Guannan Qu, Nicholas Boffi, Guanya Shi, and Max Simchowitz. Much Ado About Noising: Dispelling the Myths of Generative Robotic Control, December 2025. URL http://arxiv.org/abs/2512.01809. arXiv:2512.01809 [cs]

    arXiv 2025

  36. [36]

    PyTorch: An Imperative Style, High-Performance Deep Learning Library, December 2019

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chin- tala. PyTorch: An Imperative Style, High-Perfor...

    Pith/arXiv arXiv 2019

  37. [37]

    AMP: adversarial motion priors for stylized physics-based character control.ACM Trans- actions on Graphics, 40(4):1–20, August 2021

    Xue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine, and Angjoo Kanazawa. AMP: adversarial motion priors for stylized physics-based character control.ACM Trans- actions on Graphics, 40(4):1–20, August 2021. ISSN 0730-0301, 1557-7368. doi: 10.1145/3450626.3459670. URL https://dl.acm.org/doi/10.1145/3450626.3459670

    arXiv 2021

  38. [38]

    Visual Adversarial Imitation Learning using Variational Models

    Rafael Rafailov, Tianhe Yu, Aravind Rajeswaran, and Chelsea Finn. Visual Adversarial Imitation Learning using Variational Models. InAdvances in Neural Information Processing Systems, volume 34, pages 3016–3028. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper/2021/hash/ 1796a48fa1968edd5c5d10d42c7b1813-Abstract.html

    2021

  39. [39]

    Random Features for Large-Scale Kernel Machines

    Ali Rahimi and Benjamin Recht. Random Features for Large-Scale Kernel Machines. In Advances in Neural Information Processing Systems, volume 20. Curran Associates, Inc., 2007. URL https://papers.nips.cc/paper_files/paper/2007/hash/ 013a006f03dbc5392effeb8f18fda755-Abstract.html

    2007

  40. [40]

    URL https://proceedings.mlr

    PMLR, October 2025. URL https://proceedings.mlr. press/v305/black25a.html

    2025

  41. [41]

    Parkour in the Wild: Learning a General and Extensible Agile Locomotion Policy Using Multi-expert Distillation and RL Fine-tuning, May 2025

    Nikita Rudin, Junzhe He, Joshua Aurand, and Marco Hutter. Parkour in the Wild: Learning a General and Extensible Agile Locomotion Policy Using Multi-expert Distillation and RL Fine-tuning, May 2025. URL http: //arxiv.org/abs/2505.11164. arXiv:2505.11164 [cs]

    Pith/arXiv arXiv 2025

  42. [42]

    Proximal Policy Optimization Algorithms, August 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms, August 2017. URL http://arxiv.org/abs/1707. 06347. arXiv:1707.06347 [cs]

    Pith/arXiv arXiv 2017

  43. [43]

    Latent Plans for Task-Agnostic Offline Reinforcement Learning

    Erick Rosete-Beas, Oier Mees, Gabriel Kalweit, Joschka Boedecker, and Wolfram Burgard. Latent Plans for Task-Agnostic Offline Reinforcement Learning. InPro- ceedings of The 6th Conference on Robot Learning, pages 1838–1849. PMLR, March 2023. URL https: //proceedings.mlr.press/v205/rosete-beas23a.html

    2023

  44. [44]

    Reinforcement Learning: An Introduction

    Richard S Sutton and Andrew G Barto. Reinforcement Learning: An Introduction

  45. [45]

    Sample-efficient Adversarial Imitation Learning from Observation, June 2019

    Faraz Torabi, Sean Geiger, Garrett Warnell, and Peter Stone. Sample-efficient Adversarial Imitation Learning from Observation, June 2019. URL http://arxiv.org/abs/ 1906.07374. arXiv:1906.07374 [cs]

    Pith/arXiv arXiv 2019

  46. [46]

    Adversarial Inverse Reinforcement Learning With Self-Attention Dynamics Model.IEEE Robotics and Automation Letters, 6(2):1880–1886, April 2021

    Jiankai Sun, Lantao Yu, Pinqian Dong, Bo Lu, and Bolei Zhou. Adversarial Inverse Reinforcement Learning With Self-Attention Dynamics Model.IEEE Robotics and Automation Letters, 6(2):1880–1886, April 2021. ISSN 2377-3766. doi: 10.1109/LRA.2021.3061397. URL https://ieeexplore.ieee.org/document/9361118/

    arXiv 2021

  47. [47]

    Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U. Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, Ro- drigo Perez-Vicente, Andrea Pierré, Sander Schulhoff, Jun Jet Tai, Hannah Tan, and Omar G. Younis. Gymna- sium: A Standard Interface for Reinforcement Learning Environments, November 2025. URL http...

    Pith/arXiv arXiv 2025

  48. [48]

    DiffAIL: Diffusion Adversarial Imitation Learning.Proceedings of the AAAI Conference on Artificial Intelligence, 38(14):15447–15455, March 2024

    Bingzheng Wang, Guoqiang Wu, Teng Pang, Yan Zhang, and Yilong Yin. DiffAIL: Diffusion Adversarial Imitation Learning.Proceedings of the AAAI Conference on Artificial Intelligence, 38(14):15447–15455, March 2024. ISSN 2374-3468. doi: 10.1609/aaai.v38i14.29470. URL https://ojs.aaai.org/index.php/AAAI/article/view/29470

    doi:10.1609/aaai.v38i14.29470 2024

  49. [49]

    Genera- tive Adversarial Imitation from Observation, June 2019

    Faraz Torabi, Garrett Warnell, and Peter Stone. Genera- tive Adversarial Imitation from Observation, June 2019. URL http://arxiv.org/abs/1807.06158. arXiv:1807.06158 [cs]

    Pith/arXiv arXiv 2019

  50. [50]

    Rehg, and Evangelos A

    Grady Williams, Paul Drews, Brian Goldfain, James M. Rehg, and Evangelos A. Theodorou. Information- Theoretic Model Predictive Control: Theory and Appli- cations to Autonomous Driving.IEEE Transactions on Robotics, 34(6):1603–1622, December 2018. ISSN 1552- 3098, 1941-0468. doi: 10.1109/TRO.2018.2865891. URL https://ieeexplore.ieee.org/document/8558663/

    arXiv 2018

  51. [51]

    XIRL: Cross-embodiment Inverse Reinforcement Learning

    Kevin Zakka, Andy Zeng, Pete Florence, Jonathan Tomp- son, Jeannette Bohg, and Debidatta Dwibedi. XIRL: Cross-embodiment Inverse Reinforcement Learning. In Proceedings of the 5th Conference on Robot Learning, pages 537–546. PMLR, January 2022. URL https: //proceedings.mlr.press/v164/zakka22a.html

    2022

  52. [52]

    MimicPlay: Long-Horizon Imitation Learning by Watching Human Play, October 2023

    Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu, and Anima Anand- kumar. MimicPlay: Long-Horizon Imitation Learning by Watching Human Play, October 2023. URL http: //arxiv.org/abs/2302.12422. arXiv:2302.12422 [cs]

    Pith/arXiv arXiv 2023

  53. [55]

    somersaulting

    Jiahui Zhang, Yusen Luo, Abrar Anwar, Sumedh Anand Sontakke, Joseph J. Lim, Jesse Thomason, Erdem Biyik, and Jesse Zhang. ReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrations. In Proceedings of The 9th Conference on Robot Learning, pages 460–488. PMLR, October 2025. URL https: //proceedings.mlr.press/v305/zhang25a.html. 13 APPEN...

    2025

  54. [56]

    Real World Push Setup:The real-world Push experi- ments are conducted on a Franka robotic arm. The observation space includes64×64RGB images from a fixed table- top RGB camera (Intel RealSense D435i), a wrist-mounted RGB camera (Intel RealSense D435i) rigidly attached to the arm’s wrist and proprioception including joint position (7), joint velocities (7)...

  55. [57]

    Influence

    Real World Pick-and-Place Setup:All real-world Pick- and-Place experiments are conducted on a Kinova Gen3 6- DoF robotic arm equipped with a Robotiq 2F-85 gripper. The observation space includes64×64RGB images from a fixed table-top RGB camera (Intel RealSense D435i), a wrist- mounted RGB camera (Intel RealSense D410) rigidly attached to the arm’s wrist i...

  56. [2024]

    URL https://ieeexplore.ieee.org/document/10611477/

    doi: 10.1109/ICRA57147.2024.10611477. URL https://ieeexplore.ieee.org/document/10611477/

    arXiv 2024

  57. [2136]

    URL https://proceedings

    PMLR, October 2021. URL https://proceedings. mlr.press/v155/julian21a.html

    2021