Pith. sign in

REVIEW 3 major objections 4 minor 4 cited by

Conditioning a pretrained video diffusion model on robot-only motion renderings turns a handful of human demonstrations into large, diverse training sets that improve downstream imitation learning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 16:44 UTC pith:MZB77R37

load-bearing objection AnchorDream's robot-motion-conditioned video synthesis is a real step forward for imitation data scaling; the simulated evidence is strong, but the real-world doubling rests on an unvalidated trajectory expansion. the 3 major comments →

arxiv 2512.11797 v2 pith:MZB77R37 submitted 2025-12-12 cs.RO cs.CV

AnchorDream: Repurposing Video Diffusion for Embodiment-Aware Robot Data Synthesis

Junjie Ye , Rong Xue , Basile Van Hoorick , Pavel Tokmakov , Muhammad Zubair Irshad , Yue Wang , Vitor Guizilini This is my paper
classification cs.RO cs.CV
keywords AnchorDreamvideo diffusionrobot data synthesisimitation learningembodiment groundingtrajectory expansiondemonstration generationworld model
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

AnchorDream repurposes a pretrained video diffusion model as an embodiment-aware world model. Instead of asking the model to imagine the whole scene including the robot, the researchers render only the robot's arm motion from a proposed trajectory and condition the video model on that render. The model then fills in objects and environments consistent with the motion, producing photorealistic demonstration videos. Downstream imitation policies trained on these synthesized demos improve: average success rises from 22.5% to 30.7% in a standard tabletop benchmark (a 36% relative gain) and from 28% to 63% across six real-world tasks using roughly ten times the generated data. The core claim is that grounding generation in robot motion preserves embodiment and trajectory consistency while diversifying both observations and behaviors.

Core claim

The paper's central claim is that anchoring a video diffusion model on robot-only motion renderings — trajectories rendered deterministically as clean robot-arm motion videos, with no objects or backgrounds — preserves embodiment fidelity and produces demonstrations whose environment and object layout match the trajectory. The generated data, when added to the original human demonstrations, consistently improves downstream imitation learning, approaching the performance of simulator-executed data expansion without requiring explicit environment modeling or simulator rollouts.

What carries the argument

The core mechanism is the decoupled trajectory–environment synthesis: robot trajectories are expanded and rendered deterministically as robot-only motion videos, which are concatenated with the noisy input to a pretrained video diffusion model as an additional conditioning signal. A global trajectory embedding is also provided, letting the model anticipate future waypoints during long-horizon autoregressive generation. The robot-only render anchors the embodiment, preventing hallucinated robot bodies, while the video model's large-scale visual priors supply plausible objects and scenes consistent with the motion.

Load-bearing premise

The heuristic trajectory expansion — shifting contact points and stitching object-centric sub-trajectories — is assumed to produce kinematically feasible, task-relevant trajectories, but no simulator execution, inverse-kinematics check, or task-success filter is applied to validate them.

What would settle it

Run the augmented trajectories through a physics simulator with the task's objects; if a large fraction fail the task (e.g., missing the target or colliding), and removing those failed trajectories from the generated training set eliminates the downstream improvement, then the claim that the synthesized data teaches correct action-object associations is undermined.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Scaling imitation learning no longer requires massive real-world data collection or explicit simulator environments; a handful of teleoperated demonstrations can seed large, diverse training sets.
  • The gap between purely visual augmentation (which fixes trajectories) and simulator-based data generation (which needs environment assets) is narrowed: AnchorDream expands both behaviors and observations without simulation rollouts.
  • Because the outputs are direct visual observations paired with known trajectories, they can be used by a range of policy architectures without an inverse dynamics step.
  • The synthesized scenes are photorealistic and varied, inheriting the video model's broad visual priors, which can improve policy robustness in real-world conditions.
  • The approach may extend beyond tabletop manipulation to any domain where a robot's motion can be rendered cleanly and conditioned upon.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is to validate the heuristically expanded trajectories in simulation: if an inverse-kinematics or physics check were applied and infeasible trajectories discarded, downstream success rates might rise further, isolating the value of the video model's rendering.
  • The decoupling principle suggests that any trajectory source — including model-based planners or reinforcement-learned policies — could be used in place of heuristic perturbation, potentially broadening the behavior distribution beyond what manual perturbation yields.
  • One could probe how much of the gain comes from visual diversity versus action diversity by ablating the trajectory expansion: generate multiple scenes conditioned on the same trajectory. If visual diversity alone drives most of the improvement, the method's value is more about appearance augmentation than behavior creation.
  • The explicit global trajectory conditioning offers a route to long-horizon tasks: keeping the full planned motion as a condition may allow coherent scene layouts over minutes of generated video, beyond the demonstrated frame windows.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes AnchorDream, a data-synthesis pipeline that repurposes a pretrained video diffusion model as an 'embodiment-aware world model.' Starting from a handful of human teleoperated demonstrations, AnchorDream (i) expands the trajectory set by perturbing key states and stitching object-centric segments, (ii) renders only the robot-arm motion from those trajectories, and (iii) conditions Cosmos-Predict2 on these robot-only motion videos plus language and global trajectory embeddings to synthesize photorealistic demonstration videos. The synthesized videos are then used, together with the seed data, to train downstream imitation policies. In RoboCasa, adding 300 generated demonstrations per task to 50 human demonstrations raises average success from 22.5% to 30.7%, approaching the 33.3% of a MimicGen-based oracle; on six real-world PiPER tasks, adding 10x generated data raises average success from 28% to 63%. Ablations indicate that global trajectory conditioning and longer inference windows are both beneficial.

Significance. If the empirical findings hold, AnchorDream is a practically valuable method for scaling imitation-learning data without reconstructing scenes or running simulator rollouts. The simulation experimental design is particularly informative: Table I compares policies trained on the same augmented trajectory set rendered either by simulator execution (MimicGen oracle) or by video diffusion (AnchorDream), which isolates the contribution of visual synthesis. The ablation in Table III provides direct evidence for the paper's two main design claims. The method also has a clear conceptual idea—decouple trajectory and environment rendering—that is easy to communicate and could generalize. However, the paper's own central claims rest on empirical evaluations that currently lack error bars, and the real-world component relies on an unvalidated heuristic trajectory expansion step. These gaps are fixable but should be addressed before the paper is accepted.

major comments (3)
  1. [§III-B2/§IV-E] §III-B2/§IV-E: real-world trajectory expansion is unvalidated. §III-B2 asserts perturbing/stitching key states 'produces a large pool of trajectories that remain feasible,' but no IK, collision, simulator, or success check is reported. In RoboCasa (§IV-B) the expanded trajectories are simulator-validated by MimicGen, so the concern does not affect that result. In §IV-E, key states are perturbed up to ±10 cm and segments stitched, then rendered directly into video without validation. If a shifted contact misses the object, the video model can hallucinate an object at the gripper and produce observation–action pairs that teach wrong associations. Since the real-world doubling (28→63%) is half the evidence, this gap is load-bearing. Add validation statistics (IK/replay/manual acceptance) and ideally filter or ablate invalid trajectories.
  2. [Tables I, III, IV and Fig. 5] Tables I, III, IV and Fig. 5: no error bars, confidence intervals, or number of seeds accompany the headline success rates. Several per-task margins are small (Table I: insertion 12.0→14.0; pressing buttons 12.0→14.0), and the real-world evaluation uses 20 rollouts per task, so one rollout is 5 percentage points. Without repeated seeds or interval estimates, the claims of 'consistent improvements' (22.5→30.7 in Table I; 28→63 in Table IV) cannot be distinguished from noise. Report mean±std over at least three random seeds, or exact binomial confidence intervals for real-world counts.
  3. [Table II] Table II: the comparison to DreamGen uses a number taken from the original paper and thus is not a controlled experiment. The presentation suggests AnchorDream300 is competitive with DreamGen10K, but differences in policy architecture, task suite, and evaluation protocol make the comparison uninformative. Either evaluate DreamGen under the same protocol or label the row as external literature and soften the comparative statement.
minor comments (4)
  1. [§IV-E] Numerical inconsistency: the text says adding 10x AnchorDream data 'raises this to 60.0%,' but Table IV reports an average of 63% for the AnchorDream500 condition. Please correct one of the two.
  2. [§III-B4] The conditioning mechanism in Eq. (5) is described only informally. Please specify how the global trajectory τ′ is embedded and how the binary window indicator φ is combined with language embeddings.
  3. [§IV-D.2] When the inference window is shortened from 189 to 93 frames, it would help to state whether the same autoregressive overlap/context is used in both conditions, so the ablation isolates window length rather than context handling.
  4. [§IV-A] Training details for the downstream policies are minimal (BC-Transformer for simulation, Diffusion Policy for real). Please list hyperparameters, observation/action horizons, and training steps, since the data-efficiency claims depend on these choices.

Circularity Check

0 steps flagged

No significant circularity: downstream policy evaluation is external to the synthesis pipeline; real-world trajectory-expansion validation is a correctness risk, not a circular step.

full rationale

AnchorDream's derivation chain is not circular. The seed demonstrations provide human trajectories with paired observations; trajectory expansion τ'=T(τ) is a heuristic adopted from external prior work (MimicGen, DemoGen), not derived from the video model. The rendered motion r=Render(τ') is an input conditioning signal, and the video model pθ(o|r,l,τ',φ) is fine-tuned on the seed set's (r,o) pairs, then evaluated by whether the synthesized observations, paired with τ', improve a policy on held-out simulator and real-robot rollouts. Those success metrics are external to the synthesis process and are not used as fitted targets when constructing the demonstrations. The MimicGen300 comparison is an oracle executed in simulation, and AnchorDream is not fitted to that oracle. The unvalidated real-world perturbation step ('This produces a large pool of trajectories that remain feasible under the robot's embodiment', §III-B2) is a missing-feasibility-validation risk rather than a definitional equivalence. Self-citations (e.g., Real2Render2Real [25]) appear only as related-work context and are not load-bearing; no uniqueness theorem or ansatz is imported from the authors' prior work to force the result. The mildest concern is benchmark-driven design tuning (global trajectory conditioning, inference window) evaluated on the same benchmark, but this is ordinary engineering iteration, not circularity.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

No new physical entities are introduced. The paper's contributions hinge on the choice of conditioning signal (robot-only rendered motion) and the heuristic trajectory expansion, whose validity is assumed rather than independently verified. The main free parameters are the perturbation ranges, generation count, and window length, all hand-set.

free parameters (3)
  • Key-state perturbation range = up to ±10 cm in the horizontal plane (real world); unspecified in simulation
    Controls diversity and plausibility of expanded trajectories; hand-chosen, with no stated justification or validation that perturbed states remain task-valid.
  • Augmentation count K per seed = 300 per task in simulation; 500 per task (10x) in real world
    Dataset scale is a chosen experimental knob; performance is still rising at the largest tested scale (1000 in Fig. 5), so the choice is not principled and could be a free parameter of the reported gains.
  • Generation window length = 189 frames in simulation; 180 frames (real world), with 93-frame ablation
    The paper shows longer windows improve downstream success, and the specific window length is a hand-set design choice rather than derived from theory or data.
axioms (4)
  • domain assumption Cosmos-Predict2 2B contains sufficient world priors for tabletop manipulation scenes after LoRA fine-tuning.
    The entire synthesis pipeline depends on the pretrained video diffusion model correctly filling in objects, layouts, and environments from the robot-only motion traces (§III-B4).
  • ad hoc to paper Heuristic trajectory expansion produces kinematically feasible and task-valid trajectories without any validation step.
    The paper asserts trajectories 'remain feasible under the robot's embodiment' (§III-B2) but does not describe IK checks, simulator rollouts, success filters, or human verification.
  • domain assumption RoboCasa/RoboTwin renderings of the robot arm faithfully represent the true embodiment and camera geometry.
    Robot-only rendering assumes the URDF model, camera intrinsics/extrinsics, and rendering pipeline exactly project the augmented trajectory into the conditioning video (§III-B3).
  • domain assumption LoRA fine-tuning on roughly 50 demonstrations per task is sufficient to adapt the channel-expanded video diffusion model.
    The first layer's input channels are doubled for the concatenated motion trace, and the model is adapted with LoRA on a very small seed set; the paper does not analyze coverage, forgetting, or whether the base model's priors remain intact.

pith-pipeline@v1.3.0-alltime-deepseek · 10674 in / 10926 out tokens · 113334 ms · 2026-08-03T16:44:29.737341+00:00 · methodology

0 comments
read the original abstract

The collection of large-scale and diverse robot demonstrations remains a major bottleneck for imitation learning, as real-world data acquisition is costly and simulators offer limited diversity and fidelity with pronounced sim-to-real gaps. While generative models present an attractive solution, existing methods often alter only visual appearances without creating new behaviors, or suffer from embodiment inconsistencies that yield implausible motions. To address these limitations, we introduce AnchorDream, an embodiment-aware world model that repurposes pretrained video diffusion models for robot data synthesis. AnchorDream conditions the diffusion process on robot motion renderings, anchoring the embodiment to prevent hallucination while synthesizing objects and environments consistent with the robot's kinematics. Starting from only a handful of human teleoperation demonstrations, our method scales them into large, diverse, high-quality datasets without requiring explicit environment modeling. Experiments show that the generated data leads to consistent improvements in downstream policy learning, with relative gains of 36.4% in simulator benchmarks and nearly double performance in real-world studies. These results suggest that grounding generative world models in robot motion provides a practical path toward scaling imitation learning.

Figures

Figures reproduced from arXiv: 2512.11797 by Basile Van Hoorick, Junjie Ye, Muhammad Zubair Irshad, Pavel Tokmakov, Rong Xue, Vitor Guizilini, Yue Wang.

Figure 1
Figure 1. Figure 1: Overview of AnchorDream. AnchorDream repurposes a pretrained video diffusion model as an embodiment-aware world model. Conditioned on robot motion videos, the model anchors the robot embodiment to prevent hallucination while synthesizing objects and environments consistent with the motion, enabling large-scale, high-quality demonstration generation from only a few real demonstrations. the motion space, gen… view at source ↗
Figure 2
Figure 2. Figure 2: Outline of our proposed AnchorDream. Starting from a small set of human teleoperated demonstrations, new trajectories are created by perturbing key states and recombining motion segments to ensure kinematic feasibility. Each augmented trajectory is rendered as a robot-only motion video, which, together with the task description, conditions AnchorDream to synthesize realistic demonstrations where environmen… view at source ↗
Figure 3
Figure 3. Figure 3: Effect of missing global trajectory conditioning. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative results on RoboCasa. Comparison between rendered robot motion inputs, generated demonstrations, and ground-truth scenes across several tasks. The synthesized demonstrations preserve robot embodiment while producing diverse and visually coherent environments with object placements and interactions that align with the intended motions. These examples illustrate how AnchorDream translates abstract… view at source ↗
Figure 5
Figure 5. Figure 5: Effect of scaling AnchorDream-generated data. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Real-world evaluation setup. Six everyday manipulation tasks are used in our real-world evaluation: sweeping coffee beans with a brush, grasping and tilting a cup to pour into a bowl, closing a drawer, opening a drawer, placing a toy in a plate, and placing a book on a shelf. The lower panel shows the PiPER robot platform used for data collection and evaluation [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Real-robot qualitative results. Example visualizations of synthesized demonstrations for several tasks. Each column shows the original trajectory (Origin) and several augmented variants (Aug1–Aug4). The generated demonstrations remain visually realistic, while the augmented trajectories steer scene layouts to diversify object positions and interactions, providing greater variability in the training data co… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. 3D Generation for Embodied AI and Robotic Simulation: A Survey

    cs.RO 2026-04 accept novelty 7.0

    3D generation for embodied AI is shifting from visual realism toward interaction readiness, organized into data generation, simulation environments, and sim-to-real bridging roles.

  2. TSD: A Physics-Inspired Trajectory Saliency Detector for Efficient Imitation Learning

    cs.RO 2026-06 unverdicted novelty 5.0

    TSD applies two physics metrics to identify salient trajectory segments for dataset compression and expansion in robotic imitation learning, yielding comparable performance with 25% less data on average.

  3. 3D Generation for Embodied AI and Robotic Simulation: A Survey

    cs.RO 2026-04 unverdicted novelty 3.0

    The survey organizes 3D generation for embodied AI into data generators for assets, simulation environments for interaction, and sim-to-real bridges, noting a shift toward interaction readiness and listing bottlenecks...

  4. 3D Generation for Embodied AI and Robotic Simulation: A Survey

    cs.RO 2026-04 unverdicted novelty 2.0

    The paper surveys 3D generation techniques for embodied AI and robotics, categorizing them into data generation, simulation environments, and sim-to-real bridging while identifying bottlenecks in physical validity and...

Reference graph

Works this paper leans on

31 extracted references · 5 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware,” inProceedings of the Robotics: Science and Systems (RSS), 2023

    2023

  2. [2]

    Diffusion Policy: Visuomotor Policy Learning via Action Diffusion,

    C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion Policy: Visuomotor Policy Learning via Action Diffusion,” inProceedings of the Robotics: Science and Systems (RSS), 2023

    2023

  3. [3]

    Open X-Embodiment: Robotic Learning Datasets and RT-X Models,

    Open X-Embodiment Collaboration et al., “Open X-Embodiment: Robotic Learning Datasets and RT-X Models,” inProceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2024

    2024

  4. [4]

    DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset,

    A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, and S. D. et al., “DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset,” inProceedings of Robotics: Science and Systems (RSS), 2024

    2024

  5. [5]

    A Careful Examination of Large Behav- ior Models for Multitask Dexterous Manipulation,

    TRI LBM Team et al., “A Careful Examination of Large Behav- ior Models for Multitask Dexterous Manipulation,”arXiv preprint arXiv:2507.05331, 2025

    Pith/arXiv arXiv 2025

  6. [6]

    π 0: A Vision-Language-Action Flow Model for General Robot Control,

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinsky, “π 0: A Vision-Language-Action Flow Model for General Robot Control,” in Proceedings of the Ro...

    2025

  7. [7]

    GR00T N1: An Open Foundation Model for Generalist Humanoid Robots,

    NVIDIA, “GR00T N1: An Open Foundation Model for Generalist Humanoid Robots,”arXiv preprint arXiv:2503.14734, 2025

    Pith/arXiv arXiv 2025

  8. [8]

    Data Scaling Laws in Imitation Learning for Robotic Manipulation,

    F. Lin, Y . Hu, P. Sheng, C. Wen, J. You, and Y . Gao, “Data Scaling Laws in Imitation Learning for Robotic Manipulation,” inProceedings of the International Conference on Learning Representations (ICLR), 2024

    2024

  9. [9]

    Scaling Robot Learning with Semantically Imagined Experience,

    T. Yu, T. Xiao, A. Stone, J. Tompson, A. Brohan, S. Wang, J. Singh, C. Tan, M. Dee, J. Peralta, B. Ichter, K. Hausman, and F. Xia, “Scaling Robot Learning with Semantically Imagined Experience,” in Proceedings of the Robotics: Science and Systems (RSS), 2023

    2023

  10. [10]

    Robo- Engine: Plug-and-Play Robot Data Augmentation with Semantic Robot Segmentation and Background Generation,

    C. Yuan, S. Joshi, S. Zhu, H. Su, H. Zhao, and Y . Gao, “Robo- Engine: Plug-and-Play Robot Data Augmentation with Semantic Robot Segmentation and Background Generation,” inProceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025

    2025

  11. [11]

    ReBot: Scaling Robot Learning with Real-to-Sim-to-Real Robotic Video Synthesis,

    Y . Fang, Y . Yang, X. Zhu, K. Zheng, G. Bertasius, D. Szafir, and M. Ding, “ReBot: Scaling Robot Learning with Real-to-Sim-to-Real Robotic Video Synthesis,” inProceedings of the IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS), 2025

    2025

  12. [12]

    Imagen Editor and Edit- Bench: Advancing and Evaluating Text-Guided Image Inpainting,

    S. Wang, C. Saharia, C. Montgomery, J. Pont-Tuset, S. Noy, S. Pel- legrini, Y . Onoe, S. Laszlo, D. J. Fleet, R. Soricut, J. Baldridge, M. Norouzi, P. Anderson, and W. Chan, “Imagen Editor and Edit- Bench: Advancing and Evaluating Text-Guided Image Inpainting,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  13. [13]

    Video Diffusion Models,

    J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet, “Video Diffusion Models,” inProceedings of the Advances in Neural Information Processing Systems (NeurIPS), vol. 35, 2022

    2022

  14. [14]

    MimicGen: A Data Generation System for Scal- able Robot Learning using Human Demonstrations,

    A. Mandlekar, S. Nasiriany, B. Wen, I. Akinola, Y . Narang, L. Fan, Y . Zhu, and D. Fox, “MimicGen: A Data Generation System for Scal- able Robot Learning using Human Demonstrations,” inProceedings of the Conference on Robot Learning (CoRL), 2023

    2023

  15. [15]

    Demogen: Synthetic demonstration generation for data-efficient visuomotor pol- icy learning,

    Z. Xue, S. Deng, Z. Chen, Y . Wang, Z. Yuan, and H. Xu, “Demogen: Synthetic demonstration generation for data-efficient visuomotor pol- icy learning,” inProceedings of the Robotics: Science and Systems (RSS), 2025

    2025

  16. [16]

    Wan: Open and Advanced Large-Scale Video Generative Models,

    Team Wan, “Wan: Open and Advanced Large-Scale Video Generative Models,”arXiv preprint arXiv:2503.20314, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Cosmos World Foundation Model Platform for Physical AI,

    NVIDIA, “Cosmos World Foundation Model Platform for Physical AI,”arXiv preprint arXiv:2501.03575, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Habitat: A Platform for Embodied AI Research,

    M. Savva, A. Kadian, O. Maksymets, Y . Zhao, E. Wijmans, B. Jain, J. Straub, J. Liu, V . Koltun, J. Malik, D. Parikh, and D. Batra, “Habitat: A Platform for Embodied AI Research,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019

    2019

  19. [19]

    ThreeD- World: A Platform for Interactive Multi-Modal Physical Simulation,

    C. Gan, J. Schwartz, S. Alter, D. Mrowca, and M. S. et al., “ThreeD- World: A Platform for Interactive Multi-Modal Physical Simulation,” inProceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2021

    2021

  20. [20]

    Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World,

    J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World,” inProceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017

    2017

  21. [21]

    Reinforcement Learning with Augmented Data,

    M. Laskin, K. Lee, A. Stooke, L. Pinto, P. Abbeel, and A. Srinivas, “Reinforcement Learning with Augmented Data,” inProceedings of the Advances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020

    2020

  22. [22]

    Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels,

    D. Yarats, I. Kostrikov, and R. Fergus, “Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels,” inProceedings of the International Conference on Learning Repre- sentations (ICLR), 2021

    2021

  23. [23]

    Cosmos-Predict2: General-Purpose World Founda- tion Models for Physical AI,

    N. Cosmos, “Cosmos-Predict2: General-Purpose World Founda- tion Models for Physical AI,” https://github.com/nvidia-cosmos/ cosmos-predict2, 2025, apache License 2.0, accessed December 15, 2025

    2025

  24. [24]

    Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control,

    H. Abu Alhaija, J. Alvarez, M. Bala, T. Cai, T. Cao, L. Cha, J. Chen, M. Chen, F. Ferroni, S. Fidler, D. Fox, Y . Ge, J. Gu, A. Hassani, M. Isaev, P. Jannaty, S. Lan, T. Lasser, H. Ling, M.- Y . Liu, X. Liu, Y . Lu, A. Luo, Q. Ma, H. Mao, F. Ramos, X. Ren, T. Shen, S. Tang, T.-C. Wang, J. Wu, J. Xu, S. Xu, K. Xie, Y . Ye, X. Yang, X. Zeng, and Y . Zeng, “...

    Pith/arXiv arXiv 2025

  25. [25]

    Real2Render2Real: Scaling Robot Data Without Dynamics Simulation or Robot Hardware,

    J. Yu, L. Fu, H. Huang, K. El-Refai, R. A. Ambrus, R. Cheng, M. Z. Irshad, and K. Goldberg, “Real2Render2Real: Scaling Robot Data Without Dynamics Simulation or Robot Hardware,” inProceedings of the Conference on Robot Learning (CoRL), 2025

    2025

  26. [26]

    DreamGen: Unlocking Generalization in Robot Learning through Video World Models,

    J. Jang, S. Ye, Z. Lin, J. Xiang, J. Bjorck, Y . Fang, F. Hu, S. Huang, K. Kundalia, Y .-C. Lin, L. Magne, A. Mandlekar, A. Narayan, Y . L. Tan, G. Wang, J. Wang, Q. Wang, Y . Xu, X. Zeng, K. Zheng, R. Zheng, M.-Y . Liu, L. Zettlemoyer, D. Fox, J. Kautz, S. Reed, Y . Zhu, and L. Fan, “DreamGen: Unlocking Generalization in Robot Learning through Video Worl...

    2025

  27. [27]

    IRASim: A Fine-Grained World Model for Robot Manipulation,

    F. Zhu, H. Wu, S. Guo, Y . Liu, C. Cheang, and T. Kong, “IRASim: A Fine-Grained World Model for Robot Manipulation,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025

    2025

  28. [28]

    RoboCasa: Large-Scale Simulation of Ev- eryday Tasks for Generalist Robots,

    S. Nasiriany, A. Maddukuri, L. Zhang, A. Parikh, A. Lo, A. Joshi, A. Mandlekar, and Y . Zhu, “RoboCasa: Large-Scale Simulation of Ev- eryday Tasks for Generalist Robots,” inProceedings of the Robotics: Science and Systems (RSS), 2024

    2024

  29. [29]

    LoRA: Low-Rank Adaptation of Large Language Models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-Rank Adaptation of Large Language Models,” inProceedings of the International Conference on Learning Representations (ICLR), 2022

    2022

  30. [30]

    RoboTwin: Dual- Arm Robot Benchmark with Generative Digital Twins,

    Y . Mu, T. Chen, Z. Chen, S. Peng, Z. Lan, Z. Gao, Z. Liang, Q. Yu, Y . Zou, M. Xu, L. Lin, Z. Xie, M. Ding, and P. Luo, “RoboTwin: Dual- Arm Robot Benchmark with Generative Digital Twins,” inProceedings of the Computer Vision and Pattern Recognition Conference (CVPR), 2025

    2025

  31. [31]

    What Matters in Learning from Offline Human Demonstrations for Robot Manipu- lation,

    A. Mandlekar, D. Xu, J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, S. Savarese, Y . Zhu, and R. Mart´ın-Mart´ın, “What Matters in Learning from Offline Human Demonstrations for Robot Manipu- lation,” inProceedings of the Conference on Robot Learning (CoRL), vol. 164, 2022

    2022