pith. machine review for the scientific record. sign in

arxiv: 2510.08547 · v2 · submitted 2025-10-09 · 💻 cs.RO · cs.CV

R2RGEN: Real-to-Real 3D Data Generation for Spatially Generalized Manipulation

Xiuwei Xu , Angyuan Ma , Hankun Li , Bingyao Yu , Zheng Zhu , Jie Zhou , Jiwen Lu This is my paper

Pith reviewed 2026-05-18 08:43 UTC · model grok-4.3

classification 💻 cs.RO cs.CV
keywords real-to-real data generation3D pointcloud augmentationspatial generalizationrobotic manipulationimitation learningmobile manipulationvisuomotor policiesdata efficiency
0
0 comments X

The pith

Real-to-real 3D augmentation generates spatially diverse robotic data from minimal demonstrations without simulation.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper seeks to achieve spatial generalization in robotic manipulation policies, which requires handling varied positions of objects, environments, and the robot itself. It introduces R2RGen as a framework that directly augments real pointcloud observation-action pairs to create more diverse training data. This is done through a three-stage process that parses demonstrations into a shared 3D space, augments positions using group-wise backtracking, and aligns the output with real sensor characteristics via post-processing. The result is improved data efficiency for imitation learning, with experiments showing promise for mobile manipulation applications where new data collection is costly.

Core claim

R2RGen provides a unified three-stage real-to-real framework that pre-processes source demonstrations in a shared 3D space with scene and trajectory parsing, augments object and robot positions with a group-wise backtracking strategy, and applies camera-aware post-processing to align generated data with real-world 3D sensor distributions, producing pointcloud-action pairs suitable for training generalized visuomotor policies.

What carries the argument

The unified three-stage framework consisting of 3D space pre-processing, group-wise backtracking augmentation, and camera-aware post-processing that directly operates on real pointcloud data.

If this is right

  • Policies trained with R2RGen data exhibit robust performance under different spatial configurations of objects and the agent.
  • The approach substantially reduces the volume of human demonstrations required for effective imitation learning.
  • It demonstrates applicability and potential for scaling in mobile manipulation scenarios.
  • It operates without simulators or rendering, making it efficient and compatible with existing real datasets.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If backtracking maintains physical consistency, the method could be extended to generate data for dynamic interactions beyond static repositioning.
  • This real-to-real strategy might inspire similar augmentation techniques in other sensor modalities like RGB images for robotics.
  • Testing on a wider range of robot embodiments would reveal how well the 3D parsing generalizes across hardware.

Load-bearing premise

The pointcloud-action pairs produced by the augmentation process match real 3D sensor statistics closely enough that learned policies generalize without introducing new artifacts or executing invalid actions.

What would settle it

Observe whether a visuomotor policy trained solely on R2RGen-augmented data from limited source demonstrations successfully manipulates objects in real-world setups with novel spatial arrangements that were not present in the originals; repeated failures or invalid actions would indicate the distributions do not match sufficiently.

Figures

Figures reproduced from arXiv: 2510.08547 by Angyuan Ma, Bingyao Yu, Hankun Li, Jie Zhou, Jiwen Lu, Xiuwei Xu, Zheng Zhu.

Figure 1
Figure 1. Figure 1: R2RGen is a simulator-free data generation framework. Given one human-collected [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Pre-processing results. The 3D scene is parsed [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The pipeline of R2RGen. Given processed source demonstration, we backtrack skills and apply group-wise augmentation to maintain the spatial relationships among target objects, where a fixed object set is maintained to judge whether the augmentation is applicable. Then motion planning is performed to generate trajectories that connect adjacent skills. After augmentation, we perform camera-aware processing t… view at source ↗
Figure 4
Figure 4. Figure 4: Visualization of our real-world tasks. We show the start and end moments of each task. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Effects of the number of generated demonstrations and source demonstrations on the final [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Two implementations of Fill operation, i.e., shrinking and expanding. Pointcloud processing. We first study how object and environment pointclouds affect the final performance, as shown in [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Extension on appearance generalization. The spatial generalization of R2RGen can serve [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: The annotation UI. The users first segment all relevant objects in the first frame. Then they [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Robot platform overview. We employ two robot platforms: (a) single-arm UR5e system [PITH_FULL_IMAGE:figures/full_fig_p017_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: For each test trial, the initial positions of the objects are determined by sampling distinct [PITH_FULL_IMAGE:figures/full_fig_p018_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Visualization of mobile manipulation results. The policy trained with R2RGen successfully [PITH_FULL_IMAGE:figures/full_fig_p019_11.png] view at source ↗
read the original abstract

Towards the aim of generalized robotic manipulation, spatial generalization is the most fundamental capability that requires the policy to work robustly under different spatial distribution of objects, environment and agent itself. To achieve this, substantial human demonstrations need to be collected to cover different spatial configurations for training a generalized visuomotor policy via imitation learning. Prior works explore a promising direction that leverages data generation to acquire abundant spatially diverse data from minimal source demonstrations. However, most approaches face significant sim-to-real gap and are often limited to constrained settings, such as fixed-base scenarios and predefined camera viewpoints. In this paper, we propose a real-to-real 3D data generation framework (R2RGen) that directly augments the pointcloud observation-action pairs to generate real-world data. R2RGen is simulator- and rendering-free, thus being efficient and plug-and-play. Specifically, we propose a unified three-stage framework, which (1) pre-processes source demonstrations under different camera setups in a shared 3D space with scene / trajectory parsing; (2) augments objects and robot's position with a group-wise backtracking strategy; (3) aligns the distribution of generated data with real-world 3D sensor using camera-aware post-processing. Empirically, R2RGen substantially enhances data efficiency on extensive experiments and demonstrates strong potential for scaling and application on mobile manipulation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The manuscript introduces R2RGen, a simulator-free real-to-real 3D data generation framework for spatially generalized robotic manipulation. It proposes a three-stage pipeline that (1) parses source demonstrations into a shared 3D space across camera setups, (2) augments object and robot-base positions via group-wise backtracking, and (3) applies camera-aware post-processing to align generated pointcloud-action pairs with real sensor statistics. The central claim is that this approach substantially improves data efficiency for imitation learning of visuomotor policies and shows strong potential for scaling to mobile manipulation tasks.

Significance. If the generated pairs preserve kinematic validity and match real 3D sensor distributions, the method would offer a practical, plug-and-play route to increase spatial coverage without simulators or additional human demonstrations, directly addressing a core bottleneck in real-world robot learning.

major comments (3)
  1. [Abstract] The empirical claim of 'substantially enhances data efficiency' (abstract) rests on unshown experimental details; the manuscript supplies no quantitative metrics, ablation results, or description of how action validity is preserved after augmentation.
  2. [Three-stage framework (stage 2)] Stage 2 group-wise backtracking repositions objects and the robot base in a shared 3D space, yet the manuscript provides no explicit checks or metrics (e.g., collision rates, kinematic feasibility, or trajectory consistency) to confirm that the resulting pointcloud-action pairs remain valid and free of new artifacts.
  3. [Three-stage framework (stage 3)] Stage 3 camera-aware post-processing is intended to restore sensor statistics, but without reported measures such as per-point noise models, occlusion statistics, or distribution distances between generated and real data, it is unclear whether residual mismatches remain that could affect policy generalization.
minor comments (2)
  1. [Method] Clarify the precise definition and grouping criteria used in the 'group-wise backtracking' procedure.
  2. [Experiments] Add a figure or table summarizing the source demonstration count, augmentation factor, and resulting dataset sizes for each experiment.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. We address each major comment point by point below, clarifying aspects of the work and outlining planned revisions to improve clarity and completeness.

read point-by-point responses

  1. Referee: [Abstract] The empirical claim of 'substantially enhances data efficiency' (abstract) rests on unshown experimental details; the manuscript supplies no quantitative metrics, ablation results, or description of how action validity is preserved after augmentation.

    Authors: The manuscript body contains quantitative experimental results demonstrating data efficiency gains, including success rate comparisons and ablation studies across multiple tasks and spatial configurations. We agree the abstract is overly concise and does not preview these details. We will revise the abstract to incorporate key quantitative metrics and a brief statement on validity preservation. The group-wise backtracking maintains action validity by preserving relative end-effector trajectories and joint configurations from source demonstrations while only adjusting absolute base and object positions in the shared 3D space; we will add an explicit paragraph describing this mechanism. revision: yes

  2. Referee: [Three-stage framework (stage 2)] Stage 2 group-wise backtracking repositions objects and the robot base in a shared 3D space, yet the manuscript provides no explicit checks or metrics (e.g., collision rates, kinematic feasibility, or trajectory consistency) to confirm that the resulting pointcloud-action pairs remain valid and free of new artifacts.

    Authors: The backtracking procedure selects new configurations by sampling within feasible regions derived from the original demonstration kinematics and environment bounds, which inherently limits collisions and maintains trajectory consistency. We acknowledge that explicit quantitative validation metrics are not reported in the current version. In the revision we will add a dedicated analysis subsection with reported collision rates, kinematic feasibility ratios, and trajectory deviation statistics computed on the generated pairs. revision: yes

  3. Referee: [Three-stage framework (stage 3)] Stage 3 camera-aware post-processing is intended to restore sensor statistics, but without reported measures such as per-point noise models, occlusion statistics, or distribution distances between generated and real data, it is unclear whether residual mismatches remain that could affect policy generalization.

    Authors: The post-processing step injects camera-specific noise and applies viewpoint-consistent masking derived from real sensor calibration data. We agree that quantitative alignment metrics would strengthen the claim. We will include in the revised manuscript distribution distance measures (e.g., Chamfer distance and Earth Mover’s Distance) between generated and real point clouds, along with per-point noise statistics and occlusion rate comparisons, to demonstrate the reduction in residual mismatches. revision: yes

Circularity Check

0 steps flagged

No circularity in the procedural three-stage augmentation pipeline

full rationale

The paper presents R2RGen as a simulator-free procedural framework with three explicit stages: (1) scene/trajectory parsing into a shared 3D space, (2) group-wise backtracking to reposition objects and robot base, and (3) camera-aware post-processing to align sensor statistics. No equations, fitted parameters, or mathematical derivations are described. The central claim of improved data efficiency for spatial generalization rests on empirical policy training results rather than any reduction of outputs to inputs by construction. No load-bearing self-citations or uniqueness theorems are invoked in the provided description to justify the pipeline steps. The method is therefore self-contained as an independent data-generation procedure whose validity is tested externally via imitation learning experiments.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Abstract-only review; the method implicitly assumes accurate 3D scene and trajectory parsing across camera setups and that backtracking produces kinematically valid robot motions, but no explicit free parameters or invented entities are stated.

axioms (1)
  • domain assumption Source demonstrations from different camera setups can be reliably parsed into a shared 3D space without loss of action semantics.
    Invoked in the first stage of the unified three-stage framework described in the abstract.

pith-pipeline@v0.9.0 · 5799 in / 1284 out tokens · 30009 ms · 2026-05-18T08:43:48.790085+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. ShapeGen: Robotic Data Generation for Category-Level Manipulation

    cs.RO 2026-04 unverdicted novelty 6.0

    ShapeGen generates shape-diverse 3D robotic manipulation demonstrations without simulators by curating a functional shape library and applying a minimal-annotation pipeline for novel, physically plausible data.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages · cited by 1 Pith paper · 12 internal anchors

  1. [1]

    $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. π0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164,

    work page internal anchor Pith review Pith/arXiv arXiv

  2. [2]

    RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control.arXiv preprint arXiv:2307.15818,

    work page internal anchor Pith review Pith/arXiv arXiv

  3. [3]

    GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation

    Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, et al. Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation.arXiv preprint arXiv:2410.06158,

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    Navila: Legged robot vision-language-action model for navigation.arXiv preprint arXiv:2412.04453, 2024a

    An-Chieh Cheng, Yandong Ji, Zhaojing Yang, Zaitian Gongye, Xueyan Zou, Jan Kautz, Erdem Bıyık, Hongxu Yin, Sifei Liu, and Xiaolong Wang. Navila: Legged robot vision-language-action model for navigation.arXiv preprint arXiv:2412.04453, 2024a. Xuxin Cheng, Jialong Li, Shiqi Yang, Ge Yang, and Xiaolong Wang. Open-television: Teleoperation with immersive acti...

    work page arXiv

  5. [5]

    Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots

    Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake, and Shuran Song. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots.arXiv preprint arXiv:2402.10329,

    work page internal anchor Pith review Pith/arXiv arXiv

  6. [6]

    Bunny-visionpro: Real-time bimanual dexterous teleoperation for imitation learning.arXiv preprint arXiv:2407.03162,

    Runyu Ding, Yuzhe Qin, Jiyue Zhu, Chengzhe Jia, Shiqi Yang, Ruihan Yang, Xiaojuan Qi, and Xiaolong Wang. Bunny-visionpro: Real-time bimanual dexterous teleoperation for imitation learning.arXiv preprint arXiv:2407.03162,

    work page arXiv

  7. [7]

    Ar2-d2: Training a robot without a robot.arXiv preprint arXiv:2306.13818,

    Jiafei Duan, Yi Ru Wang, Mohit Shridhar, Dieter Fox, and Ranjay Krishna. Ar2-d2: Training a robot without a robot.arXiv preprint arXiv:2306.13818,

    work page arXiv

  8. [8]

    Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation

    Zipeng Fu, Tony Z Zhao, and Chelsea Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation.arXiv preprint arXiv:2401.02117,

    work page internal anchor Pith review Pith/arXiv arXiv

  9. [9]

    Skillmimicgen: Automated demon- stration generation for efficient skill learning and deployment.arXiv preprint arXiv:2410.18907,

    Caelan Garrett, Ajay Mandlekar, Bowen Wen, and Dieter Fox. Skillmimicgen: Automated demon- stration generation for efficient skill learning and deployment.arXiv preprint arXiv:2410.18907,

    work page arXiv

  10. [10]

    Umi on legs: Making manipulation poli- cies mobile with manipulation-centric whole-body controllers.arXiv preprint arXiv:2407.10353,

    Huy Ha, Yihuai Gao, Zipeng Fu, Jie Tan, and Shuran Song. Umi on legs: Making manipulation poli- cies mobile with manipulation-centric whole-body controllers.arXiv preprint arXiv:2407.10353,

    work page arXiv

  11. [11]

    Re-mix: Optimizing data mixtures for large scale imitation learning.arXiv preprint arXiv:2408.14037,

    Joey Hejna, Chethan Bhateja, Yichen Jiang, Karl Pertsch, and Dorsa Sadigh. Re-mix: Optimizing data mixtures for large scale imitation learning.arXiv preprint arXiv:2408.14037,

    work page arXiv

  12. [12]

    Gensim2: Scaling robot data generation with multi-modal and reasoning llms.arXiv preprint arXiv:2410.03645,

    Pu Hua, Minghuan Liu, Annabella Macaluso, Yunfeng Lin, Weinan Zhang, Huazhe Xu, and Lirui Wang. Gensim2: Scaling robot data generation with multi-modal and reasoning llms.arXiv preprint arXiv:2410.03645,

    work page arXiv

  13. [13]

    Dexmimicgen: Automated data generation for bimanual dexterous manipulation via imitation learning.arXiv preprint arXiv:2410.24185,

    Zhenyu Jiang, Yuqi Xie, Kevin Lin, Zhenjia Xu, Weikang Wan, Ajay Mandlekar, Linxi Fan, and Yuke Zhu. Dexmimicgen: Automated data generation for bimanual dexterous manipulation via imitation learning.arXiv preprint arXiv:2410.24185,

    work page arXiv

  14. [14]

    Robot see robot do: Imitating articulated object manipulation with monocular 4d reconstruction.arXiv preprint arXiv:2409.18121,

    Justin Kerr, Chung Min Kim, Mingxuan Wu, Brent Yi, Qianqian Wang, Ken Goldberg, and Angjoo Kanazawa. Robot see robot do: Imitating articulated object manipulation with monocular 4d reconstruction.arXiv preprint arXiv:2409.18121,

    work page arXiv

  15. [15]

    DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset.arXiv preprint arXiv:2403.12945,

    work page internal anchor Pith review Pith/arXiv arXiv

  16. [16]

    OpenVLA: An Open-Source Vision-Language-Action Model

    Chung Min Kim, Mingxuan Wu, Justin Kerr, Ken Goldberg, Matthew Tancik, and Angjoo Kanazawa. Garfield: Group anything with radiance fields. InCVPR, pp. 21530–21539, 2024a. Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision...

    work page internal anchor Pith review Pith/arXiv arXiv

  17. [17]

    Phantom: Training robots without robots using only human videos.arXiv preprint arXiv:2503.00779,

    Marion Lepert, Jiaying Fang, and Jeannette Bohg. Phantom: Training robots without robots using only human videos.arXiv preprint arXiv:2503.00779,

    work page arXiv

  18. [18]

    CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation

    Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, et al. Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation.arXiv preprint arXiv:2411.19650,

    work page internal anchor Pith review Pith/arXiv arXiv

  19. [19]

    Data scaling laws in imitation learning for robotic manipulation.arXiv preprint arXiv:2410.18647,

    Fanqi Lin, Yingdong Hu, Pingyue Sheng, Chuan Wen, Jiacheng You, and Yang Gao. Data scaling laws in imitation learning for robotic manipulation.arXiv preprint arXiv:2410.18647,

    work page arXiv

  20. [20]

    MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations

    Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, and Dieter Fox. Mimicgen: A data generation system for scalable robot learning using human demonstrations.arXiv preprint arXiv:2310.17596,

    work page internal anchor Pith review Pith/arXiv arXiv

  21. [21]

    Consistency policy: Accelerated visuomotor policies via consistency distillation.arXiv preprint arXiv:2405.07503,

    Aaditya Prasad, Kevin Lin, Jimmy Wu, Linqi Zhou, and Jeannette Bohg. Consistency policy: Accelerated visuomotor policies via consistency distillation.arXiv preprint arXiv:2405.07503,

    work page arXiv

  22. [22]

    SAM 2: Segment Anything in Images and Videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714,

    work page internal anchor Pith review Pith/arXiv arXiv

  23. [23]

    Manibox: Enhancing spatial grasping generalization via scalable simulation data generation.arXiv preprint arXiv:2411.01850,

    Hengkai Tan, Xuezhou Xu, Chengyang Ying, Xinyi Mao, Songming Liu, Xingxing Zhang, Hang Su, and Jun Zhu. Manibox: Enhancing spatial grasping generalization via scalable simulation data generation.arXiv preprint arXiv:2411.01850,

    work page arXiv

  24. [24]

    Equivariant diffusion policy.arXiv preprint arXiv:2407.01812, 2024a

    Dian Wang, Stephen Hart, David Surovik, Tarik Kelestemur, Haojie Huang, Haibo Zhao, Mark Yeatman, Jiuguang Wang, Robin Walters, and Robert Platt. Equivariant diffusion policy.arXiv preprint arXiv:2407.01812, 2024a. Lirui Wang, Yiyang Ling, Zhecheng Yuan, Mohit Shridhar, Chen Bao, Yuzhe Qin, Bailin Wang, Huazhe Xu, and Xiaolong Wang. Gensim: Generating rob...

    work page arXiv

  25. [25]

    Demogen: Synthetic demonstration generation for data-efficient visuomotor policy learning.arXiv preprint arXiv:2502.16932,

    Zhengrong Xue, Shuying Deng, Zhenyang Chen, Yixuan Wang, Zhecheng Yuan, and Huazhe Xu. Demogen: Synthetic demonstration generation for data-efficient visuomotor policy learning.arXiv preprint arXiv:2502.16932,

    work page arXiv

  26. [26]

    Real2render2real: Scaling robot data without dynamics simulation or robot hardware.arXiv preprint arXiv:2505.09601,

    Justin Yu, Letian Fu, Huang Huang, Karim El-Refai, Rares Andrei Ambrus, Richard Cheng, Muham- mad Zubair Irshad, and Ken Goldberg. Real2render2real: Scaling robot data without dynamics simulation or robot hardware.arXiv preprint arXiv:2505.09601,

    work page arXiv

  27. [27]

    Generalizable humanoid manipulation with improved 3d diffusion policies.arXiv preprint arXiv:2410.10803, 2024a

    Yanjie Ze, Zixuan Chen, Wenhao Wang, Tianyi Chen, Xialin He, Ying Yuan, Xue Bin Peng, and Jiajun Wu. Generalizable humanoid manipulation with improved 3d diffusion policies.arXiv preprint arXiv:2410.10803, 2024a. Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. 3d diffusion policy: Generalizable visuomotor policy learning via s...

    work page arXiv

  28. [28]

    NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation

    Jiazhao Zhang, Kunyu Wang, Rongtao Xu, Gengze Zhou, Yicong Hong, Xiaomeng Fang, Qi Wu, Zhizheng Zhang, and He Wang. Navid: Video-based vlm plans the next step for vision-and- language navigation.arXiv preprint arXiv:2402.15852,

    work page internal anchor Pith review Pith/arXiv arXiv

  29. [29]

    Scizor: A self-supervised approach to data curation for large-scale imitation learning.arXiv preprint arXiv:2505.22626,

    Yu Zhang, Yuqi Xie, Huihan Liu, Rutav Shah, Michael Wan, Linxi Fan, and Yuke Zhu. Scizor: A self-supervised approach to data curation for large-scale imitation learning.arXiv preprint arXiv:2505.22626,

    work page arXiv

  30. [30]

    Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware

    Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware.arXiv preprint arXiv:2304.13705,

    work page internal anchor Pith review Pith/arXiv arXiv

  31. [31]

    Aloha unleashed: A simple recipe for robot dexterity.arXiv preprint arXiv:2410.13126,

    Tony Z Zhao, Jonathan Tompson, Danny Driess, Pete Florence, Kamyar Ghasemipour, Chelsea Finn, and Ayzaan Wahid. Aloha unleashed: A simple recipe for robot dexterity.arXiv preprint arXiv:2410.13126,

    work page arXiv

  32. [32]

    masks": [ aaaa

    The user first draws boxes on the initial frame to label each object with index, which is then processed with SAM Kirillov et al. (2023) to get object masks. Then user watches the video and is able to click Play, Stop or Rollback at anytime to capture key frames (i.e., the start / end of skill segments). When the user stops at a key frame, they can press ...

    work page 2023

  33. [33]

    Validation performance plateaued after approximately 2,500 epochs, and we selected the checkpoint with the lowest validation loss

    using Adam (learning rate 1×10 −4, weight decay 1×10 −6). Validation performance plateaued after approximately 2,500 epochs, and we selected the checkpoint with the lowest validation loss. 16 Preprint. Work in progress (a) UR5 Arm Platform (b) HexFellow Mobile Manipulator UR5e Arm WSG-50 Gripper ORBBEC femto bolt Lifting Table ORBBEC femto bolt AgileX PiP...

    work page 2024

  34. [34]

    Each object is also assigned a random rotation sampled from the range of -20 to 20 degrees, while the robot’s base is initialized at one of three distinct locations

    For each test trial, the initial positions of the objects are determined by sampling distinct locations from a pre-defined set of 32 points on the workspace. Each object is also assigned a random rotation sampled from the range of -20 to 20 degrees, while the robot’s base is initialized at one of three distinct locations. For tasks involving more than one...

    work page 2025