Pith. sign in

REVIEW 4 major objections 5 minor 56 references

SimLauncher: Launching Sample-Efficient Real-world Robotic Reinforcement Learning via Simulation Pre-training

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A policy pre-trained in a 3D-reconstructed simulation of the workspace lets a real robot learn manipulation skills to 100% success in 20–42 minutes of online reinforcement learning.

desk verdict Solid proof-of-concept, but the headline sample-efficiency gain is hard to trust until the sim-demo count and real-demo collection time are reported. read the letter →

arxiv 2507.04452 v1 pith:F65U5ONZ submitted 2025-07-06 cs.RO

Mingdong Wu , Lehong Wu , Yizhuo Wu , Weiyao Huang , Hongwei Fan , Zheyuan Hu , Haoran Geng , Jinzhou Li
show 4 more authors
Jiahe Ying Long Yang Yuanpei Chen Hao Dong
This is my paper
classification cs.RO
keywords sample-efficientreinforcementlearningreal-to-sim-to-realdigitaltwin3DGaussianSplattingvision-basedrobotpolicycriticbootstrappingactionproposaldexterousmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SimLauncher is a proof of concept that a policy pre-trained in a digital twin—a simulation rebuilt from the real workspace using 3D Gaussian Splatting and physics simulation—can make real-world robot reinforcement learning much more sample-efficient. The paper's central claim is that simulated rollouts from this pre-trained policy can bootstrap the critic, and its actions can guide exploration, so that real robots reach 100% success on Pick and Place, Pick and Insert, and dexterous grasping in roughly 37.5, 41.8, and 20.0 minutes respectively. At the same training checkpoints, the strongest comparison baseline that uses human demonstrations plus action proposals reaches only 58.3%, 53.3%, and 81.7%, and a plain offline-to-online hybrid learner reaches far less. This matters because real-world RL is normally bottlenecked by costly human demonstrations, slow exploration, and manual resets; simulation offers safe, cheap, and scalable trajectory collection. The authors frame the result as a proof of concept restricted to fixed objects and backgrounds, with manual resets and binary rewards.

What carries the argument

The load-bearing object is the digital twin: a 3D Gaussian Splatting reconstruction of the real workspace, a visual model built from photographs that can render new views of the scene, combined with a physics engine for dynamics. A state-based teacher policy is trained in this twin, its successful trajectories are rendered into visual demonstrations, and a behavior-cloning student policy is distilled from them with camera-pose randomization, image augmentation, and background masking. The second mechanism is the modified value target $y = r + \gamma \max_{a' \in \{a_{\mathrm{rl}}, a_{\mathrm{bc}}\}} \bar{Q}_{\bar{\phi}}(s',a')$, which bootstraps the critic using the better of the online policy's action and the pre-trained policy's action at the next state. The pre-trained policy therefore has two roles at once: generator of bootstrapping demonstrations and provider of alternative actions for exploration.

What would settle it

A concrete falsifier: evaluate SimLauncher on a task in which the physics engine demonstrably mismatches real contact dynamics, for instance a deformable object, or perturb the digital twin's camera pose and lighting beyond the paper's small randomization range; if time-to-100% success falls back to the level of the strongest human-demo baseline, the claim that simulation pre-training drives the gain is falsified.

Watch

Extended reading notes

Core claim

The discovery is that the sim-to-real gap can be crossed twice: a vision-based policy pretrained in a digital twin transfers well enough to generate successful real-world rollouts, and those rollouts plus large numbers of simulated rollouts can bootstrap the critic of an online RL learner. This lets the method combine three sources of value signal: 25% simulated demonstrations, 25% real demonstrations from the pretrained policy, and 50% online replay data. Exploration is also steered by selecting, according to a Boltzmann distribution over critic values, between the pretrained policy's action and the RL policy's action. On all three tasks the combined framework reaches 100% success in 20 to 42 minutes of real-world training time, where the authors report the baselines lag by 20 to 40 percentage points at the same checkpoint. The paper argues that the advantage is largest on multi-stage tasks because simulation pre-training supplies coverage of intermediate stages that online exploration would otherwise have to discover from scratch.

Load-bearing premise

The load-bearing premise is that the reconstructed simulation, meaning 3D Gaussian Splatting visuals plus calibrated physics, is faithful enough that a policy trained in it transfers to the real workspace and its rollouts are useful for critic bootstrapping; if the residual sim-to-real gap widens, the sample-efficiency gains shrink and the simulated demonstrations can actively bias the critic toward simulation-specific features.

Editorial extensions

If this is right

  • Reducing reliance on human data: with simulation-supplied demonstrations, 20 real demonstrations suffice to match or beat methods that rely on human effort, so the same pipeline could be scaled to more tasks without more teleoperation.
  • Simulated demonstrations alone can bootstrap learning: the hybrid online learner using 100 simulated demos outperforms the same learner using 20 human demos, showing that simulation data can substitute for human data at scale.
  • Wider state coverage helps: collecting simulated rollouts throughout policy training, not just successful ones, further shortens real-world training time.
  • Multi-stage tasks gain the most: the method's margin over baselines is larger on Pick and Place and Pick and Insert, consistent with the claim that pre-training supplies stage coverage.
  • The action-proposal module is most valuable early: removing it hurts performance mostly during the cold-start phase, after which the online policy catches up.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the digital-twin fidelity assumption holds broadly, the same recipe could let simulation data replace the majority of human demonstrations in real-world RL, with the remaining human role limited to resets and reward assignment.
  • A testable extension is to sweep the visual sim-to-real gap directly, for example by training the pre-trained policy in a digital twin with deliberately wrong textures or lighting and measuring how quickly the 100% success time degrades; the paper's camera randomization suggests the method has some slack but does not quantify it.
  • The critic-overfitting mechanism could be measured: the ablation without real demos implies that simulation-only bootstrapping undervalues real transitions, so tracking the difference between critic values assigned to sim demos and real replay data would give a direct diagnostic and early-stopping signal.
  • Because the evaluation uses a fixed object and background, a natural stress test is to randomize object appearance or lighting during real-world evaluation; if the 20–42 minute convergence times survive that perturbation, the claim of practical sample efficiency would be much stronger.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SimLauncher, a framework that pre-trains a vision-based policy in a digital twin built from 3D Gaussian Splatting and physics simulation, then uses that policy to accelerate real-world reinforcement learning. The pre-trained policy contributes in two ways: its simulated rollouts (and a small number of real-world rollouts) are used to bootstrap the critic of RLPD, and its actions are combined with RL actions through Boltzmann action selection following IBRL. The method is evaluated on three real-world tasks (Pick and Place, Pick and Insert, Dex Grasp) against IBRL and RLPD, with ablations isolating the simulated demo, real demo, and action-proposal components, plus scaling experiments on simulated behavior-cloning data. The paper reports that SimLauncher reaches 100% success in roughly 20–42 minutes of real-world RL, while IBRL and RLPD lag behind at the same checkpoints.

Significance. If the reported efficiency gains hold, SimLauncher is a useful proof-of-concept: it demonstrates a concrete recipe for converting large-scale simulation pre-training into real-world RL sample efficiency, and it goes beyond a single task by covering multi-stage, contact-rich, and dexterous manipulation. The paper gives explicit credit for its ablations: Figure 5 isolates the three design choices, Table II and Figure 6 provide a falsifiable scaling analysis showing that simulated demos can partially replace human demos, and the method is evaluated on real hardware rather than only in simulation. The main weakness is that the headline claim of improved sample efficiency is not fully auditable from the reported numbers: the size of the simulated demo buffer in the main runs is not stated, the wall-clock accounting for real-world demonstration collection is omitted from Table I, and the comparison to baselines is therefore partly a comparison of offline-data quantity. These are experimental-reporting gaps that can be fixed within the manuscript's scope, so the contribution is worth publishing after a major revision.

major comments (4)
  1. [Section III-C, Table I] The number of simulated demonstrations |D_sim| used in the main experiments is never reported. Section III-C refers to 'extensive simulated demonstrations', the abstract says 'hundreds', and Figure 6 uses 100 or 20 simulated demos, but Table I does not state which quantity produced the reported 100% success rates. Because the baselines receive only 20 human demos, a large simulated demo buffer could be the main driver of the reported gains rather than the RL machinery. Please report |D_sim| for each task, and ideally show how the Table I results vary with |D_sim|.
  2. [Section IV-B, Table I] The 'Training Time' reported in Table I excludes the real-world time spent collecting the 20 real demonstrations by deploying the pre-trained BC policy, as described in Section III-C. Since the paper's central claim is sample efficiency in the real world, omitting this real-robot wall-clock time makes the comparison to IBRL and RLPD difficult to interpret. Please report the full real-robot budget, including BC rollout attempts, failed episodes, resets, and any human intervention, or explicitly separate online RL time from total time.
  3. [Section IV-B] The claim that the comparison is fair because 'the number of real demonstrations in SimLauncher is 20, identical to the baselines' does not account for the fact that SimLauncher additionally consumes an unreported number of simulated demos and an action-proposal policy pre-trained on extensive simulated data. The ablation 'Ours w/o Sim Demo' in Figure 5 partially addresses this, but only on Pick and Place and without an IBRL comparison under a matched offline-data budget. Please add a comparison that controls for total offline data quantity, or otherwise quantify how much of the improvement comes from the extra simulated data versus the proposed algorithmic components.
  4. [Section IV-D, Takeaway 2, Figure 6] Takeaway 2 states that simulated demos alone enable effective bootstrapping and that 100 simulated demos surpass RLPD with 20 human demos. This is a comparison across different data quantities, so the conclusion conflates data scaling with data quality. The claim would be stronger with a matched-count comparison (e.g., 100 simulated versus 100 human demos, or 20 simulated versus 20 human demos) in addition to the current figure. As written, the takeaway is supported for the qualitative claim but not for the more specific quantitative comparison.
minor comments (5)
  1. [Appendix A, Eq. (6)] The loss in Eq. (6) is missing the square on the temporal-difference error; it should read (Q_phi(s,a) - y)^2, as in Eq. (2) of the main text.
  2. [Appendix B, Pick and Insert] The critic ensemble size for Pick and Insert is given as '1/0', which appears to be a typo; please specify the intended ensemble size.
  3. [Section IV-C] In the discussion of 'Ours w/o AP', the sentence 'as other methods leverage a pre-trained policy with a 73.3% success rate' is unclear: the ablation does not use the pre-trained action proposal, so it should refer to the full method rather than 'other methods'.
  4. [Table I] The success rates for SimLauncher are reported as exactly 100.0 with no standard deviation, while the baselines show nonzero variance. Please report the per-seed values or confidence intervals for all methods so that the reader can assess variability at the reported checkpoint.
  5. [Section IV-A] The description of the Dex Grasp setup says the hand wrist pose is randomly initialized 'per episode', but the initialization ranges for the object are given only in the Appendix; please include the full randomization ranges in the main text for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SimLauncher's sample-efficiency claim is an empirical comparison against external baselines, not a quantity derived from its own fitted inputs.

full rationale

SimLauncher is an empirical system, not a formal derivation, and the central claim—that simulation pre-training improves real-world RL sample efficiency—is established by direct, independent measurement: Table I and Fig. 4 compare SimLauncher's real-robot success rates and wall-clock training times against the external baselines IBRL and RLPD, and the ablations in Fig. 5 evaluate the method's own components. The pre-trained policy is used to generate simulated and real demos and to propose actions, but these are method ingredients whose contribution is tested empirically; no equation in the paper defines the reported success rate or training time in terms of the pre-trained policy's outputs. The one self-citation that appears in the method (PartManip [54], whose authors include two of the present authors) supplies a generic teacher-student distillation recipe for pre-training; the present paper's own Sim-BC scaling experiment (Table II) and the baseline comparisons independently support the transfer claim, so the citation is not load-bearing. No uniqueness theorem is imported, no ansatz is smuggled in via a same-author citation, and no known result is merely renamed. The checkpoint 'earliest training checkpoint where our method reaches 100% success' (Sec. IV-B) selects the evaluation time by definition, but the reported baseline success rates at that same timestep are external measurements rather than entailments of the selection. The paper's stated limitations—manual resets, fixed objects and backgrounds, and dependence on the digital twin—bound the empirical claim but do not make it circular. Consequently, there are no circular steps to report.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard RL assumptions plus several domain assumptions about the fidelity of the digital twin and the usefulness of the pre-trained policy's rollouts. No invented physical entities. The hyperparameters listed are hand-chosen per task and are not justified by a shared rule; one value (critic ensemble '1/0' for Pick and Insert) appears to be a reporting error. The paper's ablations do not test sensitivity to these choices, which limits the breadth of the claim.

free parameters (5)
  • Action proposal inverse temperature beta (initial) = 50 (Pick and Place, Pick and Insert), 10 (Dex Grasp)
    Chosen by hand per task; annealed to infinity. Controls the exploration-exploitation trade-off between BC action and RL action. No rule given for choosing 50 vs 10.
  • Demo batch mixing ratio = 25% sim demos, 25% real demos, 50% replay
    Chosen by hand; the paper tests the effect of removing components but not the ratio.
  • Number of simulated demos used in main runs = not stated (20, 100, 200, 1000 in analyses)
    The main experiments do not report how many simulated demos populate Dsim; analyses use 20, 100, 200, and 1000. This missing value affects reproducibility of the central result.
  • Critic ensemble size = 10 (Pick and Place, Dex Grasp), '1/0' (Pick and Insert)
    Hand-selected per task; the '1/0' entry for Pick and Insert appears to be a typo or a nonstandard configuration, undermining the stated implementation.
  • Number of real demos from BC rollouts = 20
    Chosen to match the 20 human demos in baselines; the paper does not test sensitivity to this number.
assumptions (5)
  • domain assumption The task is a Markov decision process with a human-assigned sparse binary reward that reliably indicates success.
    Section IV-A: human resets and manual reward assignment. All RL objectives assume this reward signal is reliable and consistent across trials.
  • domain assumption The digital twin built from 3DGS reconstruction and Isaac Gym or MuJoCo physics is an accurate model of the real workspace after calibration.
    Section III-B. The method depends on sim-to-real transfer of the pre-trained policy and on simulated demos being useful for critic bootstrapping.
  • domain assumption Background masking with SAM2 removes enough visual sim-to-real difference that a single BC policy transfers to the real world.
    Section III-B. The reported BC success (73.3% in the ablation) supports this for these tasks but is not a general guarantee.
  • standard math RLPD-style off-policy hybrid RL converges reliably in the real-world setting with the chosen hyperparameters.
    The method builds on RLPD [32]; convergence is assumed from the prior algorithm rather than proven here.
  • domain assumption Actions proposed by the pre-trained BC policy are at least as good as random exploration and do not systematically mislead the critic.
    Section III-C uses BC policy actions in the target and for action proposal; ablations show removing action proposal hurts, supporting this assumption for these tasks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SimLauncher: Launching Sample-Efficient Real-world Robotic Reinforcement Learning via Simulation Pre-training." pith.science (2026). https://pith.science/paper/F65U5ONZ

@misc{pith2026250704452,
  author       = {Pith},
  title        = {Pith review of: SimLauncher: Launching Sample-Efficient Real-world Robotic Reinforcement Learning via Simulation Pre-training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/F65U5ONZ}},
  note         = {Machine review of arXiv:2507.04452}
}
read the original abstract

Autonomous learning of dexterous, long-horizon robotic skills has been a longstanding pursuit of embodied AI. Recent advances in robotic reinforcement learning (RL) have demonstrated remarkable performance and robustness in real-world visuomotor control tasks. However, applying RL in the real world faces challenges such as low sample efficiency, slow exploration, and significant reliance on human intervention. In contrast, simulators offer a safe and efficient environment for extensive exploration and data collection, while the visual sim-to-real gap, often a limiting factor, can be mitigated using real-to-sim techniques. Building on these, we propose SimLauncher, a novel framework that combines the strengths of real-world RL and real-to-sim-to-real approaches to overcome these challenges. Specifically, we first pre-train a visuomotor policy in the digital twin simulation environment, which then benefits real-world RL in two ways: (1) bootstrapping target values using extensive simulated demonstrations and real-world demonstrations derived from pre-trained policy rollouts, and (2) Incorporating action proposals from the pre-trained policy for better exploration. We conduct comprehensive experiments across multi-stage, contact-rich, and dexterous hand manipulation tasks. Compared to prior real-world RL approaches, SimLauncher significantly improves sample efficiency and achieves near-perfect success rates. We hope this work serves as a proof of concept and inspires further research on leveraging large-scale simulation pre-training to benefit real-world robotic RL.

Figures

Figures reproduced from arXiv: 2507.04452 by the authors.

Figure 1
Figure 1. Illustration of our motivation. Given a simulation-pretrained policy, SimLauncher leverages simulated and real-world rollouts as demonstrations for critic bootstrapping and incorporates the policy for action proposal. SimLauncher significantly improves the sample efficiency of real-world RL compared with conventional RL methods using human-collected data. existing methods that integrate state-based simulation digita… view at source ↗
Figure 2
Figure 2. Overview of SimLauncher. In simulation, we collect simulated demos and train a vision-based policy for each task. We then rollout the pre-trained policy in the real world to collect real demos. The simulated and real-world demos are used for critic bootstrapping in real-world RL. The pre-trained policy also provides action and bootstrap proposal for the actor. pretrain a task-specific policy within the corresponding… view at source ↗
Figure 3
Figure 3. Task illustrations, initialization ranges, and common failure modes. (A) Pick and Place. This task involves relocating a banana to an electronic scale. Common failure includes the banana slipping off the scale. (B) Pick and Insert. This task involves grasping a toast and inserting it into the correct slot of a toaster. Common failure includes the toast getting stuck on the toaster edge. (C) Dex Grasp. This task invo… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Comparison with baselines. SimLauncher significantly outperforms state-of-the-art RL approaches that leverage human-collected demos and behavior-cloning methods. We report the mean and standard deviation over 3 seeds. Task Training Success Rate (%) Time (min) Ours IBRL…
Figure 5
Figure 5. Figure 5: Ablation study on our key design choices, 3 seeds. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Simulated demos alone enable effective bootstrapping, 3 seeds. the necessity of real-world demos for regularization. Its critic assigns lower values to real-world interactions, possibly because bootstrapping from both successful simulated demos and low-success-rate rep…
Figure 8
Figure 8. Figure 8: Visualization of state coverage in hybrid and success-only simulated demonstrations on the Pick and Place task. Takeaway 3: Expanding state coverage in simulated demonstrations enhances bootstrapping. A key advantage of simulation is its scalability in data generation.…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 17 canonical work pages

  1. [1]

    Decomposing the generalization gap in imitation learning for visual robotic manipulation,

    A. Xie, L. Lee, T. Xiao, and C. Finn, “Decomposing the generalization gap in imitation learning for visual robotic manipulation,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2024, pp. 3153–3160

    work page 2024

  2. [2]

    Bc-z: Zero-shot task generalization with robotic imitation learning,

    E. Jang, A. Irpan, M. Khansari, D. Kappler, F. Ebert, C. Lynch, S. Levine, and C. Finn, “Bc-z: Zero-shot task generalization with robotic imitation learning,” in Proceedings of the 5th Conference on Robot Learning, ser. Proceedings of Machine Learning Research, A. Faust, D. Hsu, and G. Neumann, Eds., vol. 164. PMLR, 08–11 Nov 2022, pp. 991–1002. [Online]....

    work page 2022

  3. [3]

    Robo-abc: Affordance generalization beyond categories via semantic correspon- dence for robot manipulation,

    Y . Ju, K. Hu, G. Zhang, G. Zhang, M. Jiang, and H. Xu, “Robo-abc: Affordance generalization beyond categories via semantic correspon- dence for robot manipulation,” in European Conference on Computer Vision. Springer, 2024, pp. 222–239

    work page 2024

  4. [4]

    The colosseum: A benchmark for evaluating generalization for robotic manipulation,

    W. Pumacay, I. Singh, J. Duan, R. Krishna, J. Thomason, and D. Fox, “The colosseum: A benchmark for evaluating generalization for robotic manipulation,” arXiv preprint arXiv:2402.08191 , 2024

    arXiv 2024

  5. [5]

    Learning complex dexterous manipulation with deep reinforcement learning and demonstrations,

    A. Rajeswaran, V . Kumar, A. Gupta, G. Vezzani, J. Schulman, E. Todorov, and S. Levine, “Learning complex dexterous manipulation with deep reinforcement learning and demonstrations,” arXiv preprint arXiv:1709.10087, 2017

    arXiv 2017

  6. [6]

    Towards human-level bimanual dexterous manipulation with reinforcement learning,

    Y . Chen, T. Wu, S. Wang, X. Feng, J. Jiang, Z. Lu, S. McAleer, H. Dong, S.-C. Zhu, and Y . Yang, “Towards human-level bimanual dexterous manipulation with reinforcement learning,” Advances in Neural Information Processing Systems, vol. 35, pp. 5150–5163, 2022

    work page 2022

  7. [7]

    Aloha unleashed: A simple recipe for robot dexterity,

    T. Z. Zhao, J. Tompson, D. Driess, P. Florence, K. Ghasemipour, C. Finn, and A. Wahid, “Aloha unleashed: A simple recipe for robot dexterity,” arXiv preprint arXiv:2410.13126 , 2024

    arXiv 2024

  8. [8]

    Reinforcement learning on variable impedance con- troller for high-precision robotic assembly,

    J. Luo, E. Solowjow, C. Wen, J. A. Ojea, A. M. Agogino, A. Tamar, and P. Abbeel, “Reinforcement learning on variable impedance con- troller for high-precision robotic assembly,” in 2019 International Conference on Robotics and Automation (ICRA) . IEEE, 2019, pp. 3080–3087

    work page 2019

Show all 56 references
  1. [9]

    Precise and dexterous robotic manipulation via human-in-the-loop reinforcement learning,

    J. Luo, C. Xu, J. Wu, and S. Levine, “Precise and dexterous robotic manipulation via human-in-the-loop reinforcement learning,” arXiv preprint arXiv:2410.21845, 2024

    2024 arXiv

  2. [10]

    Serl: A software suite for sample-efficient robotic reinforcement learning,

    J. Luo, Z. Hu, C. Xu, Y . L. Tan, J. Berg, A. Sharma, S. Schaal, C. Finn, A. Gupta, and S. Levine, “Serl: A software suite for sample-efficient robotic reinforcement learning,” arXiv preprint arXiv:2401.16013 , 2024

    2024 arXiv

  3. [11]

    Manipulate-anything: Automating real-world robots using vision-language models,

    J. Duan, W. Yuan, W. Pumacay, Y . R. Wang, K. Ehsani, D. Fox, and R. Krishna, “Manipulate-anything: Automating real-world robots using vision-language models,” arXiv preprint arXiv:2406.18915 , 2024

    2024 arXiv

  4. [12]

    Robogen: Towards unleashing infinite data for automated robot learning via generative simulation,

    Y . Wang, Z. Xian, F. Chen, T.-H. Wang, Y . Wang, K. Fragkiadaki, Z. Erickson, D. Held, and C. Gan, “Robogen: Towards unleashing infinite data for automated robot learning via generative simulation,” arXiv preprint arXiv:2311.01455 , 2023

    2023 arXiv

  5. [13]

    Gensim: Generating robotic simulation tasks via large language models,

    L. Wang, Y . Ling, Z. Yuan, M. Shridhar, C. Bao, Y . Qin, B. Wang, H. Xu, and X. Wang, “Gensim: Generating robotic simulation tasks via large language models,” arXiv preprint arXiv:2310.01361 , 2023

    2023 arXiv

  6. [14]

    Gensim2: Scaling robot data generation with multi-modal and reasoning llms,

    P. Hua, M. Liu, A. Macaluso, Y . Lin, W. Zhang, H. Xu, and L. Wang, “Gensim2: Scaling robot data generation with multi-modal and reasoning llms,” arXiv preprint arXiv:2410.03645 , 2024

    2024 arXiv

  7. [15]

    Mimicgen: A data generation system for scalable robot learning using human demonstrations,

    A. Mandlekar, S. Nasiriany, B. Wen, I. Akinola, Y . Narang, L. Fan, Y . Zhu, and D. Fox, “Mimicgen: A data generation system for scalable robot learning using human demonstrations,” arXiv preprint arXiv:2310.17596, 2023

    2023 arXiv

  8. [16]

    Dexmimicgen: Automated data generation for biman- ual dexterous manipulation via imitation learning,

    Z. Jiang, Y . Xie, K. Lin, Z. Xu, W. Wan, A. Mandlekar, L. Fan, and Y . Zhu, “Dexmimicgen: Automated data generation for biman- ual dexterous manipulation via imitation learning,” arXiv preprint arXiv:2410.24185, 2024

    2024 arXiv

  9. [17]

    Skillgen: Auto- mated demonstration generation for efficient skill learning and deploy- ment,

    C. R. Garrett, A. Mandlekar, B. Wen, and D. Fox, “Skillgen: Auto- mated demonstration generation for efficient skill learning and deploy- ment,” in 2nd CoRL Workshop on Learning Effective Abstractions for Planning

  10. [18]

    Skillmimicgen: Automated demonstration generation for efficient skill learning and deployment,

    C. Garrett, A. Mandlekar, B. Wen, and D. Fox, “Skillmimicgen: Automated demonstration generation for efficient skill learning and deployment,” arXiv preprint arXiv:2410.18907 , 2024

    2024 arXiv

  11. [19]

    Robocasa: Large-scale simulation of ev- eryday tasks for generalist robots,

    S. Nasiriany, A. Maddukuri, L. Zhang, A. Parikh, A. Lo, A. Joshi, A. Mandlekar, and Y . Zhu, “Robocasa: Large-scale simulation of ev- eryday tasks for generalist robots,” arXiv preprint arXiv:2406.02523 , 2024

    2024 arXiv

  12. [20]

    Reconciling reality through simulation: A real- to-sim-to-real approach for robust manipulation,

    M. Torne, A. Simeonov, Z. Li, A. Chan, T. Chen, A. Gupta, and P. Agrawal, “Reconciling reality through simulation: A real- to-sim-to-real approach for robust manipulation,” arXiv preprint arXiv:2403.03949, 2024

    2024 arXiv

  13. [21]

    Splatsim: Zero-shot sim2real transfer of rgb manipulation policies using gaussian splatting,

    M. N. Qureshi, S. Garg, F. Yandun, D. Held, G. Kantor, and A. Silwal, “Splatsim: Zero-shot sim2real transfer of rgb manipulation policies using gaussian splatting,” arXiv preprint arXiv:2409.10161 , 2024

    2024 arXiv

  14. [22]

    Robo-gs: A physics consistent spatial-temporal model for robotic arm with hybrid representation,

    H. Lou, Y . Liu, Y . Pan, Y . Geng, J. Chen, W. Ma, C. Li, L. Wang, H. Feng, L. Shi, et al., “Robo-gs: A physics consistent spatial-temporal model for robotic arm with hybrid representation,” arXiv preprint arXiv:2408.14873, 2024

    2024 arXiv

  15. [23]

    Resim: Generating high-fidelity simulation data via 3d- photorealistic real-to-sim for robotic manipulation,

    X. Han, M. Liu, Y . Chen, J. Yu, X. Lyu, Y . Tian, B. Wang, W. Zhang, and J. Pang, “Resim: Generating high-fidelity simulation data via 3d- photorealistic real-to-sim for robotic manipulation,” arXiv preprint arXiv:2502.08645, 2025

    2025 arXiv

  16. [24]

    Rl- gsbridge: 3d gaussian splatting based real2sim2real method for robotic manipulation learning,

    Y . Wu, L. Pan, W. Wu, G. Wang, Y . Miao, and H. Wang, “Rl- gsbridge: 3d gaussian splatting based real2sim2real method for robotic manipulation learning,” arXiv preprint arXiv:2409.20291 , 2024

    2024 arXiv

  17. [25]

    A real-to-sim-to-real approach to robotic manipulation with vlm-generated iterative keypoint rewards,

    S. Patel, X. Yin, W. Huang, S. Garg, H. Nayyeri, L. Fei-Fei, S. Lazebnik, and Y . Li, “A real-to-sim-to-real approach to robotic manipulation with vlm-generated iterative keypoint rewards,” arXiv preprint arXiv:2502.08643, 2025

    2025 arXiv

  18. [26]

    What went wrong? closing the sim-to-real gap via differentiable causal discovery,

    P. Huang, X. Zhang, Z. Cao, S. Liu, M. Xu, W. Ding, J. Francis, B. Chen, and D. Zhao, “What went wrong? closing the sim-to-real gap via differentiable causal discovery,” in Conference on Robot Learning. PMLR, 2023, pp. 734–760

    2023

  19. [27]

    Bridging the sim-to-real gap from the information bottleneck per- spective,

    H. He, P. Wu, C. Bai, H. Lai, L. Wang, L. Pan, X. Hu, and W. Zhang, “Bridging the sim-to-real gap from the information bottleneck per- spective,” arXiv preprint arXiv:2305.18464 , 2023

    2023 arXiv

  20. [28]

    Isaac gym: High performance gpu-based physics simulation for robot learning,

    V . Makoviychuk, L. Wawrzyniak, Y . Guo, M. Lu, K. Storey, M. Mack- lin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, et al., “Isaac gym: High performance gpu-based physics simulation for robot learning,” arXiv preprint arXiv:2108.10470 , 2021

    2021 arXiv

  21. [29]

    Towards building ai-cps with nvidia isaac sim: An industrial benchmark and case study for robotics manipulation,

    Z. Zhou, J. Song, X. Xie, Z. Shu, L. Ma, D. Liu, J. Yin, and S. See, “Towards building ai-cps with nvidia isaac sim: An industrial benchmark and case study for robotics manipulation,” in Proceedings of the 46th International Conference on Software Engineering: Software Enginee...

    2024

  22. [30]

    Mujoco playground,

    K. Zakka, B. Tabanpour, Q. Liao, M. Haiderbhai, S. Holt, J. Y . Luo, A. Allshire, E. Frey, K. Sreenath, L. A. Kahrs, C. Sferrazza, Y . Tassa, and P. Abbeel, “Mujoco playground,” 2025. [Online]. Available: https://arxiv.org/abs/2502.08844

    2025 arXiv

  23. [31]

    Imitation bootstrapped reinforcement learning,

    H. Hu, S. Mirchandani, and D. Sadigh, “Imitation bootstrapped reinforcement learning,” arXiv preprint arXiv:2311.02198 , 2023

    2023 arXiv

  24. [32]

    Efficient online reinforcement learning with offline data,

    P. J. Ball, L. Smith, I. Kostrikov, and S. Levine, “Efficient online reinforcement learning with offline data,” in International Conference on Machine Learning . PMLR, 2023, pp. 1577–1594

    2023

  25. [33]

    Improving vision-language-action model with online reinforcement learning,

    Y . Guo, J. Zhang, X. Chen, X. Ji, Y .-J. Wang, Y . Hu, and J. Chen, “Improving vision-language-action model with online reinforcement learning,” arXiv preprint arXiv:2501.16664 , 2025

    2025 arXiv

  26. [34]

    Conrft: A reinforced fine-tuning method for vla models via consistency policy,

    Y . Chen, S. Tian, S. Liu, Y . Zhou, H. Li, and D. Zhao, “Conrft: A reinforced fine-tuning method for vla models via consistency policy,” arXiv preprint arXiv:2502.05450 , 2025

    2025 arXiv

  27. [35]

    Reboot: Reuse data for bootstrapping efficient real-world dexterous manipulation,

    Z. Hu, A. Rovinsky, J. Luo, V . Kumar, A. Gupta, and S. Levine, “Reboot: Reuse data for bootstrapping efficient real-world dexterous manipulation,” arXiv preprint arXiv:2309.03322 , 2023

    2023 arXiv

  28. [36]

    Reset-free reinforcement learning via multi-task learning: Learning dexterous manipulation behaviors without human intervention,

    A. Gupta, J. Yu, T. Z. Zhao, V . Kumar, A. Rovinsky, K. Xu, T. Devlin, and S. Levine, “Reset-free reinforcement learning via multi-task learning: Learning dexterous manipulation behaviors without human intervention,” in 2021 IEEE International Conference on Robotics and Automa...

    2021

  29. [37]

    Dexterous manipulation from images: Autonomous real- world rl via substep guidance,

    K. Xu, Z. Hu, R. Doshi, A. Rovinsky, V . Kumar, A. Gupta, and S. Levine, “Dexterous manipulation from images: Autonomous real- world rl via substep guidance,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2023, pp. 5938–5945

    2023

  30. [38]

    The ingredients of real-world robotic reinforcement learning,

    H. Zhu, J. Yu, A. Gupta, D. Shah, K. Hartikainen, A. Singh, V . Kumar, and S. Levine, “The ingredients of real-world robotic reinforcement learning,” arXiv preprint arXiv:2004.12570 , 2020

    2004 arXiv

  31. [39]

    Awac: Accelerating online reinforcement learning with offline datasets,

    A. Nair, A. Gupta, M. Dalal, and S. Levine, “Awac: Accelerating online reinforcement learning with offline datasets,” arXiv preprint arXiv:2006.09359, 2020

    2006 arXiv

  32. [40]

    Robot fine-tuning made easy: Pre-training rewards and policies for autonomous real-world reinforcement learning,

    J. Yang, M. S. Mark, B. Vu, A. Sharma, J. Bohg, and C. Finn, “Robot fine-tuning made easy: Pre-training rewards and policies for autonomous real-world reinforcement learning,” in 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 4804–4811

    2024

  33. [41]

    Efficient online reinforcement learning fine-tuning need not retain offline data,

    Z. Zhou, A. Peng, Q. Li, S. Levine, and A. Kumar, “Efficient online reinforcement learning fine-tuning need not retain offline data,” arXiv preprint arXiv:2412.07762, 2024

    2024 arXiv

  34. [42]

    Reinforcement learning with foundation priors: Let the embodied agent efficiently learn on its own,

    W. Ye, Y . Zhang, H. Weng, X. Gu, S. Wang, T. Zhang, M. Wang, P. Abbeel, and Y . Gao, “Reinforcement learning with foundation priors: Let the embodied agent efficiently learn on its own,” arXiv preprint arXiv:2310.02635, 2023

    arXiv

  35. [43]

    Rapidly adapting policies to the real world via simulation-guided fine-tuning,

    P. Yin, T. Westenbroek, S. Bagaria, K. Huang, C.-a. Cheng, A. Kobolov, and A. Gupta, “Rapidly adapting policies to the real world via simulation-guided fine-tuning,” arXiv preprint arXiv:2502.02705 , 2025

    2025 arXiv

  36. [44]

    Overcoming the sim-to-real gap: Leveraging simulation to learn to explore for real-world rl,

    A. Wagenmaker, K. Huang, L. Ke, B. Boots, K. Jamieson, and A. Gupta, “Overcoming the sim-to-real gap: Leveraging simulation to learn to explore for real-world rl,” arXiv preprint arXiv:2410.20254 , 2024

    2024 arXiv

  37. [45]

    Cherry-picking with reinforcement learning

    Y . Zhang, L. Ke, A. Deshpande, A. Gupta, and S. S. Srinivasa, “Cherry-picking with reinforcement learning.” in Robotics: Science and Systems, 2023

    2023

  38. [46]

    Sim-to-real transfer in deep reinforcement learning for robotics: a survey,

    W. Zhao, J. P. Queralta, and T. Westerlund, “Sim-to-real transfer in deep reinforcement learning for robotics: a survey,” in 2020 IEEE symposium series on computational intelligence (SSCI) . IEEE, 2020, pp. 737–744

    2020

  39. [47]

    In-hand object rotation via rapid motor adaptation,

    H. Qi, A. Kumar, R. Calandra, Y . Ma, and J. Malik, “In-hand object rotation via rapid motor adaptation,” in Conference on Robot Learning. PMLR, 2023, pp. 1722–1732

    2023

  40. [48]

    Lessons from learning to spin

    J. Wang, Y . Yuan, H. Che, H. Qi, Y . Ma, J. Malik, and X. Wang, “Lessons from learning to spin” pens”,” arXiv preprint arXiv:2407.18902, 2024

    2024 arXiv

  41. [49]

    Dextrah- rgb: Visuomotor policies to grasp anything with dexterous hands,

    R. Singh, A. Allshire, A. Handa, N. Ratliff, and K. Van Wyk, “Dextrah- rgb: Visuomotor policies to grasp anything with dexterous hands,” arXiv preprint arXiv:2412.01791 , 2024

    2024 arXiv

  42. [50]

    Automated creation of digital cousins for robust policy learning,

    T. Dai, J. Wong, Y . Jiang, C. Wang, C. Gokmen, R. Zhang, J. Wu, and L. Fei-Fei, “Automated creation of digital cousins for robust policy learning,” arXiv preprint arXiv:2410.07408 , 2024

    2024 arXiv

  43. [51]

    Robotwin: Dual-arm robot benchmark with generative digital twins (early version),

    Y . Mu, T. Chen, S. Peng, Z. Chen, Z. Gao, Y . Zou, L. Lin, Z. Xie, and P. Luo, “Robotwin: Dual-arm robot benchmark with generative digital twins (early version),” arXiv preprint arXiv:2409.02920, 2024

    2024 arXiv

  44. [52]

    Video2policy: Scaling up manipulation tasks in simulation through internet videos,

    W. Ye, F. Liu, Z. Ding, Y . Gao, O. Rybkin, and P. Abbeel, “Video2policy: Scaling up manipulation tasks in simulation through internet videos,” arXiv preprint arXiv:2502.09886 , 2025

    2025 arXiv

  45. [53]

    Mujoco: A physics engine for model-based control,

    E. Todorov, T. Erez, and Y . Tassa, “Mujoco: A physics engine for model-based control,” in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems . IEEE, 2012, pp. 5026–5033

    2012

  46. [54]

    Partmanip: Learning cross-category generalizable part manipulation policy from point cloud observations,

    H. Geng, Z. Li, Y . Geng, J. Chen, H. Dong, and H. Wang, “Partmanip: Learning cross-category generalizable part manipulation policy from point cloud observations,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 2978–2988

    2023

  47. [55]

    Sam 2: Segment anything in images and videos,

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. R ¨adle, C. Rolland, L. Gustafson, E. Mintun, J. Pan, K. V . Alwala, N. Carion, C.-Y . Wu, R. Girshick, P. Doll ´ar, and C. Feichtenhofer, “Sam 2: Segment anything in images and videos,” arXiv preprint arXiv:...

    2024 arXiv

  48. [56]

    Open x-embodiment: Robotic learning datasets and rt-x models,

    Q. Vuong, S. Levine, H. R. Walke, K. Pertsch, A. Singh, R. Doshi, C. Xu, J. Luo, L. Tan, D. Shah, et al., “Open x-embodiment: Robotic learning datasets and rt-x models,” in Towards Generalist Robots: Learning Paradigms for Scalable Skill Acquisition@ CoRL2023, 2023

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.