REVIEW 4 major objections 7 minor 3 cited by
Rapidly Adapting Policies to the Real World via Simulation-Guided Fine-Tuning
T0 review · 4 major / 7 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read The paper claims that a simulation-learned value function, when used to reshape real-world rewards and shorten the learning horizon, makes robot fine-tuning an order of magnitude more sample-efficient, with a provable near-optimality…
desk verdict SGFT's empirical results are strong and the method is a useful sim-to-real fine-tuning addition, but the theory's load-bearing 'improvable Vsim' assumption is unverified and the proof has fixable gaps. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine of SGFT is the potential-based reward shaping transformation $\bar{r}(s,s') = r(s) + \gamma V_{\mathrm{sim}}(s') - V_{\mathrm{sim}}(s)$, with $V_{\mathrm{sim}}$ as the potential, combined with an $H$-step finite-horizon objective. The telescoping identity (Equation 1) shows that optimizing the shaped $H$-step return is equivalent, up to a policy-independent constant, to maximizing $\gamma^H V_{\mathrm{sim}}(s_H) + \sum_{t=0}^{H-1} \gamma^t r(s_t)$ — i.e., the simulator bootstraps the long-horizon tail while real data only needs to support $H$ steps of local search. The proof machinery includes a monotonicity property of the optimal $H$-step value under the improvable condition (Lemma 3) and a decomposition of suboptimality (Lemma 4), which together remove the $\frac{1}{1-\gamma}$ factor on the value-gap error that appears in prior MPC-style bounds. In implementations, Dyna-SGFT uses a learned model to hallucinate $H$-step branches from real states, and MPC-SGFT uses TDMPC-2 with the frozen $V_{\mathrm{sim}}$ as terminal cost.
What would settle it
Collect real-world transitions $(s,a,s')$ and evaluate the expression $\max_a [\gamma V_{\mathrm{sim}}(s') - V_{\mathrm{sim}}(s)] + r(s)$ on visited states; if any state yields a negative value, Definition 1 is violated and the paper's bound no longer applies, predicting that SGFT will stagnate or regress at that state.
Extended reading notes
Core claim
The central claim, stated in the paper's own terms, is that optimizing a short-horizon objective with a simulation-learned value function as the terminal 'surplus' recovers near-optimal real-world behavior even when the simulator's dynamics are wrong in detail. The value function $V_{\mathrm{sim}}$ defines an ordering over states that captures task-relevant structure — reaching, grasping, placing — and this ordering is assumed to survive the sim-to-real gap. The paper introduces the notion of improvability (Definition 1): for every state there exists an action satisfying $E_{s'\sim p_{\mathrm{real}}}[\gamma V_{\mathrm{sim}}(s')] - V_{\mathrm{sim}}(s) \ge -r(s)$. Under this condition, Theorem 1 bounds the suboptimality of the learned policy by $O\!\left(\frac{\gamma}{1-\gamma}\alpha H + \gamma^H \epsilon\right)$, where $\alpha$ is model error and $\epsilon$ is the value gap — the crucial point being that the bound does not grow like $\frac{1}{1-\gamma}$ on $\epsilon$, so small $H$ remains safe.
Load-bearing premise
The entire guarantee sits on the assumption that the simulation-trained value function remains 'improvable' in the real world: at every state the robot can reach, some action must move toward states the simulator values highly without losing more than the immediate reward.
Editorial extensions
If this is right
- When the improvable condition holds, real-world fine-tuning reduces to a short-horizon (even $H=1$) policy search, so sample complexity depends on local dynamics rather than full task length.
- Model-based RL becomes practical in the real world: learned models only need to be accurate for $H$ steps before the value function takes over, sidestepping compounding model error.
- SGFT is a wrapper on top of standard RL algorithms; applying it to SAC and TDMPC-2 improves both, and prevents the catastrophic forgetting seen in standard fine-tuning.
- The theory justifies using short horizons with very few real-world rollouts, making it possible to adapt policies in under an hour on contact-rich tasks like hammering and insertion.
Reading between the lines
- We infer that the same short-horizon, value-guided objective could work with other off-domain data sources (e.g., human demonstrations or offline datasets), because the load-bearing assumption is about the value function's ordering over states, not which domain produced the value.
- We infer that improvableness can be monitored online: by estimating the quantity $\max_a [\gamma V_{\mathrm{sim}}(s') - V_{\mathrm{sim}}(s)] + r(s)$ from real transitions, a system could detect when the assumption fails and adaptively lengthen $H$ before relying on the guarantee.
- We infer that $H$ is a bias-variance dial, and that an automatic rule for choosing $H$ from estimated model error would be a natural next step; the paper's peg-in-hole ablation shows tasks with higher precision needs benefit from larger $H$.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Simulation-Guided Fine-Tuning (SGFT), a framework for sim-to-real policy adaptation that uses a value function Vsim learned in simulation as a potential-based reward shaper and shortens the fine-tuning horizon to an H-step objective. The authors implement two model-based instantiations (SGFT-SAC and SGFT-TDMPC-2) and evaluate them on five real-world contact-rich manipulation tasks plus sim-to-sim benchmarks, reporting substantial sample-efficiency gains over standard fine-tuning baselines. They also provide a theoretical analysis (Theorem 1) intended to show that, under an 'improvability' condition on Vsim with respect to the real dynamics, the H-step SGFT objective yields near-optimal real-world policies with suboptimality O(gamma/(1-gamma) alpha H + gamma^H epsilon).
Significance. If the experimental results hold, this is a strong empirical contribution: it demonstrates that a simulation-trained value function can serve as an effective exploration prior for real-world fine-tuning, succeeding on tasks where zero-shot transfer and standard fine-tuning fail, and it provides a structural justification for horizon shortening in model-based sim-to-real transfer. The core idea is simple, broadly applicable, and the paper includes real-hardware experiments across multiple contact-rich tasks. The theoretical analysis, once corrected, would be a useful addition to the literature on terminal-value MPC and potential-based reward shaping. The paper does not provide machine-checked proofs, and the theory contains several formal inconsistencies, but these appear fixable.
major comments (4)
- [Section 4.1] The definition of Q*_H is inconsistent with the H-step Bellman recursion and with the proof of Lemma 4. Section 4.1 defines Q*_H(s, pi) := E_{a~pi}[gamma V*_H(s') + r(s)], but the one-step lookahead for the H-step objective must use V*_{H-1} and the shaped reward bar{r}(s,s') = r(s) + gamma Vsim(s') - Vsim(s); indeed, Eq. (13) in the proof of Lemma 4 uses Q*_H(s, pi) = E[gamma V*_{H-1}(s') + bar{r}(s,s')]. As printed, the Section 4.1 display would not select the optimal first action for the H-step objective, and it is inconsistent with the rest of the theory. Please correct the display and align all subsequent uses of Q*_H.
- [Lemma 2, Eq. (5)] Lemma 2's statement omits the model-error factor alpha H that its own proof derives. The proof combines inequalities (6) and (7), each carrying a factor gamma(...) alpha H, so the combination yields a bound proportional to gamma((1-gamma^{H-1})/(1-gamma) Delta r + gamma^H Delta V) alpha H, not the alpha-free expression in Eq. (5). Without alpha H, the model-free case alpha=0 would leave a spurious delta = gamma Delta V when Lemma 4 is invoked, contradicting the O(gamma^H epsilon) model-free bound claimed in Theorem 1. The intended statement should include alpha H, and Theorem 1's proof should be checked against the corrected lemma.
- [Section 5, Definition 1 / Theorem 1] The theoretical guarantee is conditional on Vsim being improvable with respect to Mreal (Definition 1), but the paper provides no empirical evidence that this per-state condition holds for the tasks evaluated. The pedagogical pendulum example has identical sets of feasible next states in simulation and reality, which is precisely the regime where the condition is easiest to satisfy; the real tasks are selected because contact dynamics are misspecified, so the margin in (2) could be negative at some states. Since Lemma 3, and hence Theorem 1, fails if the margin is negative anywhere, the paper should measure or bound the margin at least in the sim-to-sim benchmarks of Section 6.3, where the target dynamics are known exactly, and ideally in the real-world experiments, before claiming the theory underpins the contact-rich results.
- [Lemma 3] Lemma 3 is stated with a strict inequality, sup_a E_{s'~preal}[gamma Vsim(s')] - Vsim(s) > -r(s), while Definition 1 and Theorem 1 use a non-strict inequality, >= -r(s). If equality holds at some state, the lemma's hypothesis as stated is not met, so the proof of Theorem 1 has a gap. The proof only needs >=, so the statement should be corrected to match Definition 1.
minor comments (7)
- [Section 6.1] The text says 'We use H = 1 in all our experiments' but later states that SGFT-TDMPC-2 uses H = 4; please clarify which instantiation uses which horizon.
- [Section 4.1] The acronym 'PRBS' in the phrase 'Potential-Based Reward Shaping (PRBS)' is a typo for PBRS.
- [Lemma 1 and Lemma 3] In Lemma 1, the definition of Delta V uses 'Vs(s)' where it should be 'Vsim(s)'; in Lemma 3, the expectation is written as E_{s~preal} where it should be E_{s'~preal}.
- [Theorem 1] The statement says 'Assumes that' and uses the norm symbol for what should be an absolute value |Vsim(s) - V*_real(s)|; the phrase 'for H sufficiently small' is also vague and should specify the intended regime.
- [Lemma 4 proof] In the displayed chain of inequalities around Eq. (14), there is an extra '+' sign that appears to be a typo.
- [Appendix C] The sentence 'We continue training SAC with a fixed temperature of alpha = 0.01 and with a UTD of 2d' appears to have a typo in '2d'; please state the correct update-to-data ratio.
- [Figure 4] The real-world success-rate curves do not report the number of trials or error bars; please state whether each curve is a single run or an average, and how many independent runs were performed.
Circularity Check
No load-bearing circularity: Theorem 1 is conditional on an explicit unverified assumption, and the empirical claims are validated against real-world baselines.
full rationale
The SGFT derivation chain is not circular. Vsim is learned in the simulator on the same reward r, then frozen and used as a potential in the PBRS reshaped reward (Eq. 1). The H-step objective with terminal Vsim is the standard MPC terminal-cost objective, and the paper's Theorem 1 is an explicit conditional statement: if Vsim is improvable w.r.t. Mreal (Definition 1) and the value gap to V*real is bounded by epsilon, then the H-step SGFT policy has suboptimality O(gamma/(1-gamma) alpha H + gamma^H epsilon). Definition 1 is an assumption about the real dynamics, not a consequence of the paper's definitions; it is not fitted to the real-world results. The paper provides a proof in Appendix A rather than citing the result as an external oracle. The real-world experiments on five tasks against SAC, TDMPC-2, IQL, RLPD, and sim-to-real baselines are external benchmarks, so the central empirical claim does not reduce to the simulation-trained Vsim by construction. The only self-citations (Westenbroek et al. 2022; Cheng et al. 2019, 2021) are used to argue that Definition 1 is reasonable and to borrow proof techniques, but Theorem 1 is proven in the appendix and the experiments stand independently; these citations are not load-bearing. Under the review rules this is a non-finding: no circular step is exhibited. The main caveat, that Definition 1 is never empirically validated, is a correctness/evidence concern, not a circularity concern.
Assumptions & free parameters
free parameters (1)
- Horizon length H =
H=1 for SGFT-SAC, H=4 for SGFT-TDMPC-2
assumptions (4)
- domain assumption Vsim is improvable with respect to Mreal: for all s, max_a E_{s'~preal}[γVsim(s')] - Vsim(s) ≥ -r(s).
- domain assumption The value gap between simulation and reality is bounded: |Vsim(s) - V*_real(s)| < ϵ for all s.
- domain assumption If a generative model is used, its error relative to real dynamics is bounded: ||ˆp(·|s,a) - preal(·|s,a)||_1 < α.
- standard math The reward function has bounded range (Δr finite) and Vsim has bounded range (ΔV finite).
Cite this review
Pith. "Pith review of Rapidly Adapting Policies to the Real World via Simulation-Guided Fine-Tuning." pith.science (2026). https://pith.science/paper/AU37BOQN
@misc{pith2026250202705,
author = {Pith},
title = {Pith review of: Rapidly Adapting Policies to the Real World via Simulation-Guided Fine-Tuning},
year = {2026},
howpublished = {\url{https://pith.science/paper/AU37BOQN}},
note = {Machine review of arXiv:2502.02705}
}
read the original abstract
Robot learning requires a considerable amount of high-quality data to realize the promise of generalization. However, large data sets are costly to collect in the real world. Physics simulators can cheaply generate vast data sets with broad coverage over states, actions, and environments. However, physics engines are fundamentally misspecified approximations to reality. This makes direct zero-shot transfer from simulation to reality challenging, especially in tasks where precise and force-sensitive manipulation is necessary. Thus, fine-tuning these policies with small real-world data sets is an appealing pathway for scaling robot learning. However, current reinforcement learning fine-tuning frameworks leverage general, unstructured exploration strategies which are too inefficient to make real-world adaptation practical. This paper introduces the Simulation-Guided Fine-tuning (SGFT) framework, which demonstrates how to extract structural priors from physics simulators to substantially accelerate real-world adaptation. Specifically, our approach uses a value function learned in simulation to guide real-world exploration. We demonstrate this approach across five real-world dexterous manipulation tasks where zero-shot sim-to-real transfer fails. We further demonstrate our framework substantially outperforms baseline fine-tuning methods, requiring up to an order of magnitude fewer real-world samples and succeeding at difficult tasks where prior approaches fail entirely. Last but not least, we provide theoretical justification for this new paradigm which underpins how SGFT can rapidly learn high-performance policies in the face of large sim-to-real dynamics gaps. Project webpage: https://weirdlabuw.github.io/sgft/{weirdlabuw.github.io/sgft}
Figures
Figures from the paper (4 more)
Forward citations
Cited by 3 Pith papers
-
Shared Voxel-Map-Based Cooperative Indoor UAV Guidance with a Multi-Agent Soft Actor-Critic Controller
A multi-agent SAC controller using a shared voxel-map BEV representation achieves 90.3% simulated corridor success and 100% success across 50 real two-drone indoor trials after A*-based imitation fine-tuning.
-
SimLauncher: Launching Sample-Efficient Real-world Robotic Reinforcement Learning via Simulation Pre-training
Simulation-pretrained policies, with digital-twin demos for critic bootstrapping and action proposals, cut real-world RL training time while reaching near-perfect success on three manipulation tasks.
-
SLAC: Safe and Efficient Real-Robot Reinforcement Learning via Unsupervised Simulation Pre-Training
SLAC learns a latent action space in a low-fidelity simulator and uses it for real-world reinforcement learning, solving whole-body mobile manipulation tasks in under an hour without demonstrations.
Reference graph
Works this paper leans on
-
[1]
Further suppose ∆r = max s r(s) − mins r(s) and ∆V = max s Vs(s) − mins Vs(s) are finite
(Bhardwaj et al., 2020, Lemma A.1.) Suppose that∥ˆp(s, a)−preal(s, a)∥1 ≤ α. Further suppose ∆r = max s r(s) − mins r(s) and ∆V = max s Vs(s) − mins Vs(s) are finite. Then, for each policy π we may bound the H-step returns under the model and true dynamics by: ∥ ˆV πH H (s) − V πH H ∥∞ ≤ γ 1 − γH−1 1 − γ ∆r 2 + γH ∆V 2 · αH. (4) Proof. This result follows...
work page 2020
-
[3]
Randomized ensembled double q-learning: Learning fast without a model
Xinyue Chen, Che Wang, Zijian Zhou, and Keith Ross. Randomized ensembled double q-learning: Learning fast without a model. arXiv preprint arXiv:2101.05982,
-
[4]
∞X t=0 γt¯r(st) # = V π real(s) − Vs(s0) ≥ Eρπ real(s)
Suppose that Vsim is improvable and further suppose that maxs∈S |Vsim(s) − V ∗ real(s)| < ϵ. Then any policy π which satisfies A∗ H (s, π) = Q∗ H (s, π) − V ∗ H (s) ≥ −δ will satisfy: V ∗ real(s) − V π real(s) ≤ γH ϵ + δ 1 − γ . (10) Proof. Our goal is first to bound how Q∗ H (s, π) changes on expectation when applying the given policy for a single step. ...
work page 2025
-
[7]
Visual fore- sight: Model-based deep reinforcement learning for vision-based robotic control
12 Published as a conference paper at ICLR 2025 Frederik Ebert, Chelsea Finn, Sudeep Dasari, Annie Xie, Alex Lee, and Sergey Levine. Visual fore- sight: Model-based deep reinforcement learning for vision-based robotic control. arXiv preprint arXiv:1812.00568,
arXiv 2025
-
[9]
URL https://arxiv. org/abs/2305.12821. Takuya Hiraoka, Takahisa Imagawa, Taisei Hashimoto, Takashi Onishi, and Yoshimasa Tsu- ruoka. Dropout q-functions for doubly efficient reinforcement learning. arXiv preprint arXiv:2110.02034,
-
[11]
What went wrong? closing the sim-to-real gap via differentiable causal discovery
Peide Huang, Xilun Zhang, Ziang Cao, Shiqi Liu, Mengdi Xu, Wenhao Ding, Jonathan Francis, Bingqing Chen, and Ding Zhao. What went wrong? closing the sim-to-real gap via differentiable causal discovery. In Jie Tan, Marc Toussaint, and Kourosh Darvish (eds.), Conference on Robot Learning, CoRL 2023, 6-9 November 2023, Atlanta, GA, USA , volume 229 of Procee...
work page 2023
-
[12]
RMA: rapid motor adaptation for legged robots
13 Published as a conference paper at ICLR 2025 Ashish Kumar, Zipeng Fu, Deepak Pathak, and Jitendra Malik. RMA: rapid motor adaptation for legged robots. In RSS,
work page 2025
-
[13]
Dara: Dynamics-aware reward augmentation in offline reinforcement learning
Jinxin Liu, Hongyin Zhang, and Donglin Wang. Dara: Dynamics-aware reward augmentation in offline reinforcement learning. arXiv preprint arXiv:2203.06662,
Show all 35 references
-
[14]
Eureka: Human-level reward design via coding large language models
Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayara- man, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models. arXiv preprint arXiv:2310.12931,
-
[15]
Isaac gym: High performance gpu-based physics simulation for robot learning
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State. Isaac gym: High performance gpu-based physics simulation for robot learning. In NeurIPS-2021 Datasets and ...
2021
-
[16]
URL https://doi.org/10.1177/02783649231224053
doi: 10.1177/ 02783649231224053. URL https://doi.org/10.1177/02783649231224053. Marius Memmel, Andrew Wagenmaker, Chuning Zhu, Patrick Yin, Dieter Fox, and Abhishek Gupta. ASID: active exploration for system identification in robotic manipulation. CoRR, abs/2404.12308,
-
[17]
Awac: Accelerating online rein- forcement learning with offline datasets
Ashvin Nair, Abhishek Gupta, Murtaza Dalal, and Sergey Levine. Awac: Accelerating online rein- forcement learning with offline datasets. arXiv preprint arXiv:2006.09359,
2006 arXiv
-
[18]
Sim-to-real transfer of robotic control with dynamics randomization
Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel. Sim-to-real transfer of robotic control with dynamics randomization. In 2018 IEEE international conference on robotics and automation (ICRA), pp. 3803–3810. IEEE,
2018
-
[19]
14 Published as a conference paper at ICLR 2025 Aravind Rajeswaran, Vikash Kumar, Abhishek Gupta, Giulia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine
URL https://proceedings.mlr.press/ v205/qi23a.html. 14 Published as a conference paper at ICLR 2025 Aravind Rajeswaran, Vikash Kumar, Abhishek Gupta, Giulia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine. Learning Complex Dexterous Manipulation with Deep Reinforce-...
2025
-
[20]
A walk in the park: Learning to walk in 20 minutes with model-free reinforcement learning
Laura Smith, Ilya Kostrikov, and Sergey Levine. A walk in the park: Learning to walk in 20 minutes with model-free reinforcement learning. arXiv preprint arXiv:2208.07860, 2022a. Laura M. Smith, J. Chase Kew, Xue Bin Peng, Sehoon Ha, Jie Tan, and Sergey Levine. Legged robots t...
-
[21]
Integrated architectures for learning, planning, and reacting based on approxi- mating dynamic programming
Richard S Sutton. Integrated architectures for learning, planning, and reacting based on approxi- mating dynamic programming. In Machine learning proceedings 1990 , pp. 216–224. Elsevier,
1990
-
[23]
Dropo: Sim-to-real transfer with offline domain randomization
Gabriele Tiboni, Karol Arndt, and Ville Kyrki. Dropo: Sim-to-real transfer with offline domain randomization. Robotics and Autonomous Systems, 166:104432, 2023a. Gabriele Tiboni, Pascal Klink, Jan Peters, Tatiana Tommasi, Carlo D’Eramo, and Georgia Chal- vatzaki. Domain random...
-
[25]
URL https: //doi.org/10.48550/arXiv.2403.03949
doi: 10.48550/ARXIV .2403.03949. URL https: //doi.org/10.48550/arXiv.2403.03949. Homer Walke, Kevin Black, Abraham Lee, Moo Jin Kim, Max Du, Chongyi Zheng, Tony Zhao, Philippe Hansen-Estruch, Quan Vuong, Andre He, Vivek Myers, Kuan Fang, Chelsea Finn, and Sergey Levine. Bridge...
-
[26]
Exploring model-based planning with policy networks.arXiv preprint arXiv:1906.08649,
Tingwu Wang and Jimmy Ba. Exploring model-based planning with policy networks.arXiv preprint arXiv:1906.08649,
1906 arXiv
-
[27]
Lyapunov design for robust and efficient robotic reinforcement learning
Tyler Westenbroek, Fernando Castaneda, Ayush Agrawal, Shankar Sastry, and Koushil Sreenath. Lyapunov design for robust and efficient robotic reinforcement learning. arXiv preprint arXiv:2208.06721,
-
[29]
roboticsproceedings.org/rss13/p48.html
URL http://www. roboticsproceedings.org/rss13/p48.html. Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, Sean Kirmani, Kuang-Huei Lee, Montse Gonzalez Are- nas, Hao-Tien Lewis Chiang, Tom Erez, Leonard Hasenclever, Jan Humplik, et al. Language to rewards for robotic skill synthesis. arX...
-
[30]
Efficient online reinforce- ment learning fine-tuning need not retain offline data
Zhiyuan Zhou, Andy Peng, Qiyang Li, Sergey Levine, and Aviral Kumar. Efficient online reinforce- ment learning fine-tuning need not retain offline data. arXiv preprint arXiv:2412.07762,
-
[33]
γH Vsim(sH ) + H−1X t=1 γtr(st) − Vsim(s0) # = E γH Vsim(sH ) − γH−1Vsim(sH−1) + γH r(sH−1) + E
Suppose that supa Es∼preal(s,a)[γVsim(s′)]−Vsim(s) > −r(s). Then we have V ∗ H (s) ≥ V ∗ H−1(s) for each s ∈ S. Then for each s ∈ Swe have: V ∗ H (s) ≥ V ∗ H−1(s) (8) Proof. Fix an initial condition s0 ∈ S. Let π be arbitrary, and fix the shorthand π∗ = {π∗ 0, . . . , π∗ H−1} ...
2025
-
[512]
The dynamics model is implemented as a delta dynamics model where model predictions are added to the input state to generate next states. The policy net- work produces the mean µa and a state-dependent log standard deviation log σa which is jointly 20 Published as a conference...
2025
-
[1536]
We don’t train on any simulation data during real-world fine-tuning because we empirically found it didn’t help fine-tuning performance in our settings
The policy, Q-network, and dynamics model are all trained jointly on the real data during SAC fine-tuning. We don’t train on any simulation data during real-world fine-tuning because we empirically found it didn’t help fine-tuning performance in our settings. 21 Published as a...
2025
-
[1991]
doi: 10.1145/122344.122377
ISSN 0163-5719. doi: 10.1145/122344.122377. URL https: //doi.org/10.1145/122344.122377. DROID Collaboration team. Droid: A large-scale in-the-wild robot manipulation dataset. In Robotics Science and Systems (RSS),
-
[2008]
16 Published as a conference paper at ICLR 2025 A P ROOFS Notation Recap
ISBN 9781577353683. 16 Published as a conference paper at ICLR 2025 A P ROOFS Notation Recap. We remind the reviewer of notation we have built up throughout the paper. We use the ‘hat’ notation to denote a generative dynamics model ˆp, as well that the optimal values ˆV ∗ H, ˆ...
2025
-
[2012]
Reconciling reality through simulation: A real-to-sim-to-real approach for robust ma- nipulation
Marcel Torne, Anthony Simeonov, Zechu Li, April Chan, Tao Chen, Abhishek Gupta, and Pulkit Agrawal. Reconciling reality through simulation: A real-to-sim-to-real approach for robust ma- nipulation. CoRR, abs/2403.03949,
-
[2016]
Imitation bootstrapped reinforcement learn- ing
Hengyuan Hu, Suvir Mirchandani, and Dorsa Sadigh. Imitation bootstrapped reinforcement learn- ing. arXiv preprint arXiv:2311.02198,
-
[2017]
Fine-tuning reinforcement learning models is secretly a forgetting mitigation problem
15 Published as a conference paper at ICLR 2025 Maciej Wołczyk, Bartłomiej Cupiał, Mateusz Ostaszewski, Michał Bortkiewicz, Michal Zajkac, Razvan Pascanu, Lukasz Kucinski, and Piotr Milo’s. Fine-tuning reinforcement learning models is secretly a forgetting mitigation problem. ...
2025 arXiv
-
[2018]
Off-dynamics reinforcement learning: Training for transfer with domain classifiers
Benjamin Eysenbach, Swapnil Asawa, Shreyas Chaudhari, Sergey Levine, and Ruslan Salakhutdi- nov. Off-dynamics reinforcement learning: Training for transfer with domain classifiers. arXiv preprint arXiv:2006.13916,
2006 arXiv
-
[2019]
Mohak Bhardwaj, Sanjiban Choudhury, and Byron Boots
URL http://arxiv.org/abs/1912.06680. Mohak Bhardwaj, Sanjiban Choudhury, and Byron Boots. Blending mpc & value function approxi- mation for efficient reinforcement learning. arXiv preprint arXiv:2012.05909,
1912 arXiv
-
[2021]
Urdformer: A pipeline for constructing articulated simula- tion environments from real-world images
Zoey Chen, Aaron Walsman, Marius Memmel, Kaichun Mo, Alex Fang, Karthikeya Vemuri, Alan Wu, Dieter Fox, and Abhishek Gupta. Urdformer: A pipeline for constructing articulated simula- tion environments from real-world images. arXiv preprint arXiv:2405.11656,
-
[2022]
Yuqing Du, Olivia Watkins, Trevor Darrell, Pieter Abbeel, and Deepak Pathak
URL http://papers.nips.cc/paper_files/paper/2022/hash/ 27c546ab1e4f1d7d638e6a8dfbad9a07-Abstract-Conference.html. Yuqing Du, Olivia Watkins, Trevor Darrell, Pieter Abbeel, and Deepak Pathak. Auto-tuned sim-to- real transfer. In 2021 IEEE International Conference on Robotics an...
2022
-
[2024]
ProcTHOR: Large-scale embodied AI using procedural generation
Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Kiana Ehsani, Jordi Salvador, Winson Han, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi. ProcTHOR: Large-scale embodied AI using procedural generation. In Advances in Neural Infor- mation Processing Systems 35: A...
2022
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.