REVIEW 4 major objections 5 minor 71 references
Egocentric human recovery clips, routed through a gated corrective-intent bottleneck, lift real-robot recovery success from failure starts to 85%, up from 52.5% with robot-only recovery and 71.2% with direct human-recovery co-training.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 11:49 UTC pith:S2ZKIIZR
load-bearing objection Genuinely new combination and a clean controlled comparison; single-seed runs and an unproven DCT target keep the mechanism claim from being airtight. the 4 major comments →
EgoRecovery: Acquiring Failure Recovery Ability Through Human Recovery Demonstration
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that failure-recovery behavior transfers from human to robot when the human signal is compressed to a corrective-intent target rather than used as action supervision. EgoRecovery co-trains robot success, robot recovery, and human recovery data: the shared intent head predicts a 4D DCT magnitude envelope of the next 16 phase points of end-effector motion on valid recovery windows; the recovery gate head predicts whether the current observation is a recovery state; and the robot action decoder is modulated by the predicted intent through a gated residual feature-wise modulation only when the gate is high. Robot recovery data supplies the same intent target together with ex
What carries the argument
The load-bearing object is corrective intent, defined in Eq. (1) as the magnitude envelope of the next L=16 phase points of end-effector motion, projected onto K=4 low-frequency DCT basis functions and scaled by embodiment scale. It captures how much correction remains and how it unfolds over time, while deliberately discarding direction and contact. The second component is the recovery gate head, which predicts the probability that the current observation requires recovery and produces a scalar that gates the residual feature-wise modulation of the robot action decoder. Together they form a bottleneck: human recovery broadens the failure states that train the intent predictor, robot recover
Load-bearing premise
The load-bearing premise is that a directionless, embodiment-scaled magnitude profile of the remaining correction is an informative signal shared by human and robot recovery; if corrective information lives mainly in direction or contact rather than magnitude, the transfer target is vacuous and the reported gains would come from elsewhere in the pipeline.
What would settle it
Look for a recovery task whose correction is predominantly directional with little translation—e.g., reorienting a grasped object in place. Compute the paper's DCT magnitude features for human and robot recovery clips: if the two distributions show no more overlap than success pairs, and EgoRecovery's Recovery SR advantage over direct human-recovery co-training disappears when the intent loss is removed, the corrective-intent target is not carrying the transfer. The paper's own cup-brush diagnostic (KSR=0.21 vs KSS=0.17) is the first place to look for this failure.
If this is right
- If correct, egocentric human recovery segments are a practical supervision source: under the paper's protocol, humans produce roughly 10.5× more accepted recovery episodes per operator hour than robot teleoperation, and scaling human recovery from 200 to 800 episodes raises Recovery SR from 75.0% to 90.0% under a fixed 50+50 robot anchor.
- The gated bottleneck is the active ingredient, not the extra data alone: with identical data, EgoRecovery reaches 85.0% Recovery SR versus 71.2% for direct human-recovery co-training; removing the intent loss drops Recovery SR to 65.0%, removing the corrective-segment mask to 55.0%, and removing gated modulation to 65.0%.
- Robot recovery grounding remains necessary: human recovery alone (300 episodes plus robot success data) reaches only 8.8% Recovery SR, and with only 20 robot recovery episodes, adding 600 human recovery episodes reaches 75.0%, below the 85.0% achieved with 50 robot + 300 human episodes.
- The recovery gate preserves nominal execution: forcing intent modulation on all states drops Initial SR from 80.0% to 70.0%, while the gated version keeps nominal behavior intact and still improves recovery.
Where Pith is reading between the lines
- Editorial inference: if the throughput and transfer results hold beyond the four tabletop tasks, the collection recipe 'stage a failure, record a short human clip, keep a modest robot anchor' could make recovery supervision cheap enough to gather at scale for many manipulation skills, including in non-laboratory settings.
- Editorial inference: the magnitude-only intent target is an explicit wager that correction information lives in 'how much' rather than 'which way'; a natural extension is a directional or orientation-aware intent target, motivated by the paper's own cup-brush diagnostic in which recovery DCT features are less aligned across embodiments (KS 0.21 vs 0.17 for success).
- Editorial inference: the recovery gate's sharp drop near the annotated recovery boundary (within ±1 frame on all tested robot episodes) hints that the same head could serve as a runtime failure detector; the paper does not claim this.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes EgoRecovery, a co-training framework for robot failure recovery that augments limited robot recovery demonstrations with egocentric human recovery segments. Human and robot data are aligned through a compact corrective-intent target: a low-frequency DCT magnitude envelope of future end-effector motion (Eq. 1). A learned recovery gate modulates robot decoder features via gated FiLM so that corrective intent affects action decoding only in recovery states. Experiments on four real-world tabletop manipulation tasks report that EgoRecovery raises average Recovery SR from 52.5% (robot-only) and 71.2% (direct human recovery mix) to 85.0%, while preserving Initial SR at 80.0%. The appendix provides confidence intervals, ablations, and mechanism diagnostics for the intent target, the learned bottleneck, and the gate.
Significance. If the result holds, the paper makes a strong practical contribution: egocentric human recovery data can be collected at roughly 10x the throughput of robot teleoperation, and a gated corrective-intent bottleneck makes that data usable for robot recovery while leaving embodiment-specific execution to robot supervision. The experimental design is a genuine strength: the key comparison (direct human recovery mix vs. EgoRecovery) holds data composition and backbone fixed, the ablation study isolates the intent loss, the mask, the modulation path, and the gate, and the appendix reports Wilson confidence intervals and count-level audits. The code release and the detailed collection/annotation protocol support reproducibility. The main weaknesses are statistical (single seed, overlapping confidence intervals for the headline contrast) and mechanistic (the paper's own KS diagnostic weakens the claim that Eq. (1)'s DCT target is the shared carrier of recovery information for cup-brush insertion).
major comments (4)
- [§4.2, Table 6, Appendix H, Appendix I] The headline claim that EgoRecovery improves Recovery SR over the direct human recovery mix rests on a difference of 85.0 vs. 71.2, but the Wilson 95% intervals overlap: EgoRecovery [75.6, 91.2], direct mix [60.5, 80.0]. Moreover, Appendix H states that each reported model was trained with a single seed. With only 80 recovery rollouts per condition and one seed, the central 13.8-point gain is not statistically distinguished at the aggregate level. Please report multiple seeds (at least 3) and their spread, or a significance test (e.g., a paired or permutation test over the fixed starts). Without this, the main quantitative claim is not established.
- [§3.2, Eq. (1), App. J, Fig. 8, Table 2] The proposed mechanism is that the DCT magnitude envelope in Eq. (1) is a shared corrective-intent signal that transfers human recovery information to the robot. The paper's own KS diagnostic in Fig. 8 shows the opposite for cup-brush insertion: the recovery-window KS distance (KSR=0.21) is larger than the success-window distance (KSS=0.17). Recovery windows are exactly where transfer should be most visible. The 'without intent loss' ablation (Table 2) shows that the intent pathway is used, but an auxiliary prediction target can regularize the trunk or gate even if y_t carries little cross-embodiment information; the 'c_t zero' intervention shows only that the predicted intent is used, not that its values encode human correction. Since the aggregate 85.0% is the primary evidence for the contribution, this leaves the mechanism under-supported for one of the four tasks. Please provide a pe
- [Appendix D and Limitations] The evaluation starts are drawn from the same task-level failure families used to collect human and robot recovery training data. The benchmark therefore measures within-family recovery, not generalization to unseen failure modes or to qualitatively new recovery skills. This is honestly acknowledged in the Limitations section, but the abstract and introduction state the contribution in broader terms ('acquiring failure recovery ability'). The framing should be adjusted throughout to make the within-family scope explicit, or new evidence should be added for held-out failure families.
- [Abstract and Table 1] The abstract claims human recovery yields 'more than 10x as much valid recovery data per hour' compared to robot teleoperation. Table 1 gives per-task ratios of 10.9x, 10.3x, 11.4x, and 9.8x; cube stacking is below 10x. The claim should be qualified as an average (10.5x) or the per-task values should be reported in the abstract. This is not load-bearing but is a factual inconsistency in a headline claim.
minor comments (5)
- [Fig. 10] The left panel title reads 'Cu brush insertion' (missing hyphen/space). Please fix.
- [§3.2 and App. G] The embodiment scale s_d is referenced in Eq. (1) and App. G but never defined precisely. State how s_d is computed for each embodiment and task.
- [App. J] The dimension-normalized sliced Wasserstein distance is central to the bottleneck diagnostic, but the normalization formula is only described verbally ('1/sqrt(d) dependence'). Please give the exact expression used.
- [App. H] The appendix says all reported models used a single seed. Even for exploratory scalability experiments, it would be useful to state this prominently in the main text, since the main result is currently a point estimate without seed variance.
- [Table 5] Color jitter and ImageNet normalization are listed as the only input augmentations; typical augmentation magnitudes and normalization statistics should be given for reproducibility.
Circularity Check
No significant circularity: corrective intent and gate are supervised auxiliary signals, and the headline Recovery SR is an independent closed-loop measurement.
full rationale
The paper's central comparison is empirical: EgoRecovery vs direct mix vs robot-only are closed-loop rollouts from fixed recovery starts, with the same data composition for EgoRecovery and the direct mix. Eq. (1) defines a hand-designed DCT-magnitude intent target from future end-effector motion; it is not derived from or fitted to the evaluation outcome. The intent head and gate are trained by supervised losses (Eq. 3) on targets computed from human/robot recovery segments and annotations; deployment uses only robot observations, so no fitted parameter is renamed as a prediction of the reported Recovery SR. The gain (71.2 to 85.0) is an empirical result and is not forced by construction. The paper discloses the main scope limitation in the Limitations section ('These findings are limited to related off-nominal states within the same task-level recovery family in tabletop manipulation.') and in Appendix D ('The benchmark therefore tests transfer within related recovery behavior rather than unrelated skills.'); this within-family evaluation is a validity/external-validity caveat, not a circularity. The Fig. 8 KS diagnostic (KSR=0.21 vs KSS=0.17 for cup brush) weakens the claim that the specific DCT magnitude target carries the shared recovery signal for that task, but a weak or contradicted assumption is a correctness risk, not a reduction of the result to its inputs. No load-bearing self-citations appear: architecture references (HPT [53], EgoMimic-style co-training [19]) and compact-behavior-transfer citations ([17,21,58]) are external works. The cost-matched budget uses the independently measured 10x throughput of Table 1; the conversion is an accounting device, not a fitted output. Accordingly, no circular step can be exhibited from the paper's equations or citations.
Axiom & Free-Parameter Ledger
free parameters (5)
- DCT coefficient count K =
4
- Future window length L =
16 phase points (8% of episode)
- Embodiment scale s_d =
per-embodiment calibration, value not reported
- Loss weights lambda_c, lambda_g, lambda_n =
0.05, 0.05, 0.01
- Active effector selection =
one task-designated active arm
axioms (6)
- domain assumption DCT magnitude envelope of normalized future end-effector motion (Eq. 1) is an informative shared corrective-intent signal across embodiments
- domain assumption Recovery boundary t_rec and phase labels s_t/m_t can be annotated reliably
- standard math HPT-style shared transformer with embodiment-specific adapters can co-train human and robot observations
- standard math Behavior cloning on 100-step action chunks from robot teleoperation is a valid policy-learning objective
- domain assumption Recovery evaluation from off-nominal starts in the same task-level failure families as training is a meaningful test of recovery ability
- domain assumption HaWoR hand-pose reconstruction from egocentric video is accurate enough for intent targets
invented entities (1)
-
Corrective-intent space (DCT magnitude envelope c_t)
no independent evidence
read the original abstract
Robust embodied robots should be able to recover from failures and retry tasks in order to operate reliably in unstructured and noisy real-world environments. Achieving this capability requires training policies on data that captures recovery behaviors. However, collecting such data through robot teleoperation is difficult to scale, as it is time-consuming to induce diverse failure states, perform corrective actions, and reset the environment. This challenge is further exacerbated by the high diversity of failure modes, which demands substantially more recovery data than success demonstrations. In this work, we show that egocentric human data capturing failure recovery processes provides a scalable alternative. By efficiently arranging task-level failure configurations and recording short recovery segments, human operators can generate more than 10x as much valid recovery data per hour compared to robot teleoperation under our protocol. To address the embodiment gap between human and robot, we propose EgoRecovery, a co-training framework for learning recovery behavior, where human recovery demonstrations are aligned to a compact corrective-intent space shared with robot data, which captures the timing and magnitude of correction. Only a small number of robot recovery demonstrations are required to connect this intent to executable robot actions. At deployment, a learned recovery gate predicts when correction is needed from robot observations and activates the corrective intent only in recovery states. Experiments on real-world recovery tasks show that EgoRecovery improves success from failure starts over robot-only recovery, direct co-training with human recovery data, and direct intent-transfer baselines.
Reference graph
Works this paper leans on
-
[1]
Human-to-robotimitationinthewild
ShikharBahl,AbhinavGupta,andDeepakPathak. Human-to-robotimitationinthewild. In ProceedingsofRobotics: Science and Systems(RSS), 2022
2022
-
[2]
H-RDT: Human manipulation enhanced bimanual robotic manipulation.arXivpreprintarXiv:2507.23523, 2025
Hongzhe Bi, Lingxuan Wu, Tianwei Lin, Hengkai Tan, Zhizhong Su, Hang Su, and Jun Zhu. H-RDT: Human manipulation enhanced bimanual robotic manipulation.arXivpreprintarXiv:2507.23523, 2025
Pith/arXiv arXiv 2025
-
[3]
Xiongyi Cai, Ri-Zhao Qiu, Geng Chen, Lai Wei, Isabella Liu, Tianshu Huang, Xuxin Cheng, and Xiaolong Wang. In-N-On: Scaling egocentric manipulation with in-the-wild and on-task data.arXivpreprintarXiv:2511.15704, 2025
arXiv 2025
-
[4]
Diffusion policy: Visuomotor policy learning via action diffusion
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. InProceedings of Robotics: Science and Systems (RSS), 2023
2023
-
[5]
Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots
Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake, and Shuran Song. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots. InProceedings of Robotics: Science and Systems(RSS), 2024
2024
-
[6]
WeishengDai,KaiLan,JianyiZhou,BoZhao,XiuSu,JunwenTong,WeiliGuan,andShuoYang. ConLA:Contrastive latent action learning from human videos for robotic manipulation.arXiv preprintarXiv:2602.00557, 2026
arXiv 2026
-
[7]
Yicheng Feng, Wanpeng Zhang, Ye Wang, Hao Luo, Haoqi Yuan, Sipeng Zheng, and Zongqing Lu. Spatial-aware VLA pretraining through visual-physical alignment from human videos.arXivpreprintarXiv:2512.13080, 2025
arXiv 2025
-
[8]
Harry Freeman, Chung Hee Kim, and George Kantor. WARPED: Wrist-aligned rendering for robot policy learning from egocentric human demonstrations.arXivpreprintarXiv:2604.10809, 2026
Pith/arXiv arXiv 2026
-
[9]
Shenyuan Gao, William Liang, Kaiyuan Zheng, Ayaan Malik, Seonghyeon Ye, Sihyun Yu, Wei-Cheng Tseng, Yuzhu Dong, Kaichun Mo, Chen-Hsuan Lin, Qianli Ma, Seungjun Nah, Loic Magne, Jiannan Xiang, Yuqi Xie, Ruijie Zheng, Dantong Niu, You Liang Tan, K. R. Zentner, George Kurian, Suneel Indupuru, Pooya Jannaty, Jinwei Gu, Jun Zhang, Jitendra Malik, Pieter Abbeel,...
Pith/arXiv arXiv 2026
-
[10]
Manish Kumar Govind, Dominick Reilly, Pu Wang, and Srijan Das. UniLACT: Depth-aware RGB latent action learning for vision-language-action models.arXivpreprintarXiv:2602.20231, 2026
Pith/arXiv arXiv 2026
-
[11]
Ego4D: Around the world in 3,000 hours of egocentric video
Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4D: Around the world in 3,000 hours of egocentric video. In ProceedingsoftheIEEE/CVFConferenceonComputerVisionandPatternRecognition(CVPR),pages18995–19012, 2022
2022
-
[12]
Yifan Han, Jianxiang Liu, Haoyu Zhang, Yuqi Gu, Yunhan Guo, and Wenzhao Lian. BridgeACT: Bridging human demonstrations to robot actions via unified tool-target affordances.arXiv preprintarXiv:2604.23249, 2026
Pith/arXiv arXiv 2026
-
[13]
2HandedAfforder: Learning precise actionable bimanual affordances from human videos
Marvin Heidinger, Snehal Jauhri, Vignesh Prasad, and Georgia Chalvatzaki. 2HandedAfforder: Learning precise actionable bimanual affordances from human videos. InProceedingsofthe IEEE/CVFInternational Conferenceon ComputerVision(ICCV), pages 14743–14753, 2025
2025
-
[14]
ThriftyDAg- ger: Budget-aware novelty and risk gating for interactive imitation learning
RyanHoque, AshwinBalakrishna, EllenNovoseller, AlbertWilcox, DanielS.Brown, andKenGoldberg. ThriftyDAg- ger: Budget-aware novelty and risk gating for interactive imitation learning. InProceedings of the 5th Conference on RobotLearning (CoRL), volume 164 ofProceedingsof MachineLearning Research, pages 598–608. PMLR, 2021
2021
-
[15]
Yoon, Mouli Sivapurapu, and Jian Zhang
Ryan Hoque, Peide Huang, David J. Yoon, Mouli Sivapurapu, and Jian Zhang. EgoDex: Learning dexterous manipulation from large-scale egocentric video.arXivpreprintarXiv:2505.11709, 2025
Pith/arXiv arXiv 2025
-
[16]
Zheyuan Hu, Robyn Wu, Naveen Enock, Jasmine Li, Riya Kadakia, Zackory Erickson, and Aviral Kumar. RaC: Robot learning for long-horizon tasks by scaling recovery and correction.arXivpreprintarXiv:2509.07953, 2025
Pith/arXiv arXiv 2025
-
[17]
MINT: Mimic intent, not just trajectories.arXiv preprintarXiv:2602.08602, 2026
Renming Huang, Chendong Zeng, Wenjing Tang, Jintian Cai, Cewu Lu, and Panpan Cai. MINT: Mimic intent, not just trajectories.arXiv preprintarXiv:2602.08602, 2026
arXiv 2026
-
[18]
Tingzheng Jia, Kan Guo, Lanping Qian, Yongli Hu, Daxin Tian, Guixian Qu, Chunmian Lin, Baocai Yin, and Jiapu Wang. AnchorRefine: Synergy-manipulationbasedontrajectoryanchorandresidualrefinementforvision-language- action models.arXiv preprintarXiv:2604.17787, 2026. 11
Pith/arXiv arXiv 2026
-
[19]
EgoMimic: Scaling imitation learning via egocentric video
SimarKareer,DhruvPatel,RyanPunamiya,PranayMathur,ShuoCheng,ChenWang,JudyHoffman,andDanfeiXu. EgoMimic: Scaling imitation learning via egocentric video. InProceedingsofthe IEEEInternational Conferenceon RoboticsandAutomation(ICRA). IEEE, 2025
2025
-
[20]
Kochenderfer
Michael Kelly, Chelsea Sidrane, Katherine Driggs-Campbell, and Mykel J. Kochenderfer. HG-DAgger: Interactive imitation learning with human experts. InProceedings of the IEEE International Conference on Robotics and Automation(ICRA), pages 8077–8083. IEEE, 2019
2019
-
[21]
UniSkill: Imitating human videos via cross-embodiment skill representations
Hanjung Kim, Jaehyun Kang, Hyolim Kang, Meedeum Cho, Seon Joo Kim, and Youngwoon Lee. UniSkill: Imitating human videos via cross-embodiment skill representations. InProceedingsofthe 9th ConferenceonRobotLearning (CoRL), volume 305 ofProceedingsofMachineLearning Research, pages 4269–4294. PMLR, 2025
2025
-
[22]
DART: Noise injection for robust imitation learning
Michael Laskey, Jonathan Lee, Roy Fox, Anca Dragan, and Ken Goldberg. DART: Noise injection for robust imitation learning. In Proceedings of the 1st Annual Conference on Robot Learning (CoRL), volume 78 ofProceedings of MachineLearning Research, pages 143–156. PMLR, 2017
2017
-
[23]
MVP-LAM:Learningaction-centriclatentactionviacross-viewpointreconstruction
Jung Min Lee, Dohyeok Lee, Seokhun Ju, Taehyun Cho, Jin Woo Koo, Li Zhao, Sangwoo Hong, and Jungwoo Lee. MVP-LAM:Learningaction-centriclatentactionviacross-viewpointreconstruction. arXivpreprintarXiv:2602.03668, 2026
Pith/arXiv arXiv 2026
-
[24]
Yu Lei, Minghuan Liu, Abhiram Maddukuri, Zhenyu Jiang, and Yuke Zhu. A mechanistic analysis of sim-and-real co-training in generative robot policies.arXivpreprint arXiv:2604.13645, 2026
Pith/arXiv arXiv 2026
-
[25]
GazeVLA: Learning human intention for robotic manipulation.arXivpreprint arXiv:2604.22615, 2026
Chengyang Li, Kaiyi Xiong, Yuan Xu, Lei Qian, Yizhou Wang, and Wentao Zhu. GazeVLA: Learning human intention for robotic manipulation.arXivpreprint arXiv:2604.22615, 2026
Pith/arXiv arXiv 2026
-
[26]
ManipTrans: Efficient dexterous bimanual manipulation transfer via residual learning
Kailin Li, Puhao Li, Tengyu Liu, Yuyang Li, and Siyuan Huang. ManipTrans: Efficient dexterous bimanual manipulation transfer via residual learning. InProceedingsofthe IEEE/CVFConferenceonComputerVisionand PatternRecognition(CVPR), 2025
2025
-
[27]
Scalable vision-language-action model pretraining for robotic manipulation with real-life human activity videos
Qixiu Li, Yu Deng, Yaobo Liang, Lin Luo, Lei Zhou, Chengtang Yao, Lingqi Zeng, Zhiyuan Feng, Huizhi Liang, Sicheng Xu, Yizhong Zhang, Xi Chen, Hao Chen, Lily Sun, Dong Chen, Jiaolong Yang, and Baining Guo. Scalable vision-language-action model pretraining for robotic manipulation with real-life human activity videos. In Proceedings ofthe IEEE Internationa...
2026
-
[28]
Yaxuan Li, Zhongyi Zhou, Yefei Chen, Yanjiang Guo, Jiaming Liu, Shanghang Zhang, Jianyu Chen, and Yichen Zhu. Hi-WM: Human-in-the-world-model for scalable robot post-training.arXiv preprintarXiv:2604.21741, 2026
Pith/arXiv arXiv 2026
-
[29]
EgoLive: A large-scale egocentric dataset from real-world human tasks
Yihang Li, Xuelong Wei, Jingzhou Luo, Yingjing Xiao, Yibo Bai, Guangyuan Zhou, Teng Zou, Chenguang Gui, Jiajun Wen, He Zhang, Kangliang Chen, Xing Pan, Shuaiyan Liu, Daming Wang, Tao An, Jiayi Li, Shibo Jin, Wanwan Zhang, Tianyu Wang, Boren Wei, Zhixuan Huang, Fangsheng Liu, Ruodai Li, Hui Zhang, Anson Li, Yicheng Gong, Peng Cao, Jiaming Liang, and Liang ...
Pith/arXiv arXiv 2026
-
[30]
Fanqi Lin, Kushal Arora, Jean Mercat, Haruki Nishimura, Paarth Shah, Chen Xu, Mengchao Zhang, Mark Zolotas, Maya Angeles, Owen Pfannenstiehl, Andrew Beaulieu, and Jose Barreiros. A systematic study of data modalities and strategies for co-training large behavior models for robot manipulation.arXiv preprintarXiv:2602.01067, 2026
arXiv 2026
-
[31]
Model-based runtime monitoring with interactive imitationlearning
Huihan Liu, Shivin Dass, Roberto Martín-Martín, and Yuke Zhu. Model-based runtime monitoring with interactive imitationlearning. In ProceedingsoftheIEEEInternationalConferenceonRoboticsandAutomation(ICRA).IEEE, 2024
2024
-
[32]
EgoZero: Robot learning from smart glasses.arXivpreprintarXiv:2505.20290, 2025
Vincent Liu, Ademi Adeniji, Haotian Zhan, Siddhant Haldar, Raunaq Bhirangi, Pieter Abbeel, and Lerrel Pinto. EgoZero: Robot learning from smart glasses.arXivpreprintarXiv:2505.20290, 2025
Pith/arXiv arXiv 2025
-
[33]
REFLECT: Summarizing robot experiences for failure explanation and correction
Zeyi Liu, Arpit Bahety, and Shuran Song. REFLECT: Summarizing robot experiences for failure explanation and correction. InProceedingsofthe7thConferenceonRobotLearning(CoRL),volume229of ProceedingsofMachine Learning Research. PMLR, 2023
2023
-
[34]
Hao Luo, Ye Wang, Wanpeng Zhang, Haoqi Yuan, Yicheng Feng, Haiweng Xu, Sipeng Zheng, and Zongqing Lu. Joint-aligned latent action: Towards scalable VLA pretraining in the wild.arXivpreprintarXiv:2602.21736, 2026. 12
arXiv 2026
-
[35]
Hao Luo, Ye Wang, Wanpeng Zhang, Sipeng Zheng, Ziheng Xi, Chaoyi Xu, Haiweng Xu, Haoqi Yuan, Chi Zhang, Yiqing Wang, Yicheng Feng, and Zongqing Lu. Being-H0.5: Scaling human-centric robot learning for cross-embodiment generalization.arXiv preprintarXiv:2601.12993, 2026
arXiv 2026
-
[36]
Learning latent plans from play
Corey Lynch, Mohi Khansari, Ted Xiao, Vikash Kumar, Jonathan Tompson, Sergey Levine, and Pierre Sermanet. Learning latent plans from play. InProceedings of the 3rd Conference on RobotLearning (CoRL), volume 100 of Proceedings ofMachineLearning Research, pages 1113–1132. PMLR, 2020
2020
-
[37]
Robot learning from human videos: A survey.arXiv preprintarXiv:2604.27621, 2026
Junyi Ma, Erhang Zhang, Haoran Yang, Ditao Li, Chenyang Xu, Guangming Wang, and Hesheng Wang. Robot learning from human videos: A survey.arXiv preprintarXiv:2604.27621, 2026
Pith/arXiv arXiv 2026
-
[38]
What matters in learning from offline human demonstrations for robot manipulation
AjayMandlekar,DanfeiXu,JosiahWong,SoroushNasiriany,ChenWang,RohunKulkarni,LiFei-Fei,SilvioSavarese, Yuke Zhu, and Roberto Martín-Martín. What matters in learning from offline human demonstrations for robot manipulation. In Proceedings of the 5th Conference on Robot Learning (CoRL), volume 164 ofProceedings of MachineLearning Research, pages 1678–1690. PMLR, 2021
2021
-
[39]
MimicGen: A data generation system for scalable robot learning using human demonstrations
Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, and Dieter Fox. MimicGen: A data generation system for scalable robot learning using human demonstrations. InProceedings ofthe 7th ConferenceonRobotLearning (CoRL), volume 229 ofProceedingsofMachineLearning Research, pages 1820–1864. PMLR, 2023
2023
-
[40]
R3M: A universal visual representation for robot manipulation
Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3M: A universal visual representation for robot manipulation. InProceedings of the 6th Conference on RobotLearning (CoRL), volume 205 ofProceedingsofMachineLearning Research, pages 892–909. PMLR, 2022
2022
-
[41]
Open X-Embodiment: Robotic learning datasets and RT-X models
Abhishek Padalkar, Acorn Pooley, Ajinkya Jain, Alex Bewley, Alex Herzog, Alex Irpan, Alexander Khazatsky, Anant Rai, Anikait Singh, Anthony Brohan, et al. Open X-Embodiment: Robotic learning datasets and RT-X models. InProceedings of the IEEE International Conferenceon Roboticsand Automation(ICRA), pages 6892–6903. IEEE, 2024
2024
-
[42]
SOP: A scalable online post-training system for vision-language-action models
Mingjie Pan, Siyuan Feng, Qinglin Zhang, Xinchen Li, Jianheng Song, Chendi Qu, Yi Wang, Chuankang Li, Ziyu Xiong, Zhi Chen, Yi Liu, and Jianlan Luo. SOP: A scalable online post-training system for vision-language-action models. arXivpreprint arXiv:2601.03044, 2026
arXiv 2026
-
[43]
FAST: Efficient action tokenization for vision-language-action models
Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. FAST: Efficient action tokenization for vision-language-action models. InProceedings of Robotics: Science and Systems(RSS), 2025
2025
-
[44]
Zhu, Simar Kareer, Judy Hoffman, and Danfei Xu
Ryan Punamiya, Dhruv Patel, Patcharapong Aphiwetsa, Pranav Kuppili, Lawrence Y. Zhu, Simar Kareer, Judy Hoffman, and Danfei Xu. EgoBridge: Domain adaptation for generalizable imitation from egocentric human data. In Advancesin NeuralInformationProcessingSystems(NeurIPS), 2025
2025
-
[45]
Ryan Punamiya, Simar Kareer, Zeyi Liu, Josh Citron, Ri-Zhao Qiu, Xiongyi Cai, Alexey Gavryushin, Jiaqi Chen, Davide Liconti, Lawrence Y. Zhu, Patcharapong Aphiwetsa, Baoyu Li, Aniketh Cheluva, Pranav Kuppili, Yangcen Liu,DhruvPatel,AidanGao,Hye-YoungChung,RyanCo,ReneeZbizika,JeffLiu,XiaomengXu,HaoyuXiong,Geng Chen, Sebastiano Oliani, Chenyu Yang, Xi Wang,...
Pith/arXiv arXiv 2026
-
[46]
Gordon, and J
Stéphane Ross, Geoffrey J. Gordon, and J. Andrew Bagnell. A reduction of imitation learning and structured predictiontono-regretonlinelearning. In Proceedingsofthe14thInternationalConferenceonArtificialIntelligence and Statistics(AISTATS), volume 15 ofProceedingsofMachineLearning Research, pages 627–635. PMLR, 2011
2011
-
[47]
Jiaqi Shi, Xulong Zhang, Xiaoyang Qu, and Jianzong Wang. CARE: Multi-task pretraining for latent continuous action representation in robot control.arXivpreprintarXiv:2601.22467, 2026
arXiv 2026
-
[48]
ZeroMimic: Distilling robotic manipulation skills from web videos
Junyao Shi, Zhuolun Zhao, Tianyou Wang, Ian Pedroza, Amy Luo, Jie Wang, Jason Ma, and Dinesh Jayaraman. ZeroMimic: Distilling robotic manipulation skills from web videos. InProceedings of the IEEE International Conferenceon Roboticsand Automation(ICRA). IEEE, 2025
2025
-
[49]
Hand-object interaction pretraining from videos
Himanshu Gaurav Singh, Antonio Loquercio, Carmelo Sferrazza, Jane Wu, Haozhi Qi, Pieter Abbeel, and Jitendra 13 Malik. Hand-object interaction pretraining from videos. InProceedings of the IEEE International Conference on RoboticsandAutomation(ICRA). IEEE, 2025
2025
-
[50]
arXivpreprintarXiv:2602.10098, 2026
Jingwen Sun, Wenyao Zhang, Zekun Qi, Shaojie Ren, Zezhi Liu, Hanxin Zhu, Guangzhong Sun, Xin Jin, and Zhibo Chen.VLA-JEPA:Enhancingvision-language-actionmodelwithlatentworldmodel. arXivpreprintarXiv:2602.10098, 2026
arXiv 2026
-
[51]
MimicPlay: Long-horizon imitation learning by watching human play
Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu, and Anima Anandkumar. MimicPlay: Long-horizon imitation learning by watching human play. InProceedings of the 7th Conference on RobotLearning (CoRL), volume 229 ofProceedingsofMachineLearning Research. PMLR, 2023
2023
-
[52]
James Wang, Primo Pu, Zephyr Fung, Alex Wang, Sam Wang, Bender Deng, Kevin Wang, Zivid Liu, Chris Pan, Panda Yang, Andy Zhai, Lucy Liang, Shalfun Li, Johnny Sun, Jacky Xu, Will Tian, Kai Yan, Kohler Ye, Scott Li, Qian Wang, Roy Gan, and Hao Wang. XRZero-G0: Pushing the frontier of dexterous robotic manipulation with interfaces, quality and ratios.arXivpre...
Pith/arXiv arXiv 2026
-
[53]
Scalingproprioceptive-visuallearningwithheterogeneous pre-trained transformers
LiruiWang, XinleiChen, JialiangZhao, andKaimingHe. Scalingproprioceptive-visuallearningwithheterogeneous pre-trained transformers. InAdvancesinNeuralInformationProcessingSystems(NeurIPS), 2024
2024
-
[54]
Yifei Wei, Linqing Zhong, Yi Liu, Yuxiang Lu, Xindong He, Maoqing Yao, and Guanghui Ren. Libra-VLA: Achieving learning equilibrium via asynchronous coarse-to-fine dual-system.arXiv preprintarXiv:2604.24921, 2026
Pith/arXiv arXiv 2026
-
[55]
Edgar Welte, Yitian Shi, Rosa Wolf, Maximillian Gilles, and Rania Rayyes. FlowCorrect: Efficient interactive correction of generative flow policies for robotic manipulation.arXivpreprintarXiv:2602.22056, 2026
arXiv 2026
-
[56]
LongyanWu, JiejiRen, Chenghang Jiang, JunxiZhou, ShijiaPeng, RanHuang, Guoying Gu, Li Chen, andHongyang Li. TAMEn: Tactile-aware manipulation engine for closed-loop data collection in contact-rich tasks.arXiv preprint arXiv:2604.07335, 2026
Pith/arXiv arXiv 2026
-
[57]
Yifan Xie, YuAn Wang, Guangyu Chen, Jinkun Liu, Yu Sun, and Wenbo Ding. Learning human-intention priors from large-scale human demonstrations for robotic manipulation.arXivpreprintarXiv:2604.24681, 2026
Pith/arXiv arXiv 2026
-
[58]
XSkill: Cross embodiment skill discovery
Mengda Xu, Zhenjia Xu, Cheng Chi, Manuela Veloso, and Shuran Song. XSkill: Cross embodiment skill discovery. InProceedings of the 7th Conferenceon RobotLearning (CoRL), volume 229 ofProceedings of Machine Learning Research, pages 3536–3555. PMLR, 2023
2023
-
[59]
Yifu Xu, Bokai Lin, Xinyu Zhan, Hongjie Fang, Yong-Lu Li, Cewu Lu, and Lixin Yang. LIDEA: Human-to-robot imitation learning via implicit feature distillation and explicit geometry alignment.arXivpreprintarXiv:2604.10677, 2026
Pith/arXiv arXiv 2026
-
[60]
Bowen Yang, Zishuo Li, Yang Sun, Changtao Miao, Yifan Yang, Man Luo, Xiaotong Yan, Feng Jiang, Jinchuan Shi, Yankai Fu, Ning Chen, Junkai Zhao, Pengwei Wang, Guocai Yao, Shanghang Zhang, Hao Chen, Zhe Li, and Kai Zhu. AoE: Always-on egocentric human video collection for embodied AI.arXiv preprintarXiv:2602.23893, 2026
arXiv 2026
-
[61]
RuihanYang,QinxiYu,YechengWu,RuiYan,BoruiLi,An-ChiehCheng,XueyanZou,YunhaoFang,XuxinCheng,Ri- Zhao Qiu, Hongxu Yin, Sifei Liu, Song Han, Yao Lu, and Xiaolong Wang. EgoVLA: Learning vision-language-action models from egocentric human videos.arXivpreprint arXiv:2507.12440, 2025
Pith/arXiv arXiv 2025
-
[62]
Anlan Yu, Zaishu Chen, Peili Song, Zhiqing Hong, Haotian Wang, Desheng Zhang, Tian He, Yi Ding, and Daqing Zhang. WM-DAgger: Enabling efficient data aggregation for imitation learning with world models.arXivpreprint arXiv:2604.11351, 2026
Pith/arXiv arXiv 2026
-
[63]
Chao Zhang, Jun Wang, Zixuan Gao, Yixuan Su, Tao Dai, Chao Zhou, Jiajun Lu, and Yansong Tang. CLAP: Contrastive latent action pretraining for learning vision-language-action models from human videos.arXivpreprint arXiv:2601.04061, 2026
arXiv 2026
-
[64]
Gu Zhang, Qicheng Xu, Haozhe Zhang, Jianhan Ma, Long He, Yiming Bao, Zeyu Ping, Zhecheng Yuan, Chenhao Lu, Chengbo Yuan, Tianhai Liang, Xiaoyu Tian, Maanping Shao, Feihong Zhang, Mingyu Ding, Yang Gao, Hao Zhao, Hang Zhao, and Huazhe Xu. UniDex: A robot foundation suite for universal dexterous hand control from egocentric human videos.arXiv preprintarXiv:...
arXiv 2026
-
[65]
HaWoR: World-space hand motion reconstruction from egocentric videos
Jinglei Zhang, Jiankang Deng, Chao Ma, and Rolandos Alexandros Potamias. HaWoR: World-space hand motion reconstruction from egocentric videos. InProceedings of the IEEE/CVF Conference on Computer Vision and PatternRecognition(CVPR), pages 1805–1815, 2025. 14
2025
-
[66]
Tao Zhang, Song Xia, Ye Wang, and Qin Jin. EasyMimic: A low-cost framework for robot imitation learning from human videos.arXiv preprintarXiv:2602.11464, 2026
arXiv 2026
-
[67]
GehanZheng,SanjaySeenivasan,MatthewJohnson-Roberson,andWeimingZhi. Rewind-IL:Onlinefailuredetection and state respawning for imitation learning.arXivpreprintarXiv:2604.16683, 2026
Pith/arXiv arXiv 2026
-
[68]
Ruijie Zheng, Dantong Niu, Yuqi Xie, Jing Wang, Mengda Xu, Yunfan Jiang, Fernando Castañeda, Fengyuan Hu, You Liang Tan, Letian Fu, Trevor Darrell, Furong Huang, Yuke Zhu, Danfei Xu, and Linxi Fan. EgoScale: Scaling dexterous manipulation with diverse egocentric human data.arXivpreprintarXiv:2602.16710, 2026
arXiv 2026
-
[69]
Yiming Zhong, Yaoyu He, Zemin Yang, Pengfei Tian, Yifan Huang, Qingqiu Huang, Xinge Zhu, and Yuexin Ma. From noise to intent: Anchoring generative VLA policies with residual bridges.arXiv preprint arXiv:2604.21391, 2026
Pith/arXiv arXiv 2026
-
[70]
Zhu, Pranav Kuppili, Ryan Punamiya, Patcharapong Aphiwetsa, Dhruv Patel, Simar Kareer, Sehoon Ha, and Danfei Xu
Lawrence Y. Zhu, Pranav Kuppili, Ryan Punamiya, Patcharapong Aphiwetsa, Dhruv Patel, Simar Kareer, Sehoon Ha, and Danfei Xu. EMMA: Scaling mobile manipulation via egocentric human data.IEEERoboticsandAutomation Letters, 2025
2025
-
[71]
Yifei Zou, Chenguang Shi, Wenhao Yu, Haoran Xue, Jiaqi Lv, Yuzhe Pan, Cheng Wen, and Cewu Lu. ActiveGlasses: Learning manipulation with active vision from ego-centric human demonstration.arXivpreprintarXiv:2604.08534, 2026. 15 A Data Sources and Budgets All episode counts in the experiments are per task and source specific. They indicate how many episodes...
Pith/arXiv arXiv 2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.