Pith. sign in

REVIEW 4 major objections 6 minor 53 references

SiMDex: Mining Similar Egocentric Videos for Cross-Embodiment Dexterous Manipulation

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Selecting similar human video clips, not simply more of them, is what improves dexterous robot policies trained from egocentric data.

desk verdict Solid empirical data-mining paper with a clean controlled comparison; the headline gain is believable, but the low-data '4x reduction' needs an accounting fix and the eval is small. read the letter →

arxiv 2608.04196 v1 pith:HT27CRUL submitted 2026-08-04 cs.RO cs.CVcs.LG

Nie Lin , Takehiko Ohkawa , Sijin Chen , Ruoshi Wen , Zhuohang Li , Liqun Huang , Zhengming Zhu , Yiming Bao
show 5 more authors
Yunfei Li Minjie Cai Xiao Ma Wei Xu Yoichi Sato
This is my paper · ORCID
classification cs.ROcs.CVcs.LG
keywords dexterousmanipulationegocentricvideodataminingcross-embodimentlearningvision-language-actionmodelselectionsimilarityretrievalrobotfromhuman
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SiMDex argues that the value of human egocentric video for dexterous robot policies lies in which clips are selected, not how many. It frames human-data selection for vision-language-action (VLA) post-training as a retrieval problem and mines, for each robot demonstration, the most similar samples from a pool of roughly 32 million egocentric human clips using a recall-ranking-re-ranking cascade. With under 5% of the pool, about 1.49 million samples, the mined data raises overall success across three real-world dexterous tasks from 47.7% to 61.1% compared with the same model trained on an equal amount of randomly sampled human data. The benefit is largest when robot demonstrations are scarce, suggesting selective human-data mining can substitute for costly teleoperated data collection.

What carries the argument

The load-bearing mechanism is a three-stage recall-ranking-re-ranking pipeline operating on a shared, morphology-agnostic action space $a_t=(d^L_t,d^R_t,p^L_t,p^R_t)\in\mathbb{R}^{42}$, where $p$ are wrist-local fingertip positions and $d$ are wrist-frame deltas; this space abstracts away kinematic differences between human and robot hands. Stage I recalls broad candidates by fingertip pose and language embedding, Stage II ranks them by fine-grained motion similarity using wrist translation and rotation waveforms plus finger and wrist trajectories, and Stage III re-ranks by optical flow as an embodiment-agnostic visual check. The same 42 shared dimensions supervise the human samples during training through a masked flow-matching loss, so a single VLA can be trained on human and robot data without architectural modification.

What would settle it

Take a target skill that is absent from the human pool but has many kinematic impostors, such as a drill task where retrieved clips show hands posing near the tool with almost no motion, and train the identical VLA on the mined subset versus random data. If the mined subset still outperforms random sampling, the similarity signal is not the source of the gain; if it falls to or below random performance, the paper's stated coverage-dependence is confirmed.

Watch

Extended reading notes

Core claim

The paper's central claim is that task-relevant mining of a large egocentric human pool improves dexterous VLA post-training without any change to the architecture or training objective. Using the same base model and the same number of human samples, replacing random sampling with SiMDex's mined subset improves the overall success rate from 47.7% to 61.1%; the largest single-task gain is on fine fingertip skills, which rise from 24.5% to 45.5%. In the data-scaling ablation, roughly six hours of robot demonstrations plus the mined human data match the success of a baseline trained on roughly twenty-five hours of robot data, a fourfold reduction in robot-data collection. The authors also report that the advantage depends on coverage: when the pool lacks quality demonstrations of a skill, mined data can add variance instead of signal.

Load-bearing premise

The argument rests on the assumption that the shared 42-dimensional action space of wrist-local fingertip positions and wrist deltas preserves enough task-relevant information that clips similar in this space are genuinely useful for training the robot policy; the paper itself notes this space ignores contact forces, object state, and interaction semantics.

Editorial extensions

If this is right

  • Selective curation of human video can outperform indiscriminate mixing of an equal number of samples, so data volume is not the only lever for VLA post-training.
  • The same large egocentric corpus used for pre-training can be re-mined task-aware during post-training, making large-scale collection useful twice.
  • Because the method only changes the data source, its gains transfer to any VLA backbone that can express the shared 42-dimensional action space.
  • When robot demonstrations are limited, mined human data holds success roughly constant as robot data shrinks, pointing to a practical way to cut teleoperation cost.
  • The Drill result implies mining helps only when the human pool contains demonstrations of the target skill, so pool coverage is a precondition for the benefit.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's scaling results imply a closed loop the authors leave implicit: a policy that monitors its own failures could query the egocentric pool again during training, concentrating retrieval on skills it has not yet mastered.
  • Adding contact forces, object state, or interaction semantics to the similarity metric would likely sharpen retrieval where fingertip kinematics alone are ambiguous, such as drilling clips with little hand motion.
  • The same recall-ranking-re-ranking recipe should transfer to other cross-embodiment control settings, such as gripper manipulation or whole-body control, whenever a shared action representation can be defined.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. SiMDex proposes a similarity-based data mining framework that selects task-relevant human egocentric video samples for VLA post-training in dexterous manipulation, without changing the policy architecture or training objective. The method maps robot and human demonstrations into a shared 42-dimensional morphology-agnostic action space, then retrieves a small subset (~1.49M samples, under 5% of a ~32M pool) through a three-stage recall–ranking–re-ranking pipeline. The paper reports a controlled comparison against an equal-sized random sample of human data on three real-world bimanual dexterous tasks, claiming an overall success-rate improvement from 47.7% to 61.1%, and a low-robot-data result in which ~6h of robot demonstrations plus mined human data matches a baseline trained on ~25h, a claimed 4x reduction in robot-data collection.

Significance. If the claims hold, SiMDex is a practical and well-scoped contribution: it reuses the same egocentric corpus used for pre-training, requires no changes to the VLA architecture or training loop, and its headline comparison is cleanly controlled (same model, same data budget, only the selection mechanism differs). The scale of the retrieval pool and the real-world evaluation are strengths. The paper is also transparent about key limitations, especially pool coverage and the purely kinematic similarity space, which bound the generality of the results. The main risk is not circularity but the accounting and statistical robustness of the scaling claim, which is the most practically important result.

major comments (4)
  1. [§4.4, Fig. 3] The practical claim that '~6h of robot demonstrations (0.5x) match a baseline trained on ~25h (2x)—a 4x reduction in robot-data collection' depends on what robot data was used to mine the human subset. The text says the mined human data is kept fixed while robot data is scaled, but it does not state whether the anchors used for retrieval were subsampled to the same robot-data budget. If the 0.5x run used the full ~12.4h (or 2x) anchor set for mining and only restricted the training stage to 6h, then the curation stage consumed robot data outside the declared budget and the 4x reduction overstates the benefit. Please report the anchor set size at each robot-data scale, and if necessary rerun the low-data experiments with anchors restricted to the same budget.
  2. [§4.3, Table 1] The headline overall success rate (47.7% → 61.1%) is reported without an error bar or significance test, and several per-task differences are within one standard deviation of the reported trial-to-trial variation (e.g., Drill Total 1.93±0.00 vs. 1.63±0.71; Flick Wheel Total 0.73±0.33 vs. 1.37±0.19). Because the central claim is an empirical comparison over 10 trials per task, please provide confidence intervals for the overall rate and per-task totals, and a paired or blocked analysis across trial rounds.
  3. [§4.4, Fig. 3] The scaling ablation is presented without error bars and the caption states that all results use a single round. The 'stable performance floor' of ~57–58% and the largest gain (+17.2 at 0.5x) are therefore consistent with noise, yet this is the plot that supports the paper's most practically important claim. Please add multiple rounds or random seeds and report error bars or confidence intervals for the scaling curves, and make explicit how the single-round 1x column relates to the two-round Table 1 result.
  4. [§4.2, §4.3] The random baseline is a single random sample from the human pool. Since the paper's thesis is that selective curation outperforms indiscriminate mixing, the comparison should show that the 13.4-point gap is not an artifact of one unlucky random draw. Please report the variance of the random baseline over multiple random subsets, or otherwise bound the sampling noise, to support the causal interpretation of the main result.
minor comments (6)
  1. [§3.2] Please define the normalization used in the rank fusion (e.g., min-max scaling, Borda count, or reciprocal rank) and give the candidate counts at each stage; the current 'sums them' description is ambiguous when the four component ranks have different scales.
  2. [§4.2, Fig. 3] When the robot data is scaled, please specify whether the 1:1 robot/human mixture is maintained by subsampling robot demonstrations, and whether the random baseline uses the same number of human samples at every robot-data scale.
  3. [Table 1] Several entries report zero standard deviation (e.g., Drill GR-Dexter Total 1.93±.00, and all Drill sub-task stds for GR-Dexter); please report unrounded values or clarify how the displayed means and standard deviations were computed, since exact zero variance across 10 trials is surprising.
  4. [Fig. 3] Add numeric labels or a supporting table for the scaling curves; the text quotes specific gains, but the reader must read them off the bars, and the figure is hard to verify without error bars.
  5. [Fig. 2] The caption refers to 'Red annotations,' which may not be distinguishable in grayscale print; consider using symbols or hatching instead.
  6. [§6] The limitation that the kinematic action space ignores contact forces and object state is important; please consider adding a quantitative diagnostic for how often the ranking stage mismatches (e.g., the fraction of retrieved clips rejected by the optical-flow re-ranking) to make the cost of this approximation concrete.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: the central success-rate claim is an empirical comparison against an equal-size random baseline, and the shared action space is the intended mechanism, not a definitional identity.

full rationale

SiMDex's headline result is not derived from its inputs by construction; it is measured on real-robot trials. The morphology-agnostic action space a_t in Sec. 3.1 is used both for mining and for cross-embodiment supervision, but similarity in R42 does not by itself entail task success, and the random baseline uses the same VLA, the same 1:1 data mixture, and an equal number of human samples, so the only variable is how the human data is selected. No parameter is fitted to the reported success rates and then renamed a prediction; the comparisons in Table 1 and Fig. 3 are direct measurements. The scaling experiment in Sec. 4.4 raises a legitimate accounting question about how many robot anchors were used to produce the fixed mined subset in each robot-data budget, since the paper does not state this, but that is a potential data-budget confound rather than a circular step. Self-citations (SiMHand [27], GR-Dexter [47]) are contextual or define the baseline model; none is load-bearing, and no uniqueness theorem is imported from the authors' prior work. The Limitations section explicitly concedes that the kinematic action space ignores contact forces, object state, and interaction semantics, and that mining quality depends on pool coverage; these weaken generality but do not indicate that any claimed result reduces to its own inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities. It relies on several domain assumptions about representation sufficiency and pool coverage, and it selects a mined subset via pipeline thresholds that are not fully reported. The central claim is empirical, so the ledger is lighter than for a derivation-based paper.

free parameters (3)
  • Mined subset size = ~1.49M samples
    The final number of samples fed to training is set by deduplication and ranking thresholds; it is not reported as a tuned quantity but directly affects the comparison.
  • Stage I recall candidate count
    The number of candidates retrieved per anchor before ranking is not specified, yet it determines whether the later stages have enough relevant samples to rank.
  • Rank fusion weights
    The per-component ranks (wrist translation, wrist rotation, finger trajectory, wrist trajectory) are summed with implicit weights that are not reported or justified.
assumptions (4)
  • domain assumption The wrist-local fingertip and wrist-delta representation (Section 3.1) is sufficient for mining and cross-embodiment supervision of dexterous manipulation.
    The entire method depends on the idea that this kinematic abstraction removes the embodiment gap while preserving task-relevant contact structure.
  • domain assumption The EgoDex pool contains enough high-quality task-relevant demonstrations for each target task.
    The paper's own Drill result and Limitations section acknowledge that when relevant clips are rare, mined data can fail to help or even hurt.
  • domain assumption Optical flow similarity is an embodiment-agnostic verification that improves on kinematic similarity.
    Stage III uses aggregated dense flow as a final re-ranking signal without evidence that it is better than the kinematic ranks in general.
  • domain assumption Pre-training on the same egocentric corpus provides a base that post-training on a mined subset can improve.
    The paper re-mines the corpus used for pre-training, but does not ablate whether pre-training is required for the observed benefit.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SiMDex: Mining Similar Egocentric Videos for Cross-Embodiment Dexterous Manipulation." pith.science (2026). https://pith.science/paper/HT27CRUL

@misc{pith2026260804196,
  author       = {Pith},
  title        = {Pith review of: SiMDex: Mining Similar Egocentric Videos for Cross-Embodiment Dexterous Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HT27CRUL}},
  note         = {Machine review of arXiv:2608.04196}
}
read the original abstract

Recent years have witnessed an explosive trend of scaling ego-centric human videos for robot manipulation, yet it remains unclear which data actually benefits dexterous manipulation. We present SiMDex, a similarity-based data mining framework that casts human data selection for VLA post-training in dexterous manipulation as a recommendation problem. For each robot demonstration, SiMDex employs a three-layer recall-ranking-re-ranking pipeline to extract task-relevant subsets from a pool of ~32M egocentric human samples, operating in a morphology-agnostic action space that requires no changes to VLA architecture or training. Against a strong baseline trained with an equal amount of randomly sampled human data, SiMDex uses only ~1.49M mined samples (<5% of the pool) yet improves the overall success rate from 47.7% to 61.1%, showing that selective curation outperforms indiscriminate data mixing.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 27 canonical work pages

  1. [1]

    Amro Abbas, Kushal Tirumala, Dániel Simig, Surya Ganguli, and Ari S. Morcos. SemDeDup: Data-efficient learning at web-scale through semantic deduplication.arXiv preprint arXiv:2303.09540, 2023

    arXiv 2023

  2. [2]

    CUPID: Curating data your robot loves with influence functions

    Christopher Agia, Rohan Sinha, Jingyun Yang, Rika Antonova, Marco Pavone, Haruki Nishimura, Masha Itkina, and Jeannette Bohg. CUPID: Curating data your robot loves with influence functions. InConference on Robot Learning (CoRL), 2025

    work page 2025

  3. [3]

    Human-to-robot imitation in the wild

    Shikhar Bahl, Abhinav Gupta, and Deepak Pathak. Human-to-robot imitation in the wild. InProceedings of Robotics: Science and Systems (RSS), 2022

    work page 2022

  4. [4]

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, et al.π0: A vision-language- action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024

    arXiv 2024

  5. [5]

    RT-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, et al. RT-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    arXiv 2022

  6. [6]

    RT-2: Vision-language-action models transfer web knowledge to robotic control.arXiv preprint arXiv:2307.15818, 2023

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control.arXiv preprint arXiv:2307.15818, 2023

    arXiv 2023

  7. [7]

    GR-3 technical report.arXiv preprint arXiv:2507.15493, 2025

    Chilam Cheang, Sijin Chen, Zhongren Cui, Yingdong Hu, Liqun Huang, et al. GR-3 technical report.arXiv preprint arXiv:2507.15493, 2025

    arXiv 2025

  8. [8]

    Translation as a Bridging Action: Transferring Manipulation Skills from Humans to Robots

    Sijin Chen, Kaixuan Jiang, Haixin Shi, Yanhui Wang, Weiheng Zhong, et al. Translation as a bridging action: Transferring manipulation skills from humans to robots.arXiv preprint arXiv:2606.28133, 2026

    work page Pith review arXiv 2026

Show all 53 references
  1. [9]

    Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, 2024

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, 2024

    2024

  2. [10]

    Rescaling egocentric vision: Collection, pipeline and challenges for EPIC-KITCHENS-100.International Journal of Computer Vision, 130(1):33–55, 2022

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Antonino Furnari, Jian Ma, et al. Rescaling egocentric vision: Collection, pipeline and challenges for EPIC-KITCHENS-100.International Journal of Computer Vision, 130(1):33–55, 2022

    2022

  3. [11]

    HumanNet: Scaling human-centric video learning to one million hours.arXiv preprint arXiv:2605.06747, 2025

    Yufan Deng and Daquan Zhou. HumanNet: Scaling human-centric video learning to one million hours.arXiv preprint arXiv:2605.06747, 2025

    2025 arXiv

  4. [12]

    Behavior retrieval: Few-shot imitation learning by querying unlabeled datasets.arXiv preprint arXiv:2304.08742, 2023

    Maximilian Du, Suraj Nair, Dorsa Sadigh, and Chelsea Finn. Behavior retrieval: Few-shot imitation learning by querying unlabeled datasets.arXiv preprint arXiv:2304.08742, 2023

    2023 arXiv

  5. [13]

    DataComp: In search of the next generation of multimodal datasets

    Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, et al. DataComp: In search of the next generation of multimodal datasets. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  6. [14]

    Ego4D: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, et al. Ego4D: Around the world in 3,000 hours of egocentric video. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18995–19012, 2022

    2022

  7. [15]

    Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives

    Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Triantafyllos Afouras, Kumar Ashutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote, et al. Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives. InProceedings ...

    2024

  8. [16]

    Ego-Exo4D: Understanding skilled human activity from first- and third-person perspectives

    Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, et al. Ego-Exo4D: Understanding skilled human activity from first- and third-person perspectives. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19383–19400, 2024

    2024

  9. [17]

    DexPilot: Vision-based teleoperation of dexterous robotic hand-arm system

    Ankur Handa, Karl Van Wyk, Wei Yang, Jacky Liang, Yu-Wei Chao, et al. DexPilot: Vision-based teleoperation of dexterous robotic hand-arm system. InIEEE International Conference on Robotics and Automation (ICRA), pages 9164–9170, 2020

    2020

  10. [18]

    Re-mix: Optimizing data mixtures for large scale imitation learning

    Joey Hejna, Chethan Bhateja, Yichen Jia, and Dorsa Sadigh. Re-mix: Optimizing data mixtures for large scale imitation learning. arXiv preprint arXiv:2408.14575, 2024. 10

    2024 arXiv

  11. [19]

    Yoon, Mouli Sivapurapu, and Jian Zhang

    Ryan Hoque, Peide Huang, David J. Yoon, Mouli Sivapurapu, and Jian Zhang. EgoDex: Learning dexterous manipulation from large-scale egocentric video. InInternational Conference on Learning Representations (ICLR), 2026

    2026

  12. [20]

    EgoMimic: Scaling imitation learning via egocentric video.arXiv preprint arXiv:2410.24221, 2024

    Simar Kareer, Dhruv Patel, Ryan Punamiya, Pranay Mathur, Shuo Cheng, Chen Wang, Judy Hoffman, and Danfei Xu. EgoMimic: Scaling imitation learning via egocentric video.arXiv preprint arXiv:2410.24221, 2024

    2024 arXiv

  13. [21]

    DROID: A large-scale in-the-wild robot manipulation dataset

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, et al. DROID: A large-scale in-the-wild robot manipulation dataset. InProceedings of Robotics: Science and Systems (RSS), 2024

    2024

  14. [22]

    OpenVLA: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, et al. OpenVLA: An open-source vision-language-action model. InConference on Robot Learning (CoRL), pages 2679–2713, 2024

    2024

  15. [23]

    Masquerade: Learning from in-the-wild human videos using data-editing

    Marion Lepert, Jiaying Fang, and Jeannette Bohg. Masquerade: Learning from in-the-wild human videos using data-editing. arXiv preprint arXiv:2508.09976, 2025

    2025 arXiv

  16. [24]

    CogACT: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation.arXiv preprint arXiv:2411.19650, 2024

    Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, et al. CogACT: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation.arXiv preprint arXiv:2411.19650, 2024

    2024 arXiv

  17. [25]

    Scalable vision-language-action model pretraining for robotic manipulation with real-life human activity videos

    Qixiu Li, Yu Deng, Yaobo Liang, Lin Luo, Lei Zhou, Chengtang Yao, Lingqi Zeng, Zhiyuan Feng, Huizhi Liang, Sicheng Xu, et al. Scalable vision-language-action model pretraining for robotic manipulation with real-life human activity videos. arXiv preprint arXiv:2510.21571, 2025

  18. [26]

    Egolive: A large-scale egocentric dataset from real-world human tasks.arXiv preprint arXiv:2604.23570, 2026

    Yihang Li, Xuelong Wei, Jingzhou Luo, Yingjing Xiao, Yibo Bai, Guangyuan Zhou, Teng Zou, Chenguang Gui, Jiajun Wen, He Zhang, et al. Egolive: A large-scale egocentric dataset from real-world human tasks.arXiv preprint arXiv:2604.23570, 2026

    2026 arXiv

  19. [27]

    SiMHand: Mining similar hands for large-scale 3d hand pose pre-training

    Nie Lin, Takehiko Ohkawa, Yifei Huang, Mingfang Zhang, Minjie Cai, Ming Li, Ryosuke Furuta, and Yoichi Sato. SiMHand: Mining similar hands for large-scale 3d hand pose pre-training. InInternational Conference on Learning Representations (ICLR), 2025

    2025

  20. [28]

    Egobrain: Synergizing minds and eyes for human action understanding

    Nie Lin, Yansen Wang, Dongqi Han, Wei-Bang Jiang, Jingyuan Li, Ryosuke Furuta, Yoichi Sato, and Dongsheng Li. Egobrain: Synergizing minds and eyes for human action understanding. InInternational Conference on Learning Representations (ICLR), 2026

    2026

  21. [29]

    Being-H0: Vision-language-action pretraining from large-scale human videos.arXiv preprint arXiv:2507.15597, 2025

    Hao Luo, Yicheng Feng, Wanpeng Zhang, Sipeng Zheng, Ye Wang, et al. Being-H0: Vision-language-action pretraining from large-scale human videos.arXiv preprint arXiv:2507.15597, 2025

    2025 arXiv

  22. [30]

    VIP: Towards universal visual reward and representation via value-implicit pre-training

    Yecheng Jason Ma, Shagun Sodhani, Dinesh Jayaraman, Osbert Bastani, Vikash Kumar, and Amy Zhang. VIP: Towards universal visual reward and representation via value-implicit pre-training. InInternational Conference on Learning Representations (ICLR), 2023

    2023

  23. [31]

    Where are we in the search for an artificial visual cortex for embodied intelligence? InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    Arjun Majumdar, Karmesh Yadav, Sergio Arnaud, Yecheng Jason Ma, Claire Chen, et al. Where are we in the search for an artificial visual cortex for embodied intelligence? InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  24. [32]

    Deximit: Learning bimanual dexterous manipulation from monocular human videos.arXiv preprint arXiv:2602.10105, 2026

    Juncheng Mu, Sizhe Yang, Yiming Bao, Hojin Bae, Tianming Wei, Linning Xu, Boyi Li, Huazhe Xu, and Jiangmiao Pang. Deximit: Learning bimanual dexterous manipulation from monocular human videos.arXiv preprint arXiv:2602.10105, 2026

    2026

  25. [33]

    R3M: A universal visual representation for robot manipulation

    Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3M: A universal visual representation for robot manipulation. InConference on Robot Learning (CoRL), 2022

    2022

  26. [34]

    Octo: An open-source generalist robot policy

    Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, et al. Octo: An open-source generalist robot policy. InProceedings of Robotics: Science and Systems (RSS), 2024

    2024

  27. [35]

    Yubi: Yielding universal bidigital interface for bimanual dexterous manipulation at scale.arXiv preprint arXiv:2606.10244, 2026

    Takehiko Ohkawa, Jumpei Arima, Yuki Noguchi, Masatoshi Tateno, Makoto Sugiura, et al. Yubi: Yielding universal bidigital interface for bimanual dexterous manipulation at scale.arXiv preprint arXiv:2606.10244, 2026

    2026 arXiv

  28. [36]

    Open X-Embodiment: Robotic learning datasets and RT-X models

    Open X-Embodiment Collaboration. Open X-Embodiment: Robotic learning datasets and RT-X models. In IEEE International Conference on Robotics and Automation (ICRA), 2024

    2024

  29. [37]

    R+x: Retrieval and execution from everyday human videos

    Georgios Papagiannis, Norman Di Palo, Pietro Vitiello, and Edward Johns. R+x: Retrieval and execution from everyday human videos. InIEEE International Conference on Robotics and Automation (ICRA), 2025

    2025

  30. [38]

    FineWeb: Decanting the web for the finest text data at scale.arXiv preprint arXiv:2406.17557, 2024

    Guilherme Penedo, Hynek Kydlíček, Loubna Ben Allal, Anton Lozhkov, et al. FineWeb: Decanting the web for the finest text data at scale.arXiv preprint arXiv:2406.17557, 2024. 11

    2024 arXiv

  31. [39]

    Physical Intelligence, Ali Amin, Raichelle Aniceto, Ashwin Balakrishna, Kevin Black, et al.π∗ 0.6: A VLA that learns from experience.arXiv preprint arXiv:2511.14759, 2025

    2025 arXiv

  32. [40]

    InConference on Robot Learning (CoRL), pages 17–40, 2025

    Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, et al.π0.5: A vision-language-action model with open-world generalization. InConference on Robot Learning (CoRL), pages 17–40, 2025

    2025

  33. [41]

    Egobridge: Domain adaptation for generalizable imitation from egocentric human data

    Ryan Punamiya, Dhruv Patel, Patcharapong Aphiwetsa, Pranav Kuppili, Lawrence Y Zhu, Simar Kareer, Judy Hoffman, and Danfei Xu. Egobridge: Domain adaptation for generalizable imitation from egocentric human data. In Human to Robot: Workshop on Sensorizing, Modeling, and Learnin...

    2025

  34. [42]

    Egoverse: An egocentric human dataset for robot learning from around the world

    Ryan Punamiya, Simar Kareer, Zeyi Liu, Josh Citron, Ri-Zhao Qiu, Xiongyi Cai, Alexey Gavryushin, Jiaqi Chen, Davide Liconti, Lawrence Y Zhu, et al. Egoverse: An egocentric human dataset for robot learning from around the world. arXiv preprint arXiv:2604.07607, 2026

    2026 arXiv

  35. [43]

    Real-world robot learning with masked visual pre-training

    Ilija Radosavovic, Tete Xiao, Stephen James, Pieter Abbeel, Jitendra Malik, and Trevor Darrell. Real-world robot learning with masked visual pre-training. InConference on Robot Learning (CoRL), pages 416–426, 2023

    2023

  36. [44]

    Kushal Tirumala, Daniel Simig, Armen Aghajanyan, and Ari S. Morcos. D4: Improving LLM pretraining via document de-duplication and diversification.arXiv preprint arXiv:2308.12284, 2023

    2023 arXiv

  37. [45]

    MimicPlay: Long-horizon imitation learning by watching human play

    Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu, and Anima Anandkumar. MimicPlay: Long-horizon imitation learning by watching human play. InConference on Robot Learning (CoRL), 2024

    2024

  38. [46]

    Karen Liu

    Chen Wang, Haochen Shi, Weizhuo Wang, Ruohan Zhang, Li Fei-Fei, and C. Karen Liu. DexCap: Scalable and portable mocap data collection system for dexterous manipulation. InProceedings of Robotics: Science and Systems (RSS), 2024

    2024

  39. [47]

    GR-Dexter technical report.arXiv preprint arXiv:2512.24210, 2025

    Ruoshi Wen, Guangzeng Chen, Zhongren Cui, Min Du, Yang Gou, et al. GR-Dexter technical report.arXiv preprint arXiv:2512.24210, 2025

    2025

  40. [48]

    Dexterous teleoperation of 20-dof ByteDexter hand via human motion retargeting.arXiv preprint arXiv:2507.03227, 2025

    Ruoshi Wen, Jiajun Zhang, Guangzeng Chen, Zhongren Cui, Min Du, et al. Dexterous teleoperation of 20-dof ByteDexter hand via human motion retargeting.arXiv preprint arXiv:2507.03227, 2025

    2025 arXiv

  41. [49]

    Egovla: Learning vision-language-action models from egocentric human videos.arXiv preprint arXiv:2507.12440, 2025

    Ruihan Yang, Qinxi Yu, Yecheng Wu, Rui Yan, Borui Li, An-Chieh Cheng, Xueyan Zou, Yunhao Fang, Xuxin Cheng, Ri-Zhao Qiu, et al. Egovla: Learning vision-language-action models from egocentric human videos.arXiv preprint arXiv:2507.12440, 2025

    2025 arXiv

  42. [50]

    Being-0: A humanoid robotic agent with vision-language models and modular skills.arXiv preprint arXiv:2503.12533, 2025

    Haoqi Yuan, Yu Bai, Yuhui Fu, Bohan Zhou, Yicheng Feng, et al. Being-0: A humanoid robotic agent with vision-language models and modular skills.arXiv preprint arXiv:2503.12533, 2025

    2025 arXiv

  43. [51]

    EgoScale: Scaling dexterous manipulation with diverse egocentric human data

    Ruijie Zheng, Dantong Niu, Yuqi Xie, Jing Wang, Mengda Xu, Yunfan Jiang, Fernando Castañeda, Fengyuan Hu, You Liang Tan, Letian Fu, et al. EgoScale: Scaling dexterous manipulation with diverse egocentric human data. arXiv preprint arXiv:2602.16710, 2026

    2026

  44. [52]

    DexGraspVLA: A vision-language- action framework towards general dexterous grasping

    Yifan Zhong, Xuchuan Huang, Ruochong Li, Ceyao Zhang, Zhang Chen, et al. DexGraspVLA: A vision-language- action framework towards general dexterous grasping. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 18836–18844, 2026

    2026

  45. [53]

    Let me show you: Learning by retrieving from egocentric video for robotic manipulation

    Yichen Zhu and Feifei Feng. Let me show you: Learning by retrieving from egocentric video for robotic manipulation. arXiv preprint arXiv:2511.05199, 2025. 12

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.