Pith. sign in

REVIEW 3 major objections 6 minor 5 cited by

ImMimic: Cross-Domain Imitation from Human Videos via Mapping and Interpolation

T0 review · 3 major / 6 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ImMimic shows that retargeted human hand trajectories, time-aligned and blended with robot demonstrations, train robot policies that outperform robot-only and co-training baselines on success rate and smoothness across four embodiments and

desk verdict Solid integration of known components with a useful new comparison (action vs visual mapping); undercut by thin statistics and an unvalidated retargeting assumption. read the letter →

arxiv 2509.10952 v1 pith:A7DQDGOH submitted 2025-09-13 cs.RO

Yangcen Liu , Woo Chul Shin , Yunhai Han , Zhenyang Chen , Harish Ravichandar , Danfei Xu This is my paper
classification cs.RO
keywords imitationlearninghumanvideosrobotmanipulationdomainadaptationhandposeretargetingdiffusionpolicydynamictimewarpingmixupinterpolation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ImMimic claims that robot policies can learn from abundant human videos plus a small set of robot demonstrations by retargeting human hand poses into the robot's action space, time-aligning human and robot trajectories with Dynamic Time Warping, and blending the paired samples with interpolation to create intermediate training examples. The paper reports that this approach beats robot-only, two-stage fine-tuning, vanilla co-training, and retrieval baselines across Pick and Place, Push, Hammer, and Flip on four end-effectors. If correct, it offers a practical recipe for turning cheap human video into usable robot supervision without simulators or heavy visual preprocessing, and it suggests that action information from human hands is more valuable than visual similarity for cross-embodiment transfer.

What carries the argument

Mapping-guided MixUp: Dynamic Time Warping aligns each human timestep with the best-matching robot timestep using either an action-distance or visual-feature distance; then a linearly decaying mixing coefficient blends the condition inputs and future-action sequences of the paired human and robot samples. This creates a continuum of intermediate human–robot samples that gradually shifts the policy from the human domain to the robot domain during co-training, while a diffusion policy serves as the shared action decoder for both data sources.

What would settle it

Run ImMimic-A on a contact-rich task such as peg insertion and compare against a co-training baseline without human data: if success does not improve, the retargeting's neglect of contact forces is the bottleneck. Alternatively, replace the human demonstrations with randomly generated trajectories matched only in statistics and check whether performance is retained; if it is, the benefit comes from interpolation alone rather than from human action content.

Watch

Extended reading notes

Core claim

The central discovery is that interpolating between DTW-paired human and robot demonstrations—in both the observation-condition space and the predicted-action space—creates a continuous domain bridge that lets a single diffusion policy learn from both sources. The paper reports that this 'mapping-guided MixUp' improves task success and rollout smoothness over baselines on all tested embodiments and tasks, with action-based DTW mapping consistently outperforming visual-based mapping. It also finds that retargeted human hand trajectories are informative enough to serve as action labels, and that the benefit grows as the average action distance between retargeted human and robot trajectories sh

Load-bearing premise

The load-bearing premise is that retargeted human hand positions, computed from monocular video without modeling contact, forces, or object interaction, are trustworthy action labels for real robot motion; if these retargeted actions are dynamically inconsistent with what the robot can execute, the DTW pairing and interpolation are built on systematically biased supervision.

Editorial extensions

If this is right

  • If ImMimic is correct, 100 human videos plus 5 robot demonstrations can match or exceed the success rate of 20 robot demonstrations alone, sharply reducing the cost of collecting robot-specific data.
  • Action-based DTW mapping being more reliable than visual-based mapping implies that retargeted action labels, not just pixel-level appearance, are a productive channel for cross-embodiment transfer.
  • The method is embodiment-agnostic: it improves results on both two-finger grippers and multi-fingered hands without embodiment-specific changes, widening the set of robots that can benefit from human video data.
  • The progressive interpolation schedule (linearly decreasing alpha) outperforming a beta-distributed alpha suggests that gradual, controlled domain adaptation is more effective than random mixing for human-to-robot transfer.
  • Human-video augmentation also improves rollout smoothness as measured by Spectral Arc Length, indicating the benefit extends beyond task completion to motion quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The DTW+MixUp recipe could generalize beyond the hand to whole-arm or bimanual manipulation if retargeting is extended, because the mapping-and-blending mechanism does not depend on a specific hand model.
  • The paper's observation that more human-like hands do not always transfer better suggests a testable design principle: end-effectors should be designed to minimize action distance to human demonstrations, not merely to resemble a human hand.
  • The reliance on retargeted actions as labels means the method inherits the accuracy of monocular hand-pose estimation; contact-rich tasks likely need force or contact modeling, which the current retargeting lacks.
  • The action-DTW mapping could be reused as a general retrieval tool for long, unsegmented human videos, as the appendix begins to explore, but full closed-loop adaptation during live teleoperation remains a separate challenge.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes ImMimic, a co-training framework for learning robot manipulation policies from human videos plus a small number of teleoperated robot demonstrations. Human hand keypoints are retargeted to robot joint commands via the constrained optimization in Eq. (1). Retargeted human trajectories are then aligned with robot trajectories using DTW, based on either action distance or visual features, and MixUp interpolation is applied to the mapped pairs in both observation and action space (Eq. 2). A diffusion policy is trained jointly on robot, human, and interpolated samples. Experiments across four end-effectors (Robotiq, Fin Ray, Allegro, Ability) and four tasks (Pick and Place, Push, Hammer, Flip) report success rates (Tables 1-2), SPARC smoothness (Table 3), sample-efficiency curves, and ablations of mapping variants. The authors claim that ImMimic improves task success rates and smoothness over robot-only, co-training, and retrieval baselines.

Significance. ImMimic is a conceptually clean and practical recipe: use retargeted human trajectories as action labels, then build an interpolation path between human and robot data with DTW-based mapping and MixUp. The real-robot evaluation across four embodiments is an asset, and the paper includes useful ablations (action- versus visual-based mapping, random mapping, fine-tuning, and STRAP retrieval) plus a GMS-SDTW extension for untrimmed videos. If the reported gains are reproducible, the method is likely to be useful to the learning-from-videos community. However, the evidence for the central claim is not yet strong enough: success rates are based on only 10 rollouts with no variance or multiple seeds, and the validity of the offline retargeted action labels—the linchpin of the method—is never directly established.

major comments (3)
  1. [§3.1, Eq. (1); App. B; Table A.1] The offline retargeting used for human videos is not validated as producing executable robot commands. Eq. (1) is a purely kinematic optimization with no contact, force, or object-interaction modeling; for two-finger grippers and contact-rich tasks (Hammer, Flip), many retargeted joint trajectories are likely infeasible. App. B explicitly distinguishes this offline RGB retargeting from the online depth-based vector retargeting used for teleoperation, so teleoperation success (Table A.1) does not validate the offline labels. Table A.1 itself shows low teleoperation success for Allegro and Ability on Hammer, suggesting that retargeting validity is embodiment- and task-dependent. The paper should directly test the offline retargeted trajectories (e.g., open-loop replay, or comparison against teleoperated execution) or provide an experiment showing that ImMimic is robust to substantially cor
  2. [Tables 1 and 2; Figs. 4-5] All success rates are computed from only 10 rollouts with no confidence intervals, variance, or multiple training seeds. Many cells are at floor or ceiling (e.g., Hammer-Allegro 0.00/0.20, Hammer-Ability 0.00 for all methods, several 1.00 cells), and the differences used to support the 'consistently higher' claim are often 0.1-0.2, which with n=10 is within sampling noise (e.g., Robotiq Push: Co-Training 0.20 vs ImMimic-A 0.40; Allegro Hammer 0.00 vs 0.20). Please report binomial confidence intervals or per-seed results, run multiple training seeds for the main comparisons, and state how rollouts were selected. This is load-bearing because the paper's headline claim is an empirical generalization across embodiments and tasks.
  3. [§3.2, 'Human Prediction Loss'] The human condition z_h includes the retargeted action sequence a^{h→r}_{t-τ:t} as pseudo-proprioception, while the prediction target is the future of the same retargeted action sequence. Because human trajectories are smooth, L_human can be minimized by copying or extrapolating the action condition, without using the visual features. At inference the robot condition contains proprioception instead, so this shortcut is not available during robot rollouts; the reported improvement may therefore partly reflect an action-copy artifact rather than genuine cross-domain transfer. Please ablate this design—for example, remove the action condition from z_h, or evaluate a human branch that does not condition on past retargeted actions—and report the effect on robot success rates. This directly affects the interpretation of the method's mechanism (interpolation versus action leakage).
minor comments (6)
  1. [§7, §4, §4 Hardware] Typos: 'Exisiting limitation' in Section 7; 'desinged' in Section 4; 'Fripper' in the hardware setup (should be 'Gripper').
  2. [Appendix A.2] The observation history length is called ϵ=2 in the appendix but τ in the main text (Sec. 3.2). Please unify the notation.
  3. [Table 3] SPARC is reported as an average over '5 successful rollouts', but some cells have fewer than 5 successes (e.g., Robotiq Robot Only 0.40 = 4/10). Clarify whether extra rollouts were collected or whether the table is based on a selected subset; also explain the N/A entry for Allegro Robot Only.
  4. [Sec. 5, first paragraph] The text says 'Tab. 2 ... across all tasks and embodiments', but Table 2 contains only Robotiq and Ability and only Pick and Place and Flip. Restrict the claim to Table 1 or add the missing baselines to all cells.
  5. [Eq. (B.1)] The Perspective-n-Point formulation cites [59], which appears to be a thermal hand benchmark and is unrelated. Please cite a standard PnP reference or remove.
  6. [Figs. 4-5] Please add error bars or state explicitly that these are single-run results; without variance, the sample-efficiency claims are difficult to evaluate.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are evaluated on physical robot rollouts, and no equation reduces to its own inputs.

full rationale

ImMimic's headline claim is empirical: policies trained with mapping-guided MixUp achieve higher real-world success rates and smoother rollouts than baselines across four embodiments and four tasks (Tables 1–3, Appendix E). The success rates are measured on physical robot executions with 10 rollouts per condition, not on fitted training quantities. The method's components—Eq. 1 retargeting, DTW mapping, Eq. 2 MixUp interpolation, and the diffusion-policy losses—form a training pipeline, not a closed derivation in which a predicted quantity is defined in terms of itself. The only self-citation of note is EgoMimic [4], which motivates the co-training design (Sec. 3.2), but co-training is a baseline the paper then improves upon empirically; the citation is not load-bearing for the claimed improvement. Concerns about the physical validity of offline-retargeted human action labels (App. B distinguishes offline RGB retargeting from online depth-based teleoperation retargeting) are a correctness/assumption risk, not circularity: even if the labels are biased, the evaluation remains an independent real-robot outcome. No 'uniqueness theorem' or prior-work ansatz is invoked to forbid alternatives, and no known result is merely renamed. Therefore the derivation chain is self-contained against external benchmarks, and no circular step can be exhibited from the paper's own equations.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on three domain assumptions: retargeted human actions are usable labels, DTW pairing finds shared states, and MixUp interpolation creates valid intermediate domains. Several hyperparameters are tuned or left unspecified, including retargeting weights, action-distance weights, the MixUp schedule, and retrieval thresholds. No new physical or latent entities are introduced; the intermediate domains are algorithmic constructs.

free parameters (6)
  • Retargeting weights alpha, beta
    Eq. (1): balance keypoint position matching and temporal smoothness in hand-to-robot retargeting. Values not reported in the paper.
  • Action distance weights lambda1, lambda2
    Sec. 3.3: weighting coefficients for hand pose and orientation in d_act. Values not reported.
  • MixUp coefficient alpha schedule = linearly decreasing, endpoints not reported
    Sec. 3.3 and Table E.2: progressive interpolation is claimed to outperform beta-sampled alpha; exact schedule parameters are absent.
  • Sample rate gamma per embodiment-task = 32, 100, 155, 185, 193, etc.
    Table A.2: computed from demonstration durations, not fitted to performance, but it determines temporal alignment and is a design choice.
  • SPARC parameters K, omega_max_c, S = K=4, omega_max_c=15, S=0.05
    Appendix E.6: metric cutoff settings.
  • GMS-SDTW threshold epsilon and window bounds
    Appendix C: distance threshold and window range for retrieval, values unspecified.
assumptions (5)
  • domain assumption Retargeted human hand trajectories provide informative action labels for robot execution.
    Key insight (1), Sec. 1 and 3.2; if retargeting errors are large, the human loss is trained on biased targets.
  • domain assumption DTW-mapped human and robot segments with similar action or visual patterns correspond to shared task states.
    Sec. 3.3, citing Dynamo [56]; load-bearing for the mapping quality.
  • domain assumption Linear interpolation in latent and action spaces (MixUp) creates valid intermediate domains between human and robot data.
    Sec. 3.3, Eq. (2), inspired by DLOW [16]; if latents are not linearly interpolable, augmented samples may be implausible.
  • domain assumption Uniform subsampling by gamma aligns human and robot demonstration speeds and temporal coverage.
    Appendix A.2; assumes human and robot perform the task with comparable temporal structure after rescaling.
  • domain assumption Success rate estimated from 10 rollouts without variance is a reliable comparison signal.
    Sec. 4 metrics, Tables 1-2; statistical noise is not quantified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ImMimic: Cross-Domain Imitation from Human Videos via Mapping and Interpolation." pith.science (2026). https://pith.science/paper/A7DQDGOH

@misc{pith2026250910952,
  author       = {Pith},
  title        = {Pith review of: ImMimic: Cross-Domain Imitation from Human Videos via Mapping and Interpolation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A7DQDGOH}},
  note         = {Machine review of arXiv:2509.10952}
}
read the original abstract

Learning robot manipulation from abundant human videos offers a scalable alternative to costly robot-specific data collection. However, domain gaps across visual, morphological, and physical aspects hinder direct imitation. To effectively bridge the domain gap, we propose ImMimic, an embodiment-agnostic co-training framework that leverages both human videos and a small amount of teleoperated robot demonstrations. ImMimic uses Dynamic Time Warping (DTW) with either action- or visual-based mapping to map retargeted human hand poses to robot joints, followed by MixUp interpolation between paired human and robot trajectories. Our key insights are (1) retargeted human hand trajectories provide informative action labels, and (2) interpolation over the mapped data creates intermediate domains that facilitate smooth domain adaptation during co-training. Evaluations on four real-world manipulation tasks (Pick and Place, Push, Hammer, Flip) across four robotic embodiments (Robotiq, Fin Ray, Allegro, Ability) show that ImMimic improves task success rates and execution smoothness, highlighting its efficacy to bridge the domain gap for robust robot manipulation. The project website can be found at https://sites.google.com/view/immimic.

Figures

Figures reproduced from arXiv: 2509.10952 by the authors.

Figure 1
Figure 1. ImMimic enables embodiment-agnostic co-training between human and robot demonstra￾tions. It leverages large-scale human videos and a small amount of teleoperated robot data, using a MixUp interpolation to enable smooth domain transfer. We validate ImMimic on four diverse ma￾nipulation tasks across four robotic embodiments. 1 Introduction Teaching robots to perform diverse manipulation tasks in real-world environment… view at source ↗
Figure 2
Figure 2. Overview of how we collect, map, and interpolate human and robot data. (a) Robot demon [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of our embodiment-agnostic co-training framework [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Sample efficiency of ImMimic-A and Robot-Only with varying numbers of robot demonstrations. trained with ImMimic-A consistently achieve higher success rates across all tasks and embodiments compared to robot-only, two-stage fine-tuning, and co-training baselines. These…
Figure 6
Figure 6. Figure 6: Comparison of Mean IoU across different disturbance settings. Interpolation with Action-based Mapping leads to better performance. We compare action-based and visual-based mapping to evaluate their effectiveness in bridging human-robot domain gap. As shown in Tab. 2, a…
Figure 7
Figure 7. Figure 7: Desired behavior and corresponding failure cases. (a) Unstable push due to thin tip. (b) [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data

    cs.RO 2026-07 accept novelty 6.5 of 10

    World Action Model co-training with DINO or 3D-flow targets scales human-to-robot transfer on bimanual tasks far better than behavior cloning, while pixel prediction transfers weakly.

  2. Static In, Dynamic Out: Counterfactual Action Augmentation for Moving Object Manipulation

    cs.RO 2026-07 conditional novelty 6.0 of 10

    SIDO morphs static demonstrations into counterfactual future-pose samples, training a goal-conditioned policy that, paired with a pose predictor, grasps objects whose motion was unseen during training.

  3. OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A policy-agnostic two-stage real-world RL method learns tactile residual corrections on frozen visual policies, lifting contact-rich task success from 5–40% to 85–100% in under 80 minutes.

  4. ObjRetarget: An Object-Aware Motion Retargeting Framework with Anthropomorphic Arm Constraints and Polyhedral Hand Modeling

    cs.RO 2026-07 conditional novelty 5.0 of 10

    Decoupled arm–hand retargeting with anthropomorphic arm-plane constraints and polyhedral contact invariants raises real-robot dexterous-task success to 75.8% versus 61.6% and 50.8% for OKAMI and ORION.

  5. Going with the Flow: Koopman Behavioral Models as Pseudo Planners for Visuo-Motor Dexterity

    cs.RO 2026-02 conditional novelty 5.0 of 10

    A single learned linear Koopman model over coupled visual and proprioceptive states generates full-horizon dexterous manipulation plans and triggers replanning when its own visual predictions diverge from reality.

Reference graph

Works this paper leans on

66 extracted references · 26 linked inside Pith · cited by 5 Pith papers

  1. [1]

    Srivastava, C

    S. Srivastava, C. Li, M. Lingelbach, R. Mart ´ın-Mart´ın, F. Xia, K. E. Vainio, Z. Lian, C. Gok- men, S. Buch, K. Liu, et al. Behavior: Benchmark for everyday household activities in virtual, interactive, and ecological environments. InConference on robot learning, pages 477–490. PMLR, 2022

    2022

  2. [2]

    C. Li, R. Zhang, J. Wong, C. Gokmen, S. Srivastava, R. Mart ´ın-Mart´ın, C. Wang, G. Levine, W. Ai, B. Martinez, et al. Behavior-1k: A human-centered, embodied ai benchmark with 1,000 everyday activities and realistic simulation.arXiv preprint arXiv:2403.09227, 2024

    arXiv 2024

  3. [3]

    C. Wang, L. Fan, J. Sun, R. Zhang, L. Fei-Fei, D. Xu, Y . Zhu, and A. Anandkumar. Mimicplay: Long-horizon imitation learning by watching human play.arXiv preprint arXiv:2302.12422, 2023

    arXiv 2023

  4. [4]

    Kareer, D

    S. Kareer, D. Patel, R. Punamiya, P. Mathur, S. Cheng, C. Wang, J. Hoffman, and D. Xu. Egomimic: Scaling imitation learning via egocentric video, 2024. URLhttps://arxiv. org/abs/2410.24221

    arXiv 2024

  5. [5]

    K. Shaw, S. Bahl, and D. Pathak. Videodex: Learning dexterity from internet videos. In Conference on Robot Learning, pages 654–665. PMLR, 2023

    2023

  6. [6]

    Y . Qin, W. Yang, B. Huang, K. Van Wyk, H. Su, X. Wang, Y .-W. Chao, and D. Fox. Anyteleop: A general vision-based dexterous robot arm-hand teleoperation system.arXiv preprint arXiv:2307.04577, 2023

    arXiv 2023

  7. [7]

    R. Ding, Y . Qin, J. Zhu, C. Jia, S. Yang, R. Yang, X. Qi, and X. Wang. Bunny- visionpro: Real-time bimanual dexterous teleoperation for imitation learning.arXiv preprint arXiv:2407.03162, 2024

    arXiv 2024

  8. [8]

    Sivakumar, K

    A. Sivakumar, K. Shaw, and D. Pathak. Robotic telekinesis: Learning a robotic hand imitator by watching humans on youtube.RSS, 2022

    2022

Show all 66 references
  1. [9]

    S. Bahl, A. Gupta, and D. Pathak. Human-to-robot imitation in the wild. 2022

    2022

  2. [10]

    Heppert, M

    N. Heppert, M. Argus, T. Welschehold, T. Brox, and A. Valada. Ditto: Demonstration imitation by trajectory transformation. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7565–7572. IEEE, 2024

    2024

  3. [11]

    M. Xu, Z. Xu, C. Chi, M. Veloso, and S. Song. Xskill: Cross embodiment skill discovery. In Conference on robot learning, pages 3536–3555. PMLR, 2023

    2023

  4. [12]

    M. Xu, Z. Xu, Y . Xu, C. Chi, G. Wetzstein, M. Veloso, and S. Song. Flow as the cross-domain manipulation interface.arXiv preprint arXiv:2407.15208, 2024

    2024 arXiv

  5. [13]

    L.-H. Lin, Y . Cui, A. Xie, T. Hua, and D. Sadigh. Flowretrieval: Flow-guided data retrieval for few-shot imitation learning.arXiv preprint arXiv:2408.16944, 2024

    2024 arXiv

  6. [14]

    Memmel, J

    M. Memmel, J. Berg, B. Chen, A. Gupta, and J. Francis. Strap: Robot sub-trajectory retrieval for augmented policy learning.arXiv preprint arXiv:2412.15182, 2024

    2024 arXiv

  7. [15]

    Zhang, M

    H. Zhang, M. Cisse, Y . N. Dauphin, and D. Lopez-Paz. mixup: Beyond empirical risk mini- mization, 2018. URLhttps://arxiv.org/abs/1710.09412

    2018 arXiv

  8. [16]

    R. Gong, W. Li, Y . Chen, and L. V . Gool. Dlow: Domain flow for adaptation and generalization,

  9. [17]

    Antunes, L

    R. Antunes, L. Lang, M. L. de Aguiar, T. Assis Dutra, and P. D. Gaspar. Design of fin ray effect soft robotic gripper for improved mechanical performance and adaptability: Numerical simulations and experimental validation. In2024 20th IEEE/ASME International Conference on Mech...

    2024

  10. [18]

    C. Chi, Z. Xu, C. Pan, E. Cousineau, B. Burchfiel, S. Feng, R. Tedrake, and S. Song. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots.arXiv preprint arXiv:2402.10329, 2024

    2024 arXiv

  11. [19]

    K. Liu, C. Guan, Z. Jia, Z. Wu, X. Liu, T. Wang, S. Liang, P. Chen, P. Zhang, H. Song, et al. Fastumi: A scalable and hardware-independent universal manipulation interface with dataset. arXiv e-prints, pages arXiv–2409, 2024

    2024

  12. [20]

    Papagiannis, N

    G. Papagiannis, N. D. Palo, P. Vitiello, and E. Johns. R+x: Retrieval and execution from everyday human videos, 2025. URLhttps://arxiv.org/abs/2407.12957

    2025 arXiv

  13. [21]

    Saxena, M

    V . Saxena, M. Bronars, N. R. Arachchige, K. Wang, W. C. Shin, S. Nasiriany, A. Mandlekar, and D. Xu. What matters in learning from large-scale datasets for robot manipulation. InThe Thirteenth International Conference on Learning Representations

  14. [22]

    Shang, K

    J. Shang, K. Schmeckpeper, B. B. May, M. V . Minniti, T. Kelestemur, D. Watkins, and L. Her- lant. Theia: Distilling diverse vision foundation models for robot learning, 2024. URL https://arxiv.org/abs/2407.20179

    2024 arXiv

  15. [23]

    S. Nair, A. Rajeswaran, V . Kumar, C. Finn, and A. Gupta. R3m: A universal visual represen- tation for robot manipulation, 2022. URLhttps://arxiv.org/abs/2203.12601

    2022 arXiv

  16. [24]

    Bahety, P

    A. Bahety, P. Mandikal, B. Abbatematteo, and R. Mart ´ın-Mart´ın. Screwmimic: Bimanual imitation from human videos with screw space projection, 2024. URLhttps://arxiv.org/ abs/2405.03666

    2024 arXiv

  17. [25]

    K. Yu, Y . Han, Q. Wang, V . Saxena, D. Xu, and Y . Zhao. Mimictouch: Leveraging multi- modal human tactile demonstrations for contact-rich manipulation, 2025. URLhttps:// arxiv.org/abs/2310.16917

    2025 arXiv

  18. [26]

    A. Iyer, Z. Peng, Y . Dai, I. Guzey, S. Haldar, S. Chintala, and L. Pinto. Open teach: A versatile teleoperation system for robotic manipulation, 2024. URLhttps://arxiv.org/abs/2403. 07870

    2024

  19. [27]

    K. Shaw, Y . Li, J. Yang, M. K. Srirama, R. Liu, H. Xiong, R. Mendonca, and D. Pathak. Bimanual dexterity for complex tasks. In8th Annual Conference on Robot Learning, 2024

    2024

  20. [28]

    R. Li, H. Wang, and Z. Liu. Survey on mapping human hand motion to robotic hands for teleoperation.IEEE Transactions on Circuits and Systems for Video Technology, 32(5):2647– 2665, 2021

    2021

  21. [29]

    Z. Q. Chen, K. Van Wyk, Y .-W. Chao, W. Yang, A. Mousavian, A. Gupta, and D. Fox. Dex- transfer: Real world multi-fingered dexterous grasping with minimal human demonstrations. arXiv preprint 2209.14284, 2022

    2022 arXiv

  22. [30]

    Q. Chen, K. V . Wyk, Y .-W. Chao, W. Yang, A. Mousavian, A. Gupta, and D. Fox. Learning robust real-world dexterous grasping policies via implicit shape augmentation. InConference on Robot Learning, 2023

    2023

  23. [31]

    S. P. Arunachalam, S. Silwal, B. Evans, and L. Pinto. Dexterous imitation made easy: A learning-based framework for efficient dexterous manipulation. InIEEE International confer- ence on robotics and automation (ICRA), 2023

    2023

  24. [32]

    Y . Han, Z. Chen, K. A. Williams, and H. Ravichandar. Learning prehensile dexterity by imi- tating and emulating state-only observations.IEEE Robotics and Automation Letters, 2024

    2024

  25. [33]

    Qin, Y .-H

    Y . Qin, Y .-H. Wu, S. Liu, H. Jiang, R. Yang, Y . Fu, and X. Wang. Dexmv: Imitation learning for dexterous manipulation from human videos, 2021. 11

    2021

  26. [34]

    Q. She, R. Hu, J. Xu, M. Liu, K. Xu, and H. Huang. Learning high-dof reaching-and-grasping via dynamic representation of gripper-object interaction.ACM Transactions on Graphics (Pro- ceedings of SIGGRAPH), 41(4):97:1–97:14, 2022

    2022

  27. [35]

    Y . Chen, C. Wang, Y . Yang, and C. K. Liu. Object-centric dexterous manipulation from human motion data.arXiv preprint arXiv:2411.04005, 2024

    2024 arXiv

  28. [36]

    C. Wang, H. Shi, W. Wang, R. Zhang, L. Fei-Fei, and C. K. Liu. Dexcap: Scalable and portable mocap data collection system for dexterous manipulation.arXiv preprint arXiv:2403.07788, 2024

    2024 arXiv

  29. [37]

    Ganin, E

    Y . Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V . Lempitsky. Domain-adversarial training of neural networks. InJournal of Machine Learning Research, volume 17, pages 2096–2030, 2016

  30. [38]

    Hoffman, E

    J. Hoffman, E. Tzeng, T. Park, J.-Y . Zhu, P. Isola, K. Saenko, A. A. Efros, and T. Darrell. Cycada: Cycle-consistent adversarial domain adaptation. InInternational Conference on Ma- chine Learning (ICML), pages 1989–1998, 2018

    1989

  31. [39]

    K. Gong, X. Liu, Y . Zhang, J. Feng, and D. Tao. Dlow: Domain flow for adaptation and generalization. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2477–2486, 2019

    2019

  32. [40]

    Abnar, M

    S. Abnar, M. Mutny, B. Scholkopf, and S. Bauer. Gift: Gradual domain adaptation via virtual intermediate domain generation. InInternational Conference on Learning Representations (ICLR), 2021

    2021

  33. [41]

    Z. Xu, Y . Wang, Y . Chen, H. Jin, Y . Wang, and J. Shao. Adversarial domain adaptation with domain mixup. InProceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 6502–6509, 2020

    2020

  34. [42]

    Stadie, P

    B. Stadie, P. Abbeel, and I. Sutskever. Third-person imitation learning. InAdvances in Neural Information Processing Systems (NeurIPS), pages 2296–2306, 2017

    2017

  35. [43]

    J. Choi, S. Oh, J. Choi, and J. Lee. Visual domain-invariant policy learning for visual imitation. InInternational Conference on Learning Representations (ICLR), 2023

    2023

  36. [44]

    T. Yu, C. Finn, A. Xie, S. Dasari, T. Zhang, P. Abbeel, and S. Levine. One-shot imitation from observing humans via domain-adaptive meta-learning. InRobotics: Science and Systems (RSS), 2018

    2018

  37. [45]

    A. D. Edwards and C. L. Isbell. Imitation learning from observations by minimizing inverse dynamics disagreement. InInternational Conference on Machine Learning (ICML), pages 1745–1754, 2019

    2019

  38. [46]

    C. Tan, Y . Su, and J. Wang. Enhancing offline reinforcement learning via dynamics-aware mixup. In2024 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2024. doi:10.1109/IJCNN60899.2024.10650992

    2024

  39. [47]

    Fickinger, M

    T. Fickinger, M. Xu, J. B. Tenenbaum, and R. Urtasun. Cross-domain imitation learning via optimal transport. InConference on Robot Learning (CoRL), 2022

    2022

  40. [48]

    Zhang, X

    P. Zhang, X. Gao, Y . Wu, K. Liu, D. Wang, Z. Wang, B. Zhao, Y . Ding, and X. Li. Moma- kitchen: A 100k+ benchmark for affordance-grounded last-mile navigation in mobile manipu- lation.arXiv preprint arXiv:2503.11081, 2025

    2025 arXiv

  41. [49]

    X. Gao, P. Zhang, D. Qu, D. Wang, Z. Wang, Y . Ding, and B. Zhao. Learning 2d invariant affordance knowledge for 3d affordance grounding. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 3095–3103, 2025. 12

    2025

  42. [50]

    S. Choi, S. Han, W. Kim, J. Chae, W. Jung, and Y . Sung. Domain adaptive imitation learning with visual observation, 2023. URLhttps://arxiv.org/abs/2312.00548

    2023 arXiv

  43. [51]

    Lugaresi, J

    C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.-L. Chang, M. G. Yong, J. Lee, et al. Mediapipe: A framework for building perception pipelines.arXiv preprint arXiv:1906.08172, 2019

    1906 arXiv

  44. [52]

    Y . Rong, T. Shiratori, and H. Joo. Frankmocap: A monocular 3d whole-body pose estimation system via regression and integration. InProceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 1749–1759, 2021

    2021

  45. [53]

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, 2024

    2024

  46. [54]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition, 2015. URL https://arxiv.org/abs/1512.03385

    2015 arXiv

  47. [55]

    M ¨uller

    M. M ¨uller. Dynamic time warping.Information retrieval for music and motion, pages 69–84, 2007

    2007

  48. [56]

    Z. Cui, H. Pan, A. Iyer, S. Haldar, and L. Pinto. Dynamo: In-domain dynamics pretraining for visuo-motor control.Advances in Neural Information Processing Systems, 37:33933–33961, 2024

    2024

  49. [57]

    On the analysis of movement smoothness.Journal of NeuroEngineering and Rehabilitation, 12(1):112, 2015

    Balasubramanian, Sivakumar and Melendez-Calderon, Alejandro and Roby-Brami, Agnes and Burdet, Etienne. On the analysis of movement smoothness.Journal of NeuroEngineering and Rehabilitation, 12(1):112, 2015. doi:10.1186/s12984-015-0090-9. URLhttps://doi.org/ 10.1186/s12984-015-0090-9

    2015 doi

  50. [58]

    Pavlakos, V

    G. Pavlakos, V . Choutas, N. Ghorbani, T. Bolkart, A. A. A. Osman, D. Tzionas, and M. J. Black. Expressive body capture: 3D hands, face, and body from a single image. InProceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 10975–10985, 2019

    2019

  51. [59]

    F. Ding, Y . Zhu, X. Wen, G. Liu, and C. X. Lu. Thermohands: A benchmark for 3d hand pose estimation from egocentric thermal images.arXiv preprint arXiv:2403.09871, 2024

    2024 arXiv

  52. [60]

    Liu and Y

    Z. Liu and Y . Liu. Bridge the gap: From weak to full supervision for temporal action localiza- tion with pseudoformer, 2025. URLhttps://arxiv.org/abs/2504.14860

    2025 arXiv

  53. [61]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haz- iza, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P.-Y . Huang, S.-W. Li, I. Misra, M. Rabbat, V . Sharma, G. Synnaeve, H. Xu, H. Jegou, J. Mairal, P. Labatut, A...

    2024 arXiv

  54. [62]

    Van der Maaten and G

    L. Van der Maaten and G. Hinton. Visualizing data using t-sne.Journal of machine learning research, 9(11), 2008

    2008

  55. [63]

    Alm ´ecija, J

    S. Alm ´ecija, J. B. Smaers, and W. L. Jungers. The evolution of human and ape hand propor- tions.Nature communications, 6(1):7717, 2015

    2015

  56. [64]

    domainness

    S. Balasubramanian, A. Melendez-Calderon, and E. Burdet. A Robust and Sensitive Metric for Quantifying Movement Smoothness.IEEE Transactions on Biomedical Engineering, 59 (8):2126–2136, 2012. doi:10.1109/TBME.2011.2179545. 13 Appendix A Demonstration Collection System The over...

    2012

  57. [66]

    The initial position of the cube is roughly fixed but includes a random offset within the start area

    Basic Object Manipulation.(1)Pick and Place: The robot must pick up a cube from a start position and place it at a designated target location. The initial position of the cube is roughly fixed but includes a random offset within the start area. This task evaluates the robot’s ...

  58. [2019]

    URLhttps://arxiv.org/abs/1812.05418

    arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.