Pith. sign in

REVIEW 3 major objections 5 minor 47 references

C2Dex: Contact-Consistent Reconstruction and Retargeting for Dexterous Manipulation from Monocular Video

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read C2Dex claims that recovering stable object-side contacts in canonical object space is the missing link between monocular hand-object reconstruction and executable dexterous retargeting.

desk verdict C2Dex has a genuinely useful new idea—stable object-side contacts shared between HOI reconstruction and retargeting—but the headline numbers rest on an untested pose-drift assumption and the paper ships no code, data, or error bars. read the letter →

arxiv 2608.07045 v1 pith:25MSWZPO submitted 2026-08-07 cs.RO cs.CV

Jie Ren , Zhehao Jiang , Yinhong Yang , Haorui Jia , Han Jiang , Ben Li , Yao Yao , Cheng Lin
show 4 more authors
Qiu Shen Zhenshan Bing Xiao-Xiao Long Xun Cao
This is my paper · ORCID
classification cs.ROcs.CV
keywords dexterousmanipulationhand-objectinteractionmonocularvideocontactconsistencyretargetingresidualreinforcementlearningimitationcanonicalobjectspace
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

C2Dex claims that a single shared representation—stable object-side contacts, built by aggregating noisy per-frame contact observations in the object's own coordinate frame—can fix both halves of the video-to-robot pipeline. Guided by these contacts, hand-object reconstruction from monocular video becomes temporally coherent and physically plausible, and the same contacts become explicit targets that let a differently-shaped robot hand preserve the demonstrated grasp. The paper reports end-to-end trajectory success rates of 57.78% on DexYCB and 26.67% on TACO, against at most 17.78% and 10.00% for the strongest baselines under identical criteria, with real-robot replay on eight contact-rich tasks. If true, abundant monocular human videos become a substantially more reliable source of executable dexterous manipulation demonstrations.

What carries the argument

The central mechanism is the stable object-side contact map: per-frame hand-object contact observations, filtered by silhouette overlap and normal compatibility, are transformed into the canonical object frame using the estimated object pose, clustered with DBSCAN within locally stable temporal segments, and summarized by the cluster medoid. This map is the shared interface between the two coupled modules—contact-consistent HOI reconstruction, which uses the map as a trajectory-level loss alongside an SDF penetration penalty, and contact-interaction-preserving retargeting, which uses the same contact targets plus Laplacian optimization over a hand-object interaction graph to keep local geometry intact across embodiments, with residual RL in simulation for physical executability.

What would settle it

Perturb the per-frame object pose estimates in the canonical-space aggregation stage by injecting controlled drift (for example, sinusoidal additive errors of increasing amplitude on DexYCB sequences with ground-truth poses) and measure end-to-end trajectory success. If success rates remain at or near the reported 57.78% and 26.67% even when the injected drift displaces canonical contacts by more than about a centimeter, the claim that stable contacts derived from accurate object pose drive the gains would be falsified; the paper predicts a collapse in success as drift grows.

Watch

Extended reading notes

Core claim

The paper's central claim is that temporally unstable contacts are the root cause of both poor monocular HOI reconstruction and failed cross-embodiment retargeting, and that the cure is to anchor interactions to the object rather than to a particular hand. By projecting frame-wise contact observations into the canonical object frame, clustering them with DBSCAN over locally stable temporal segments, and taking cluster medoids as stable contact points, C2Dex obtains a contact map that is robust to per-frame noise and independent of hand embodiment. This map is used twice: as a trajectory-level contact-consistency loss that pulls the reconstructed human hand into stable contact with the object, and as an explicit retargeting target for a dexterous hand, together with a Laplacian interaction loss that preserves the local hand-object geometry and a residual-RL stage that makes the trajectory executable in simulation. The result, the paper argues, is that monocular human videos can yield executable robot trajectories at success rates several times those of prior video-to-dexterity pipelines.

Load-bearing premise

The load-bearing premise is that the per-frame object poses estimated by the upstream pose tracker are accurate enough that transforming contact observations into the canonical object frame yields clusters that correspond to real physical contacts; the paper itself states that pose drift directly corrupts the canonical-space contact aggregation.

Editorial extensions

If this is right

  • If the central claim holds, monocular internet-scale human videos become a practical demonstration source for dexterous manipulation rather than just visualization content.
  • End-to-end trajectory success on DexYCB would be roughly three times that of prior video-to-dexterity pipelines, making the video-to-robot route competitive with teleoperation and mocap collection for many tasks.
  • Reconstructed hand-object interactions become physically stable: simulation displacement drops to 1.57 cm from 4.55 cm and maximum penetration falls to 3.99 mm from 22.92 mm, so downstream uses of reconstructed HOI would inherit less physically implausible data.
  • Retargeting to a different hand embodiment preserves which object regions are grasped and how, not just finger positions, which is what matters for contact-rich tasks like hanging, sweeping, wiping, and dropping.
  • Ablations show that removing the cross-frame contact consistency collapses success to baseline levels (17.78% on DexYCB, 10.00% on TACO), indicating the stable-contact representation itself is doing the work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The canonical-object-space contact map is a compact, embodiment-independent interaction descriptor, so it could plausibly serve as a training target or reward signal for other robot-learning setups, such as contact-aware skill segmentation or grasp affordance prediction, beyond the trajectory-retargeting pipeline studied here.
  • Because the representation lives on the object, it should transfer across robot hands without re-annotation; a natural stress test would be to evaluate C2Dex with two or more different target hands and check whether the same recovered contacts yield similar success rates.
  • Pose drift is the stated soft spot; jointly optimizing object pose along with hand articulation in the HOI stage, rather than keeping the object trajectory fixed, is a direct extension that would address the failure mode the paper flags.
  • The method currently targets stable contact phases; extending the clustering to handle frequent contact switching could open up in-hand manipulation and finger gaiting, which the paper lists as out of scope.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This manuscript presents C2Dex, a pipeline that takes a monocular human video and produces a dexterous-hand manipulation trajectory. The key idea is to aggregate frame-wise hand-object contact observations over temporally stable segments in the canonical object frame, producing a 'stable object-side contact map' (Eq. 1). This map is used twice: as a trajectory-level constraint in HOI reconstruction (Eq. 2, Eq. 5) and as explicit retargeting targets for the dexterous hand (Eq. 7, Eq. 11), with Laplacian interaction preservation and residual RL refinement. Experiments on DexYCB and TACO report end-to-end trajectory success rates of 57.78% and 26.67%, versus 17.78% and 10.00% for the strongest baselines; ablations show large drops when the contact-consistency module is removed. Real-robot replay of 24 demonstrations across 8 tasks is reported qualitatively.

Significance. The stable-contact representation is a clean, well-motivated idea, and the dual-use design (the same representation for reconstruction and retargeting) is elegant. The evaluation is structured into four parts and includes an ablation study; the retargeting evaluation correctly isolates retargeting from reconstruction by using ground-truth human trajectories. If the reported success rates are robust, C2Dex would make monocular video a substantially more reliable source of dexterous-manipulation demonstrations. However, the paper currently ships no code, no data splits, no error bars or statistical tests, and no sensitivity analysis for the object-pose-estimation assumption that the authors themselves identify as load-bearing; these gaps currently limit the strength of the claim.

major comments (3)
  1. [Sec. III-A(b), Eq. (1), Conclusion] The stable object-side contact map is computed by transforming frame-wise ray-cast contact observations through ProxyPose object poses into canonical object space. The authors state in the Conclusion that 'pose drift directly corrupts the canonical-space contact aggregation,' but the paper provides no measurement of this effect: no ProxyPose pose-error statistics on the evaluation sequences, no correlation between object-pose error and trajectory success, and no ablation that replaces estimated object poses with ground-truth poses. Since Eq. (2) anchors MANO hand vertices to x_stab and Eq. (7) transfers the same contacts to the dexterous hand, a drifted object trajectory can satisfy all internal losses while failing the ground-truth ATE criterion. DexYCB provides ground-truth object poses, so a pose-replacement ablation is feasible on that dataset; I request it, together with pose-error curves, before the headline claim can be considered established.
  2. [Sec. IV-A (End-to-End Trajectory Generation)] The evaluation set is described only as '45 manipulation sequences sampled from DexYCB and 30 demonstration sequences sampled from TACO.' The sampling procedure, sequence-length distribution, object/task composition, and inclusion criteria are not given. Because the main result is a count-based success rate (26/45 vs 8/45 in Table I), the choice of sequences can materially change the comparison. Please specify the exact sampling protocol, list the selected sequences, or release the evaluation split.
  3. [Table I and Sec. IV-A] No error bars, confidence intervals, or statistical tests are reported for the trajectory success rates. With 45 and 30 trials, the difference between C2Dex and the strongest baseline on TACO (8/30 vs 3/30) is not obviously significant under a simple binomial test. Report binomial confidence intervals for each rate and, if feasible, multiple evaluation seeds or a paired test on the shared sequences.
minor comments (5)
  1. [Sec. III-B(d)] The residual-RL stage is described at a high level, but the PPO hyperparameters, reward weights, simulation timestep, and training duration are not reported; since this stage is part of the end-to-end pipeline, these details are needed for reproducibility.
  2. [Sec. IV-C, Table II] The authors correctly note that the object-reconstruction metrics (CD, F@5) reflect the complete reconstruction systems rather than the isolated effect of contact-consistent refinement; this caveat should appear in the main text near Table II, not only in the prose, to avoid over-reading the object-side comparison.
  3. [Sec. IV-F] The real-robot study is explicitly qualitative, which is acceptable for a feasibility claim, but the paper should state how many of the 24 demonstrations were attempted per task and whether any trajectories failed during replay; a short success/failure table would strengthen the claim.
  4. [Sec. III-A(b)] The normal-consistency threshold gamma_n and the temporal segmentation threshold epsilon_theta are introduced but their values are never given; please report them in the implementation details.
  5. [Sec. IV-D, Table III] The definitions of E_prec and E_align do not specify how ground-truth contact regions and contact normals are determined for the retargeting evaluation; a precise description is needed for the retargeting results to be reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: central success rates are judged against external ground-truth object trajectories, while the stable contact map is only an internal constraint and no fitted parameter defines the target result.

full rationale

The derivation chain is: estimate an initial HOI trajectory and object poses with external trackers (DynHaMR, SAM 3D, ProxyPose); aggregate frame-wise contact observations into a stable object-side contact map in canonical object space (Eq. 1); use those contacts as soft constraints in HOI optimization (Eqs. 2 and 5) and as retargeting targets (Eqs. 7 and 11); refine with residual RL; and finally evaluate by simulating the dexterous-hand trajectory and comparing the resulting object trajectory's ATE against ground truth (Table I). No step defines the evaluation target in terms of the derived stable contacts or any fitted parameter: the headline success rates are therefore externally benchmarked, and the baselines are measured under identical criteria. The fact that the stable contacts are computed from the same video as the trajectory means they do not add new external evidence about the true interaction, but this is a self-supervision/robustness property rather than circularity, because the prediction being tested is not the contact map itself but the executability of the resulting trajectory against ground-truth object motion. The Conclusion's limitation that 'pose drift directly corrupts the canonical-space contact aggregation' flags a real dependency on upstream pose accuracy, yet that dependency is a correctness/evidence gap (no sensitivity analysis is reported), not a reduction of the claimed result to its input. No load-bearing self-citation or imported uniqueness claim appears; ManipTrans [10] is cited only for the residual-RL framework, and the benchmark comparison does not rely on that citation. Under the paper's own equations, no circular step can be exhibited.

Assumptions & free parameters 11 free parameters · 6 assumptions · 1 invented entities

The method does not derive constants from first principles; it relies on hand-chosen loss weights and thresholds, on the accuracy of upstream monocular estimators (DynHaMR, SAM 3D, ProxyPose), and on assumptions about contact stability and cross-embodiment geometry transfer. The paper discloses two of these limitations, object pose drift and in-hand manipulation, in its Conclusion.

free parameters (11)
  • lambda_h_c (HOI contact-consistency weight) = 8.0
    Hand-chosen loss weight in Eq. (5); controls how strongly stable contacts pull the hand trajectory.
  • lambda_h_sdf (HOI penetration weight) = 0.01
    Hand-chosen in Sec. IV-A implementation details; balances penetration penalty.
  • lambda_h_reg (HOI regularization weight) = 10.0
    Hand-chosen in Sec. IV-A; anchors optimized articulation to initial estimates.
  • lambda_Lap (Laplacian interaction loss weight) = 500.0
    Hand-chosen in Eq. (11); strength of local geometry preservation.
  • lambda_contact (stable contact loss weight) = 2.0e4
    Hand-chosen in Eq. (11); aligns robot finger contacts with stable object contacts.
  • lambda_pene (collision loss weight) = 1.0e5
    Hand-chosen in Eq. (11); penalizes penetration and self-collision.
  • lambda_smooth (smoothness weight) = 1.0
    Hand-chosen in Eq. (11); regularizes temporal variation.
  • gamma_n (normal-consistency threshold) = not reported
    Filters candidate contact pairs by surface-normal compatibility in Sec. III-A.b; value not given.
  • epsilon_theta (temporal segment threshold) = not reported
    Defines locally stable segments from articulation and pose differences in Sec. III-A.b; value not given.
  • DBSCAN cluster parameters = not reported
    Clusters canonical contact observations per segment in Sec. III-A.b; eps and min_samples not specified.
  • Residual-RL reward weights = not reported
    Reward components for object tracking, hand tracking, contact, and smoothness in Sec. III-B.d; weights not reported.
assumptions (6)
  • domain assumption Per-frame object poses estimated by ProxyPose are accurate enough for canonical-space contact aggregation.
    Contact observations are transformed into the object frame using inverse object pose (Sec. III-A.b), so pose error directly displaces every contact cluster; the paper admits pose drift would corrupt the representation (Conclusion).
  • domain assumption Within a locally stable contact phase, a hand vertex remains associated with the same local object region, so temporal segmentation by articulation and relative pose thresholds is valid.
    The method partitions the sequence using thresholds on hand articulation and relative hand-object pose change (Sec. III-A.b). This assumption fails during in-hand manipulation or frequent contact switching, which the paper lists as a limitation.
  • domain assumption Hand-object contact can be adequately represented as point correspondences between hand vertices and object-surface points detected via silhouette overlap and normal compatibility.
    Contact candidates are obtained from silhouette overlap and filtered by surface-normal compatibility (Sec. III-A.b); occlusions can create false or missing correspondences.
  • domain assumption The Delaunay graph and Laplacian coordinates built from human hand-object interaction can be transferred to a different hand embodiment to preserve task-relevant geometry.
    Laplacian interaction optimization applies the human graph topology and edge weights to dexterous-hand keypoints (Sec. III-B.b); the validity of this cross-embodiment transfer is assumed, not proven.
  • domain assumption MANO hand model and Inspire dexterous hand keypoint correspondences are sufficient to map human hand vertices to robot fingers.
    The stable contact set bC_k is mapped to fingers f via a vertex correspondence (Sec. III-B.a); inaccurate correspondences would misassign contact targets.
  • standard math Standard tools (DBSCAN, PPO, Delaunay triangulation) operate as assumed.
    The method relies on standard algorithms without modification: DBSCAN for clustering, PPO for residual RL, and Delaunay triangulation for Laplacian coordinates.
invented entities (1)
  • Stable object-side contact map
    purpose: Shared representation of where the hand interacts with the object, aggregated across frames in canonical object space; used both to constrain HOI reconstruction and as retargeting targets.
    This is an internal representation computed from the same noisy monocular estimates. It has no external falsifiable handle independent of the paper's pipeline, so it is a paper-introduced construct rather than an independently evidenced entity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of C2Dex: Contact-Consistent Reconstruction and Retargeting for Dexterous Manipulation from Monocular Video." pith.science (2026). https://pith.science/paper/25MSWZPO

@misc{pith2026260807045,
  author       = {Pith},
  title        = {Pith review of: C2Dex: Contact-Consistent Reconstruction and Retargeting for Dexterous Manipulation from Monocular Video},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/25MSWZPO}},
  note         = {Machine review of arXiv:2608.07045}
}
read the original abstract

High-quality demonstrations for dexterous robot manipulation are costly and difficult to collect, whereas monocular human videos provide a scalable source of diverse manipulation behaviors. However, transferring such demonstrations to dexterous robots remains challenging: monocular hand-object interaction (HOI) reconstruction often produces temporally unstable contacts and physically implausible interactions, while conventional retargeting methods struggle to preserve task-relevant contacts and local interaction geometry across different hand embodiments. We present C2Dex, a video-to-dexterous-manipulation framework built around a shared interaction representation: stable object-side contacts recovered by aggregating noisy frame-wise observations in the canonical object space. These stable contacts serve a dual role: as trajectory-level constraints that guide reconstruction toward temporally coherent and physically plausible human HOI trajectories, and as explicit transfer targets for the dexterous hand, where Laplacian interaction optimization preserves the local hand-object geometry across embodiments and residual reinforcement learning refines the trajectory in simulation. Experiments on DexYCB and TACO show that C2Dex achieves end-to-end trajectory success rates of 57.78% and 26.67%, respectively, substantially outperforming the strongest baselines (17.78% and 10.00%) under identical evaluation criteria. Real-robot replay experiments further demonstrate physical feasibility across diverse contact-rich manipulation tasks. Project page: https://k-jie.github.io/C2Dex/

Figures

Figures reproduced from arXiv: 2608.07045 by the authors.

Figure 1
Figure 1. Overview of C2Dex. Given a monocular human video, C2Dex first performs contact-consistent HOI reconstruction by aggregating frame-wise [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Qualitative comparison of HOI reconstruction. C2Dex produces [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 4
Figure 4. Representative real-world trajectory replay results on four contact-rich tasks: [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 11 canonical work pages

  1. [1]

    Dexcap: Scalable and portable mocap data collection system for dexterous manipulation,

    C. Wang, H. Shi, W. Wang, R. Zhang, L. Fei-Fei, and C. K. Liu, “Dexcap: Scalable and portable mocap data collection system for dexterous manipulation,”arXiv preprint arXiv:2403.07788, 2024

    arXiv 2024

  2. [2]

    Holo-dex: Teaching dexterity with immersive mixed reality,

    S. P. Arunachalam, I. G ¨uzey, S. Chintala, and L. Pinto, “Holo-dex: Teaching dexterity with immersive mixed reality,”arXiv preprint arXiv:2210.06463, 2022

    arXiv 2022

  3. [3]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,”arXiv preprint arXiv:2304.13705, 2023

    arXiv 2023

  4. [4]

    Dexmv: Imitation learning for dexterous manipulation from human videos,

    Y . Qin, Y .-H. Wu, S. Liu, H. Jiang, R. Yang, Y . Fu, and X. Wang, “Dexmv: Imitation learning for dexterous manipulation from human videos,” 2022. [Online]. Available: https://arxiv.org/abs/2108.05877

    arXiv 2022

  5. [5]

    Dexvip: Learning dexterous grasping with human hand pose priors from video,

    P. Mandikal and K. Grauman, “Dexvip: Learning dexterous grasping with human hand pose priors from video,” 2022. [Online]. Available: https://arxiv.org/abs/2202.00164

    arXiv 2022

  6. [6]

    Vividex: Learning vision-based dexterous manipulation from human videos,

    Z. Chen, S. Chen, E. Arlaud, I. Laptev, and C. Schmid, “Vividex: Learning vision-based dexterous manipulation from human videos,”

    work page

  7. [7]

    Demobot: Efficient learning of bimanual manipulation with dexterous hands from third-person human videos,

    Y . Xu, X. Mao, E. Miller, X. Yi, Y . Li, Z. Li, and R. B. Fisher, “Demobot: Efficient learning of bimanual manipulation with dexterous hands from third-person human videos,” 2026. [Online]. Available: https://arxiv.org/abs/2601.01651

    arXiv 2026

  8. [8]

    Do as I Do: Dexterous Manipulation Data from Everyday Human Videos

    B. Paliwal, H. Etukuru, W. Liang, P. Abbeel, N. M. M. Shafiullah, and J. Malik, “Do as I Do: Dexterous manipulation data from everyday human videos,” 2026. [Online]. Available: https://arxiv.org/abs/2606.19333

    work page Pith review arXiv 2026

Show all 47 references
  1. [9]

    DexImit: Learning bimanual dexterous manipulation from monocular human videos,

    J. Mu, S. Yang, Y . Bao, H. Bae, T. Wei, L. Xu, B. Li, H. Xu, and J. Pang, “DexImit: Learning bimanual dexterous manipulation from monocular human videos,” 2026. [Online]. Available: https://arxiv.org/abs/2602.10105

    2026

  2. [10]

    Maniptrans: Efficient dexterous bimanual manipulation transfer via residual learning,

    K. Li, P. Li, T. Liu, Y . Li, and S. Huang, “Maniptrans: Efficient dexterous bimanual manipulation transfer via residual learning,” in Proceedings of the Computer Vision and Pattern Recognition Confer- ence, 2025, pp. 6991–7003

    2025

  3. [11]

    Reconstructing hands in 3d with transformers,

    G. Pavlakos, D. Shan, I. Radosavovic, A. Kanazawa, D. Fouhey, and J. Malik, “Reconstructing hands in 3d with transformers,” 2023. [Online]. Available: https://arxiv.org/abs/2312.05251

    2023 arXiv

  4. [12]

    Predicting 4d hand trajectory from monocular videos,

    Y . Ye, Y . Feng, O. Taheri, H. Feng, S. Tulsiani, and M. J. Black, “Predicting 4d hand trajectory from monocular videos,” 2025. [Online]. Available: https://arxiv.org/abs/2501.08329

    2025 arXiv

  5. [13]

    Hold: Category-agnostic 3d reconstruction of interacting hands and objects from video,

    Z. Fan, M. Parelli, M. E. Kadoglou, M. Kocabas, X. Chen, M. J. Black, and O. Hilliges, “Hold: Category-agnostic 3d reconstruction of interacting hands and objects from video,” 2023. [Online]. Available: https://arxiv.org/abs/2311.18448

    2023 arXiv

  6. [14]

    Diffusion-guided reconstruction of everyday hand-object interaction clips,

    Y . Ye, P. Hebbar, A. Gupta, and S. Tulsiani, “Diffusion-guided reconstruction of everyday hand-object interaction clips,” 2023. [Online]. Available: https://arxiv.org/abs/2309.05663

    2023 arXiv

  7. [15]

    Bigs: Bimanual category-agnostic interaction reconstruction from monocular videos via 3d gaussian splatting,

    J. On, K. Gwak, G. Kang, J. Cha, S. Hwang, H. Hwang, and S. Baek, “Bigs: Bimanual category-agnostic interaction reconstruction from monocular videos via 3d gaussian splatting,” 2025. [Online]. Available: https://arxiv.org/abs/2504.09097

    2025 arXiv

  8. [16]

    Easyhoi: Unleashing the power of large models for reconstructing hand-object interactions in the wild,

    Y . Liu, X. Long, Z. Yang, Y . Liu, M. Habermann, C. Theobalt, Y . Ma, and W. Wang, “Easyhoi: Unleashing the power of large models for reconstructing hand-object interactions in the wild,” in2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Jun....

    2025

  9. [17]

    Magichoi: Leveraging 3d priors for accurate hand- object reconstruction from short monocular video clips,

    S. Wang, H. He, M. Parelli, C. Gebhardt, Z. Fan, and J. Song, “Magichoi: Leveraging 3d priors for accurate hand- object reconstruction from short monocular video clips,” 2025. [Online]. Available: https://arxiv.org/abs/2508.05506

    2025 arXiv

  10. [18]

    Agile: Hand-object interaction reconstruction from video via agentic generation,

    J.-C. Shi, B. Ye, T. Liu, J. He, Y . Xu, X. Liu, Z. Li, H. Chen, and C. Shen, “Agile: Hand-object interaction reconstruction from video via agentic generation,” 2026. [Online]. Available: https://arxiv.org/abs/2602.04672

    2026 arXiv

  11. [19]

    DexPilot: Vision-based tele- operation of dexterous robotic hand-arm system,

    A. Handa, K. Van Wyk, W. Yang, J. Liang, Y .-W. Chao, Q. Wan, S. Birchfield, N. Ratliff, and D. Fox, “DexPilot: Vision-based tele- operation of dexterous robotic hand-arm system,” in2020 IEEE International Conference on Robotics and Automation. IEEE, 2020, pp. 9164–9170

    2020

  12. [20]

    AnyTeleop: A general vision-based dexterous robot arm- hand teleoperation system,

    Y . Qin, W. Yang, B. Huang, K. Van Wyk, H. Su, X. Wang, Y .-W. Chao, and D. Fox, “AnyTeleop: A general vision-based dexterous robot arm- hand teleoperation system,” inRobotics: Science and Systems, 2023

    2023

  13. [21]

    Geometric retargeting: A principled, ultrafast neural hand retargeting algorithm,

    Z.-H. Yin, C. Wang, L. Pineda, K. Bodduluri, T. Wu, P. Abbeel, and M. Mukadam, “Geometric retargeting: A principled, ultrafast neural hand retargeting algorithm,” 2025. [Online]. Available: https://arxiv.org/abs/2503.07541

    2025 arXiv

  14. [22]

    Dexflow: A unified approach for dexterous hand pose retargeting and interaction,

    X. Lin, K. Yao, L. Xu, X. Wang, X. Li, Y . Wang, and M. Li, “Dexflow: A unified approach for dexterous hand pose retargeting and interaction,” 2025. [Online]. Available: https://arxiv.org/abs/2505.01083

    2025 arXiv

  15. [23]

    Dexmachina: Functional retargeting for bimanual dexterous manipulation,

    Z. Mandi, Y . Hou, D. Fox, Y . Narang, A. Mandlekar, and S. Song, “Dexmachina: Functional retargeting for bimanual dexterous manipulation,” 2025. [Online]. Available: https://arxiv.org/abs/2505.24853

    2025 arXiv

  16. [24]

    Okami: Teaching humanoid robots manipulation skills through single video imitation,

    J. Li, Y . Zhu, Y . Xie, Z. Jiang, M. Seo, G. Pavlakos, and Y . Zhu, “Okami: Teaching humanoid robots manipulation skills through single video imitation,”arXiv preprint arXiv:2410.11792, 2024

    2024 arXiv

  17. [25]

    Egomimic: Scaling imitation learning via egocentric video,

    S. Kareer, D. Patel, R. Punamiya, P. Mathur, S. Cheng, C. Wang, J. Hoffman, and D. Xu, “Egomimic: Scaling imitation learning via egocentric video,” 2024. [Online]. Available: https://arxiv.org/abs/2410.24221

    2024 arXiv

  18. [26]

    Dexman: Learning bimanual dexterous manipulation from human and generated videos,

    J. Hsieh, K.-H. Tu, K.-H. Hung, and T.-W. Ke, “Dexman: Learning bimanual dexterous manipulation from human and generated videos,”

  19. [27]

    You only teach once: Learn one-shot bimanual robotic manipulation from video demonstrations,

    H. Zhou, R. Wang, Y . Tai, Y . Deng, G. Liu, and K. Jia, “You only teach once: Learn one-shot bimanual robotic manipulation from video demonstrations,”arXiv preprint arXiv:2501.14208, 2025

    2025 arXiv

  20. [28]

    Available: https://arxiv.org/abs/2510.08475

    [Online]. Available: https://arxiv.org/abs/2510.08475

  21. [29]

    Dyn-hamr: Recovering 4d interacting hand motion from a dynamic camera,

    Z. Yu, S. Zafeiriou, and T. Birdal, “Dyn-hamr: Recovering 4d interacting hand motion from a dynamic camera,” 2025. [Online]. Available: https://arxiv.org/abs/2412.12861

    2025 arXiv

  22. [30]

    Learning to transfer human hand skills for robot manipulations,

    S. Park, S. Lee, M. Choi, J. Lee, J. Kim, J. Kim, and H. Joo, “Learning to transfer human hand skills for robot manipulations,”arXiv preprint arXiv:2501.04169, 2025

    2025 arXiv

  23. [31]

    Sam 3d: 3dfy anything in images,

    S. D. Team, X. Chen, F.-J. Chu, P. Gleize, K. J. Liang, A. Sax, H. Tang, W. Wang, M. Guo, T. Hardin, X. Li, A. Lin, J. Liu, Z. Ma, A. Sagar, B. Song, X. Wang, J. Yang, B. Zhang, P. Doll ´ar, G. Gkioxari, M. Feiszli, and J. Malik, “Sam 3d: 3dfy anything in images,” 2025. [Onlin...

    2025 arXiv

  24. [32]

    Embodied hands: Mod- eling and capturing hands and bodies together,

    J. Romero, D. Tzionas, and M. J. Black, “Embodied hands: Mod- eling and capturing hands and bodies together,”arXiv preprint arXiv:2201.02610, 2022

    2022 arXiv

  25. [33]

    A density-based algorithm for discovering clusters in large spatial databases with noise,

    M. Ester, H.-P. Kriegel, J. Sander, X. Xuet al., “A density-based algorithm for discovering clusters in large spatial databases with noise,” inkdd, vol. 96, no. 34, 1996, pp. 226–231

    1996

  26. [34]

    Proxypose: 6-dof pose tracking via video-to-video translation,

    R. Zhang, F. Taubner, P. Ravi, K. N. Kutulakos, and D. B. Lindell, “Proxypose: 6-dof pose tracking via video-to-video translation,”arXiv preprint arXiv:2607.06555, 2026

    2026 arXiv

  27. [35]

    Omniretarget: Interaction-preserving data generation for humanoid whole-body loco-manipulation and scene interaction,

    L. Yang, X. Huang, Z. Wu, A. Kanazawa, P. Abbeel, C. Sferrazza, C. K. Liu, R. Duan, and G. Shi, “Omniretarget: Interaction-preserving data generation for humanoid whole-body loco-manipulation and scene interaction,”arXiv preprint arXiv:2509.26633, 2025

    2025 arXiv

  28. [36]

    Spatial relationship preserving character motion adaptation,

    E. S. Ho, T. Komura, and C.-L. Tai, “Spatial relationship preserving character motion adaptation,” inACM SIGGRAPH 2010 papers, 2010, pp. 1–8

    2010

  29. [37]

    Residual reinforcement learning for robot control,

    T. Johannink, S. Bahl, A. Nair, J. Luo, A. Kumar, M. Loskyll, J. A. Ojea, E. Solowjow, and S. Levine, “Residual reinforcement learning for robot control,” in2019 international conference on robotics and automation (ICRA). IEEE, 2019, pp. 6023–6029

    2019

  30. [38]

    Residual policy learning,

    T. Silver, K. Allen, J. Tenenbaum, and L. Kaelbling, “Residual policy learning,”arXiv preprint arXiv:1812.06298, 2018

    2018 arXiv

  31. [39]

    Teach a robot to fish: Versatile imitation from one minute of demonstrations,

    S. Haldar, J. Pari, A. Rai, and L. Pinto, “Teach a robot to fish: Versatile imitation from one minute of demonstrations,”arXiv preprint arXiv:2303.01497, 2023

    2023 arXiv

  32. [40]

    Residual reinforcement learning from demonstrations,

    M. Alakuijala, G. Dulac-Arnold, J. Mairal, J. Ponce, and C. Schmid, “Residual reinforcement learning from demonstrations,”arXiv preprint arXiv:2106.08050, 2021

    2021 arXiv

  33. [41]

    Isaac gym: High performance gpu-based physics simulation for robot learning,

    V . Makoviychuk, L. Wawrzyniak, Y . Guo, M. Lu, K. Storey, M. Mack- lin, D. Hoeller, N. Rudin, A. Allshire, A. Handaet al., “Isaac gym: High performance gpu-based physics simulation for robot learning,” arXiv preprint arXiv:2108.10470, 2021

    2021 arXiv

  34. [42]

    Residual off-policy rl for finetuning behavior cloning policies,

    L. Ankile, Z. Jiang, R. Duan, G. Shi, P. Abbeel, and A. Nagabandi, “Residual off-policy rl for finetuning behavior cloning policies,”

  35. [43]

    Available: https://arxiv.org/abs/2509.19301

    [Online]. Available: https://arxiv.org/abs/2509.19301

  36. [44]

    TACO: Benchmarking generalizable bimanual tool-action- object understanding,

    Y . Liu, H. Yang, X. Si, L. Liu, Z. Li, Y . Zhang, Y . Liu, and L. Yi, “TACO: Benchmarking generalizable bimanual tool-action- object understanding,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 21 740–21 751

    2024

  37. [45]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” 2017. [Online]. Available: https://arxiv.org/abs/1707.06347

    2017 arXiv

  38. [46]

    Dexycb: A benchmark for capturing hand grasping of objects,

    Y .-W. Chao, W. Yang, Y . Xiang, P. Molchanov, A. Handa, J. Tremblay, Y . S. Narang, K. V . Wyk, U. Iqbal, S. Birchfield, J. Kautz, and D. Fox, “Dexycb: A benchmark for capturing hand grasping of objects,” 2021. [Online]. Available: https://arxiv.org/abs/2104.04631

    2021 arXiv

  39. [2025]

    Available: https://arxiv.org/abs/2404.15709

    [Online]. Available: https://arxiv.org/abs/2404.15709

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.