The reviewed record of science sign in
Pith

arxiv: 2607.06438 · v1 · pith:VTHQZT6K · submitted 2026-07-07 · cs.RO · cs.CV· cs.GR

WristMimic: Full-Body Humanoid Control with Wrist-Guided Manipulation

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-08 05:37 UTCglm-5.2pith:VTHQZT6Krecord.jsonopen to challenge →

classification cs.RO cs.CVcs.GR
keywords humanoid manipulationwhole-body controlphysics-based retargetingwrist-guided graspingdecoupled supervisionreinforcement learninghand-object interactionfinger-agnostic retargeting
0
0 comments X

The pith

Wrist placement alone matches full finger supervision for humanoid manipulation

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

WristMimic is a whole-body humanoid control framework that retargets human-object interaction demonstrations into physics-based simulation. The paper's central claim is that precise wrist positioning, combined with object-tracking and contact rewards, provides a sufficient structural signal for learning grasping and manipulation — without any explicit supervision of finger poses. The method decouples control into two regimes: contact-free body parts (including the wrist) are guided by kinematic pose targets from motion capture, while the 30 finger joints receive no kinematic reference at all and instead learn through reinforcement learning driven by object motion tracking and contact alignment. The wrist serves as the bridge between these regimes because it is typically free from direct object contact (so it can be tracked kinematically) yet it determines the global configuration of the hand and places fingers within reachable grasp affordances. To make this work, the authors introduce a contact window around the first frame of object contact, within which they relax upper-arm supervision to let the policy freely adjust for optimal wrist placement, and they apply phase-specific reset thresholds (7 cm position / 0.2 rad orientation during grasping, looser bounds during approach and stabilization) to keep exploration near feasible wrist poses. Experiments across 40 sequences from two datasets show WristMimic achieving 91.1% average success rate with roughly half the object-tracking error of prior methods that use full finger pose supervision, while also generalizing across three different hand morphologies without finger-specific data.

Core claim

The paper establishes that the wrist is a structural bottleneck for grasp success: if the wrist is correctly placed relative to the object, finger configurations can be left unsupervised and will emerge from physics-based exploration guided by object motion and contact outcomes. This is demonstrated by the ablation showing that adding explicit finger-tracking guidance back into the system actually degrades performance (from 83.3% to 68.7% success rate on ParaHome), likely because enforcing high-dimensional finger pose targets introduces conflicts that destabilize wrist placement. The phase-specific wrist reset thresholds are the load-bearing mechanism: without them, the decoupled formulation

What carries the argument

The core mechanism is the decoupled reward structure: kinematic pose targets are computed only for 19 body joints and 2 wrist joints (the contact-free set), while 30 finger joints (the contact-rich set) are excluded from all pose-tracking rewards and instead shaped by object position/rotation tracking and hand-level contact alignment. Within a contact window of [tc−10, tc+15] frames around first contact, upper-arm joint rewards are zeroed, wrist rewards are held constant, and remaining body joint rewards are reduced — allowing the policy to sacrifice arm fidelity to achieve correct wrist placement. Phase-specific reset thresholds terminate episodes early when the wrist deviates beyond tight

If this is right

  • Manipulation learning pipelines could drop expensive finger motion capture entirely, retaining only wrist and body tracking, which would simplify data collection and reduce hardware requirements for demonstration-based robot training.
  • The principle of identifying structurally important joints and supervising only those — rather than densely supervising all degrees of freedom — could extend to other kinematic chains where a proximal joint determines the reachable workspace of distal joints.
  • The finger-agnostic formulation means a single demonstration could be retargeted to robot hands with different numbers of fingers, different link lengths, or different joint limits without re-capturing finger data, potentially accelerating cross-embodiment skill transfer.
  • The finding that adding finger supervision hurts performance suggests that current finger-tracking approaches may be overconstraining exploration in high-DoF hand spaces, which has implications for how dexterous manipulation benchmarks are designed.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If wrist placement is the true bottleneck, then tasks where the wrist cannot be independently positioned — such as regrasping, in-hand reorientation, or fingertip pinching — would show a sharp performance cliff, which is consistent with the paper's stated limitation but implies the method's applicability boundary is determined by whether the task permits wrist-first planning.
  • The 7 cm / 0.2 rad reset thresholds are hand-tuned heuristics; an adaptive threshold that scales with object size or grasp affordance density could extend the approach to a wider range of object scales without manual tuning per scene.
  • The scene-specific training requirement may stem from the fact that wrist trajectories are still drawn from per-scene motion capture; a learned wrist-placement policy conditioned on object geometry rather than reference trajectories could potentially generalize across scenes.
  • The degradation from added finger supervision suggests a tension between kinematic imitation and contact-driven learning that may be a general phenomenon in high-DoF manipulation, not specific to this framework.

Load-bearing premise

The method assumes that wrist placement is the primary structural bottleneck for grasp success — that once the wrist is correctly positioned, finger behavior can be left to emerge from contact dynamics. If grasp success in more complex tasks depends on fine-grained finger coordination rather than wrist placement, the decoupled formulation breaks down, as the authors acknowledge for in-hand reorientation and finger repositioning tasks.

What would settle it

A task where correct wrist placement is necessary but not sufficient for grasp success — for example, a precision pinch grip requiring specific fingertip contact points, or an in-hand rotation task requiring coordinated finger repositioning — would falsify the claim that wrist guidance plus object/contact rewards is a sufficient signal for manipulation learning.

Figures

Figures reproduced from arXiv: 2607.06438 by Minsu Cho, Wongyun Yu, Youngwoon Kim.

Figure 1
Figure 1. Figure 1: Kinematic imitation does not guarantee successful manipulation and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Training pipeline. The policy observes proprioceptive and goal state to generate actions. Rewards measure alignment with reference motion using time-varying weights that prioritize con￾tacts and object dynamics. Alg.1 details the complete training procedure. Algorithm 1: WristMimic Training 1: Input: Reference {qˆt, qˆ obj t , cˆt}, contact frame tc 2: Preprocessing: Identify grasping hands Jarm 3: for epi… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison across 6 representative scenes. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Key frames from manipulation sequences. Our method successfully han￾dles challenging manipulation scenarios, including grasping a book from a flat table without a handle and grasping a kettle followed by turning the body while maintaining a stable grasp. 4.3 Qualitative Results We present representative qualitative results comparing our method with Inter￾Mimic [66] on six scenes from the ParaHome dataset (… view at source ↗
Figure 5
Figure 5. Figure 5: Grasping pose comparison. Al￾though SkillMimicV2 demonstrates plausible finger closure, the incorrect wrist pose prevents a successful grasp, whereas WristMimic maintains precise wrist alignment. Fig.4 further shows sequen￾tial frames sampled at 50- frame intervals from two rep￾resentative manipulation se￾quences. WristMimic main￾tains stable object contact while producing natural hand poses throughout the… view at source ↗
Figure 6
Figure 6. Figure 6: Scenes involving non￾hand body contact. Scene SR (%) ↑ Pos. err. (cm) ↓ Rot. err. (◦ ) ↓ Largetable 99.9 9.9 10.6 Whitechair 99.7 8.0 6.8 Sit on chair #1 97.2 9.8 5.0 Sit on chair #2 99.8 8.5 6.1 [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Ablation on hand-generalization. Our finger-agnostic approach generalizes across different hand morphologies without finger-specific data. We demonstrate suc￾cessful grasping with three different hands that differ in joint length, size, and pose. 5 Conclusion We have presented WristMimic, a wrist-guided framework for physics-based re￾targeting of human–object interaction demonstrations. Our approach decoup… view at source ↗
read the original abstract

Retargeting human object interaction demonstrations to physics based simulation requires reproducing not only body motion but also the object motion and contacts that make manipulation succeed. However, position only hand trajectories do not specify the contact forces needed to manipulate objects, and directly tracking them can overconstrain contact rich finger behavior. We introduce WristMimic, a wrist guided whole body control framework that explicitly separates contact free body motion from contact rich hand manipulation. The contact free body and wrist are guided by kinematic pose targets, whereas the fingers are not directly supervised by human hand pose. Instead, they learn grasping and manipulation behaviors from object tracking and contact outcomes. Our key insight is that the wrist is the natural gate between these two regimes. It is largely free from contact and can be tracked kinematically, yet it determines the global hand configuration and places the fingers within reachable grasp affordances. To ensure reliable wrist placement during interaction, we introduce wrist specific reset constraints and reward prioritization. Experiments show that WristMimic matches or surpasses methods using full finger pose supervision while enabling finger agnostic retargeting across diverse hand embodiments.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 6 minor

Summary. WristMimic proposes a decoupled framework for physics-based retargeting of human-object interaction demonstrations. The key idea is to supervise contact-free body parts (including the wrist) via kinematic pose targets while leaving 30 finger joints unsupervised, instead learning grasping through object trajectory tracking and contact alignment rewards. The wrist is identified as the structural bridge between these two regimes, motivating wrist-specific reset boundaries and reward weight modulation during a contact window. Experiments on 40 sequences from OMOMO and ParaHome show WristMimic achieving high success rates (98.9% and 83.3%) compared to baselines InterMimic and SkillMimicV2. Ablations isolate the contribution of wrist constraints, and additional experiments demonstrate generalization across hand morphologies and non-hand-contact scenarios.

Significance. The paper presents a clean and well-motivated idea: decoupling finger supervision from wrist/body supervision in physics-based HOI retargeting. The ablation in Table A showing that adding finger guidance actually degrades performance (83.3% → 68.7%) is a genuinely interesting finding that supports the central thesis. The hand morphology generalization experiment (Table 5) concretely demonstrates the finger-agnostic claim. The method is reproducible in principle, with detailed hyperparameters in the supplementary material.

major comments (3)
  1. Table 1, ParaHome rows: InterMimic achieves 0.1% success and SkillMimicV2 achieves 1.3% success. These near-zero rates for published methods on everyday grasping tasks (cups, kettles, books) raise concerns about baseline fairness. The paper states (§4.1) that 'for fair comparison, we train per-scene policies for all methods.' However, InterMimic is described in its original work as a 'universal whole-body control' method; forcing it into per-scene training may not reflect its intended use. The paper should either (a) justify why per-scene training is the appropriate evaluation paradigm for these baselines by citing how they were originally validated, or (b) provide results with baselines trained in their native paradigm. Without this, the 'matches or surpasses' claim in the abstract is difficult to assess. This is load-bearing because the central contribution claim depends on the delta.
  2. §4.1, Success rate definition: The success metric requires <10cm object position error, ≥80% contact duration, and no early termination. The 10cm threshold is a specific choice that directly determines the reported success rates. The paper does not report success rates at alternative thresholds (e.g., 5cm, 15cm) or provide per-scene breakdowns. Given that the baselines fail at 0.1–1.3% on ParaHome, a sensitivity analysis would help distinguish whether the gap reflects a genuine capability difference or a threshold artifact. This is particularly important because the object position errors for WristMimic on ParaHome are 15.3cm (Table 1), which exceeds the 10cm threshold — suggesting the success rate depends on the average-error interpretation rather than per-frame compliance.
  3. Table A vs Table 1 inconsistency: Table A (supplementary) reports WristMimic on ParaHome as 83.3% / 15.3 / 33.9, matching Table 1. But Table 2 (main, 8-sequence ablation) reports 86.5% / 9.9 / 36.0 for the same method, and Table 4 (4-sequence ablation) reports 96.3% / 9.5 / 18.1. The paper should clarify which split each table uses and ensure consistency. The 8-sequence vs 20-sequence distinction is mentioned in the Table A caption but not in Table 2's caption, which could mislead readers.
minor comments (6)
  1. §3.3, Eq. (5): The variable w_red is introduced but its value is not stated in the main text. The supplementary (Table F) mentions body weight reduction (30→10, 2.5→1) but does not explicitly state w_red. This should be clarified.
  2. Fig. 2: The algorithm box and pipeline diagram contain LaTeX rendering artifacts (latexit sha1 strings) that should be replaced with clean rendered equations.
  3. Table 5: The 'Scene-specific' hand achieves 75.8% success, lower than InterMimic hand (95.2%) and OmniGrasp hand (76.4%). The text states performance 'remains comparable,' but a 20-point gap between Scene-specific and InterMimic hand is substantial. The framing should be adjusted.
  4. §4.5: The limitation that scene-specific policies are required is important. The paper could note the training cost per scene to help readers assess practicality.
  5. Table 1 caption: 'SkillMimicV2 is evaluated only on ParaHome, as it specifically utilizes bone-vector representations.' This is a reasonable justification but could note whether OMOMO could be converted.
  6. References: Several arXiv preprints are cited as published (e.g., Ref [36] SONIC, Ref [43] SPIDER, Ref [76] EgoScale). Please verify publication status and update citations where applicable.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the careful and constructive review. The referee raises three major points: (1) concerns about baseline fairness given near-zero success rates for InterMimic and SkillMimicV2 on ParaHome, (2) the need for sensitivity analysis of the 10cm success threshold and clarification of the apparent inconsistency between the 15.3cm average object position error and the success metric, and (3) an inconsistency across Tables 1, 2, 4, and A regarding which data split is used. We address each below and describe revisions to the manuscript.

read point-by-point responses
  1. Referee: Table 1, ParaHome rows: InterMimic achieves 0.1% success and SkillMimicV2 achieves 1.3% success. These near-zero rates for published methods on everyday grasping tasks raise concerns about baseline fairness. The paper should either (a) justify why per-scene training is the appropriate evaluation paradigm for these baselines by citing how they were originally validated, or (b) provide results with baselines trained in their native paradigm.

    Authors: We agree that the near-zero success rates for baselines on ParaHome warrant justification, and we acknowledge that the current manuscript does not adequately explain why per-scene training is the appropriate evaluation paradigm. We will revise the manuscript to address this. Our reasoning is as follows: InterMimic and SkillMimicV2 are designed as universal policies trained on large datasets, but their original evaluations focus on sequences where object geometries and interaction patterns are within the training distribution. ParaHome contains dexterous manipulation scenarios (e.g., grasping small handles, narrow contact regions) that are significantly more challenging than the interaction types these methods were originally validated on. When we attempted to evaluate InterMimic's released universal policy on ParaHome sequences, it failed because the hand morphology, object geometry, and interaction patterns were out-of-distribution. Per-scene training was therefore the fairest comparison we could construct: it gives each method access to the same scene-specific information (object geometry, reference trajectory, contact labels) and isolates the method's learning formulation rather than its training data coverage. We will add this justification to §4.1 with citations to how InterMimic and SkillMimicV2 were originally validated, and we will also report qualitative results showing failure modes of the universal InterMimic policy on ParaHome to make the comparison more transparent. That said, we recognize that providing results with baselines in their native universal paradigm would strengthen the comparison further. We will attempt to retrain InterMimic's universal policy with ParaHome data included in its training set and report those results if feasible within the rebut revision: partial

  2. Referee: §4.1, Success rate definition: The 10cm threshold is a specific choice that directly determines the reported success rates. The paper does not report success rates at alternative thresholds or provide per-scene breakdowns. Given that the baselines fail at 0.1–1.3% on ParaHome, a sensitivity analysis would help distinguish whether the gap reflects a genuine capability difference or a threshold artifact. This is particularly important because the object position errors for WristMimic on ParaHome are 15.3cm, which exceeds the 10cm threshold.

    Authors: The referee raises an important point about the apparent inconsistency between the 15.3cm average object position error and the 10cm success threshold. We clarify that the success metric requires the average object position error over the entire sequence to remain below 10cm, while the 15.3cm reported in Table 1 is the average across all rollouts (including failed ones). Successful rollouts have much lower per-sequence errors, but failed rollouts with large errors inflate the overall average. We agree this is confusing and will revise the metric description to make clear that the 10cm threshold applies per-rollout (i.e., a rollout succeeds only if its average object position error stays below 10cm), while the reported error in Table 1 is the mean across all 10,000 rollouts including failures. We will also add a sensitivity analysis reporting success rates at 5cm, 10cm, and 15cm thresholds, along with per-scene breakdowns for both OMOMO and ParaHome. This will allow readers to assess whether the gap between WristMimic and baselines is robust to threshold choice. revision: yes

  3. Referee: Table A vs Table 1 inconsistency: Table A reports WristMimic on ParaHome as 83.3% / 15.3 / 33.9, matching Table 1. But Table 2 reports 86.5% / 9.9 / 36.0 and Table 4 reports 96.3% / 9.5 / 18.1 for the same method. The paper should clarify which split each table uses and ensure consistency.

    Authors: The referee is correct that the inconsistency across tables is confusing and could mislead readers. The discrepancy arises because different tables use different subsets of ParaHome sequences: Table 1 and Table A use the full 20-sequence ParaHome split, Table 2 uses an 8-sequence subset, and Table 4 uses a 4-sequence subset. This is mentioned in the Table A caption but not in the captions of Tables 2 and 4. We will revise all table captions to explicitly state the number of sequences used and clarify that Tables 2 and 4 use subsets selected for ablation efficiency. We will also add a note in §4.4 directing readers to Table A for full-split results corresponding to the ablation variants. revision: yes

Circularity Check

0 steps flagged

No significant circularity; the method's reward formulation and evaluation are self-contained against external baselines and datasets.

full rationale

The paper's central claim is that wrist guidance plus object/contact rewards suffices for manipulation without finger kinematic supervision. The reward function (Eq. 3-5) is defined in terms of object pose tracking, contact alignment, and wrist/body pose tracking — none of which are defined in terms of the output (finger behavior or success rate). The decoupled formulation excludes finger joints from kinematic supervision by construction (a design choice, not a circular definition), and finger behavior emerges from RL optimization over externally-defined objectives. The ablation in Table A shows that adding finger guidance actually hurts performance (68.7% vs 83.3%), which is an empirical finding, not a tautological consequence of the method's definition. The evaluation metrics (success rate, object position/rotation error) are defined independently of the method's internal reward structure. Baselines (InterMimic, SkillMimicV2) are external methods evaluated on standard datasets (OMOMO, ParaHome). While the near-zero baseline success rates on ParaHome raise fairness concerns (a correctness risk, not circularity), this is a question about experimental setup rather than self-referential logic. The paper cites InterMimic [66] for the reward formulation structure and exponential map coordinates, but these are standard components, not a self-citation chain — the authors have no overlap with InterMimic. No step in the derivation reduces to its own inputs by construction.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The method relies on standard humanoid models and RL frameworks. The primary contributions are conceptual (decoupling strategy) and methodological (wrist-specific constraints), not new entities. The free parameters are typical RL reward shaping and reset thresholds, tuned empirically.

free parameters (6)
  • Contact window parameters (tb, ta) = 10, 15 frames
    Defines the temporal scope of wrist prioritization; chosen empirically.
  • Phase transition points (τ1, τ2) = -2, 12 frames
    Defines grasping vs. stabilization phases; chosen empirically.
  • Wrist reset thresholds (grasping) = 7cm, 0.2 rad
    Hand-tuned to balance exploration and feasibility.
  • Wrist reset thresholds (approach/stabilization) = 15cm, 0.5 rad
    Hand-tuned to allow trajectory exploration.
  • Reward weights (λ) = Various (e.g., 30.0, 2.5, 5.0)
    Standard RL reward shaping parameters, tuned per task.
  • Reduced weight factor (wred) = Unspecified (implied < 1)
    Used to relax non-wrist body joints during contact window.
axioms (4)
  • domain assumption Wrist pose is largely free from object contact and can be tracked kinematically.
    Foundational to the method's decoupling strategy (§1, §3.3).
  • domain assumption Wrist pose determines the global hand configuration and places fingers within reachable grasp affordances.
    Justifies wrist-centric guidance as sufficient for manipulation (§1, §3.3).
  • domain assumption Object kinematics and contact outcomes provide a stronger learning signal than direct finger pose supervision.
    Core hypothesis tested by the method and ablations (§4.4, Tab. A).
  • standard math SMPL-X configuration with 51 actuated joints is a sufficient humanoid model.
    Standard assumption in physics-based character animation (§3.2).

pith-pipeline@v1.1.0-glm · 22696 in / 2137 out tokens · 232934 ms · 2026-07-08T05:37:44.097492+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

78 extracted references · 78 canonical work pages · 21 internal anchors

  1. [1]

    Solving Rubik's Cube with a Robot Hand

    Akkaya, I., Andrychowicz, M., Chociej, M., Litwin, M., McGrew, B., Petron, A., Paino, A., Plappert, M., Powell, G., Ribas, R., et al.: Solving rubik’s cube with a robot hand. arXiv preprint arXiv:1910.07113 (2019)

  2. [2]

    The International Journal of Robotics Research39(1), 3–20 (2020)

    Andrychowicz, O.M., Baker, B., Chociej, M., Jozefowicz, R., McGrew, B., Pa- chocki, J., Petron, A., Plappert, M., Powell, G., Ray, A., et al.: Learning dexterous in-hand manipulation. The International Journal of Robotics Research39(1), 3–20 (2020)

  3. [3]

    In: International Conference on Computer Graphics and Interactive Techniques (2023)

    Bae, J., Won, J., Lim, D., Min, C.H., Kim, Y.: Pmp: Learning to physically interact with environments using part-wise motion priors. In: International Conference on Computer Graphics and Interactive Techniques (2023)

  4. [4]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Banerjee, P., Shkodrani, S., Moulon, P., Hampali, S., Han, S., Zhang, F., Zhang, L., Fountain, J., Miller, E., Basol, S., et al.: Hot3d: Hand and object tracking in 3d from egocentric multi-view videos. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 7061–7071 (2025)

  5. [5]

    In: 2024 International Conference on 3D Vision (3DV)

    Braun, J., Christen, S., Kocabas, M., Aksan, E., Hilliges, O.: Physically plausible full-body hand-object interaction synthesis. In: 2024 International Conference on 3D Vision (3DV). pp. 464–473. IEEE (2024)

  6. [6]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Chao,Y.W.,Yang,W.,Xiang, Y.,Molchanov,P.,Handa,A., Tremblay, J.,Narang, Y.S., Van Wyk, K., Iqbal, U., Birchfield, S., et al.: Dexycb: A benchmark for capturing hand grasping of objects. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 9044–9053 (2021)

  7. [7]

    Dexonomy: Synthesizing All Dexterous Grasp Types in a Grasp Taxonomy

    Chen, J., Ke, Y., Peng, L., Wang, H.: Dexonomy: Synthesizing all dexterous grasp types in a grasp taxonomy. arXiv preprint arXiv:2504.18829 (2025)

  8. [8]

    Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulation

    Chen,Y.,Wang,C.,Fei-Fei,L.,Liu,C.K.:Sequentialdexterity:Chainingdexterous policies for long-horizon manipulation. arXiv preprint arXiv:2309.00987 (2023)

  9. [9]

    Object-Centric Dexterous Manipulation from Human Motion Data

    Chen, Y., Wang, C., Yang, Y., Liu, C.K.: Object-centric dexterous manipulation from human motion data. arXiv preprint arXiv:2411.04005 (2024)

  10. [10]

    In: 2025 IEEE International Conference on Robotics and Automation (ICRA)

    Chen, Z., Chen, S., Arlaud, E., Laptev, I., Schmid, C.: Vividex: Learning vision- based dexterous manipulation from human videos. In: 2025 IEEE International Conference on Robotics and Automation (ICRA). pp. 3336–3343. IEEE (2025)

  11. [11]

    GMT: General Motion Tracking for Humanoid Whole-Body Control

    Chen, Z., Ji, M., Cheng, X., Peng, X., Peng, X.B., Wang, X.: Gmt: General motion tracking for humanoid whole-body control. arXiv preprint arXiv:2506.14770 (2025)

  12. [12]

    Expressive Whole-Body Control for Humanoid Robots

    Cheng, X., Ji, Y., Chen, J., Yang, R., Yang, G., Wang, X.: Expressive whole-body control for humanoid robots. arXiv preprint arXiv:2402.16796 (2024)

  13. [13]

    IEEE transactions on robotics26(1), 1–20 (2009)

    Dahiya, R.S., Metta, G., Valle, M., Sandini, G.: Tactile sensing—from humans to humanoids. IEEE transactions on robotics26(1), 1–20 (2009)

  14. [14]

    In: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Fan, Z., Taheri, O., Tzionas, D., Kocabas, M., Kaufmann, M., Black, M.J., Hilliges, O.: Arctic: A dataset for dexterous bimanual hand-object manipulation. In: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 12943–12954 (2023)

  15. [15]

    HumanPlus: Humanoid Shadowing and Imitation from Humans

    Fu, Z., Zhao, Q., Wu, Q., Wetzstein, G., Finn, C.: Humanplus: Humanoid shad- owing and imitation from humans. arXiv preprint arXiv:2406.10454 (2024)

  16. [16]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Hampali, S., Rad, M., Oberweger, M., Lepetit, V.: Honnotate: A method for 3d annotation of hand and object poses. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 3196–3206 (2020)

  17. [17]

    In: 2023 IEEE 18 W

    Handa, A., Allshire, A., Makoviychuk, V., Petrenko, A., Singh, R., Liu, J., Makovi- ichuk, D., Van Wyk, K., Zhurkevich, A., Sundaralingam, B., et al.: Dextreme: Transfer of agile in-hand manipulation from simulation to reality. In: 2023 IEEE 18 W. Yu et al. International Conference on Robotics and Automation (ICRA). pp. 5977–5984. IEEE (2023)

  18. [18]

    In: ACM SIGGRAPH 2023 Conference Pro- ceedings

    Hassan, M., Guo, Y., Wang, T., Black, M., Fidler, S., Peng, X.B.: Synthesizing physical character-scene interactions. In: ACM SIGGRAPH 2023 Conference Pro- ceedings. pp. 1–9 (2023)

  19. [19]

    DexVLG: Dexterous Vision-Language-Grasp Model at Scale

    He, J., Li, D., Yu, X., Qi, Z., Zhang, W., Chen, J., Zhang, Z., Zhang, Z., Yi, L., Wang, H.: Dexvlg: Dexterous vision-language-grasp model at scale. arXiv preprint arXiv:2507.02747 (2025)

  20. [20]

    OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning

    He,T.,Luo,Z.,He,X.,Xiao,W.,Zhang,C.,Zhang,W.,Kitani,K.,Liu,C.,Shi,G.: Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning. arXiv preprint arXiv:2406.08858 (2024)

  21. [21]

    Hendrycks, D., Gimpel, K.: Gaussian error linear units (GELUs) (Jun 2016)

  22. [22]

    3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing

    Huang, B., Wang, Y., Yang, X., Luo, Y., Li, Y.: 3d-vitac: Learning fine-grained manipulation with visuo-tactile sensing. arXiv preprint arXiv:2410.24091 (2024)

  23. [23]

    In: 2025 IEEE International Conference on Robotics and Automation (ICRA)

    Kareer, S., Patel, D., Punamiya, R., Mathur, P., Cheng, S., Wang, C., Hoffman, J., Xu, D.: Egomimic: Scaling imitation learning via egocentric video. In: 2025 IEEE International Conference on Robotics and Automation (ICRA). pp. 13226–13233. IEEE (2025)

  24. [24]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Kim, J., Kim, J., Na, J., Joo, H.: Parahome: Parameterizing everyday home activi- ties towards 3d generative modeling of human-object interactions. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 1816–1828 (2025)

  25. [25]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Kwon, T., Tekin, B., Stühmer, J., Bogo, F., Pollefeys, M.: H2o: Two hands ma- nipulating objects for first person interaction recognition. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 10138–10148 (2021)

  26. [26]

    DextER: Language-driven Dexterous Grasp Generation with Embodied Reasoning

    Lee, J., Park, E., Cho, M.: Dexter: Language-driven dexterous grasp generation with embodied reasoning. arXiv preprint arXiv:2601.16046 (2026)

  27. [27]

    ACM Transactions on Graphics (TOG)42(6), 1–11 (2023)

    Li, J., Wu, J., Liu, C.K.: Object motion guided human motion synthesis. ACM Transactions on Graphics (TOG)42(6), 1–11 (2023)

  28. [28]

    OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitation

    Li, J., Zhu, Y., Xie, Y., Jiang, Z., Seo, M., Pavlakos, G., Zhu, Y.: Okami: Teaching humanoid robots manipulation skills through single video imitation. arXiv preprint arXiv:2410.11792 (2024)

  29. [29]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Li, K., Li, P., Liu, T., Li, Y., Huang, S.: Maniptrans: Efficient dexterous bimanual manipulation transfer via residual learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6991–7003 (2025)

  30. [30]

    In: European Conference on Computer Vision

    Li, K., Wang, J., Yang, L., Lu, C., Dai, B.: Semgrasp: Semantic grasp generation via language aligned discretization. In: European Conference on Computer Vision. pp. 109–127. Springer (2024)

  31. [31]

    BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion

    Liao, Q., Truong, T.E., Huang, X., Gao, Y., Tevet, G., Sreenath, K., Liu, C.K.: Beyondmimic: From motion tracking to versatile humanoid control via guided dif- fusion. arXiv preprint arXiv:2508.08241 (2025)

  32. [32]

    In: European Conference on Computer Vision

    Lu, J., Kang, H., Li, H., Liu, B., Yang, Y., Huang, Q., Hua, G.: Ugg: Unified generative grasping. In: European Conference on Computer Vision. pp. 414–433. Springer (2024)

  33. [33]

    Advances in Neural Information Processing Systems37, 2161–2184 (2024)

    Luo, Z., Cao, J., Christen, S., Winkler, A., Kitani, K., Xu, W.: Omnigrasp: Grasp- ing diverse objects with simulated humanoids. Advances in Neural Information Processing Systems37, 2161–2184 (2024)

  34. [34]

    Luo, Z., Cao, J., Kitani, K., Xu, W., et al.: Perpetual humanoid control for real- timesimulatedavatars.In:ProceedingsoftheIEEE/CVFInternationalConference on Computer Vision. pp. 10895–10904 (2023) WristMimic 19

  35. [35]

    Universal Humanoid Motion Representations for Physics-Based Control

    Luo, Z., Cao, J., Merel, J., Winkler, A., Huang, J., Kitani, K., Xu, W.: Uni- versal humanoid motion representations for physics-based control. arXiv preprint arXiv:2310.04582 (2023)

  36. [36]

    SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control

    Luo, Z., Yuan, Y., Wang, T., Li, C., Chen, S., Castaneda, F., Cao, Z.A., Li, J., Minor, D., Ben, Q., et al.: Sonic: Supersizing motion tracking for natural humanoid whole-body control. arXiv preprint arXiv:2511.07820 (2025)

  37. [37]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Mahmood, N., Ghorbani, N., Troje, N.F., Pons-Moll, G., Black, M.J.: Amass: Archive of motion capture as surface shapes. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 5442–5451 (2019)

  38. [38]

    In: NeurIPS Datasets and Benchmarks (2021)

    Makoviychuk, V., Wawrzyniak, L., Guo, Y., Lu, M., Storey, K., Macklin, M., Hoeller, D., Rudin, N., Allshire, A., Handa, A., State, G.: Isaac gym: High perfor- mance gpu-based physics simulation for robot learning. In: NeurIPS Datasets and Benchmarks (2021)

  39. [39]

    DexMachina: Functional Retargeting for Bimanual Dexterous Manipulation

    Mandi, Z., Hou, Y., Fox, D., Narang, Y., Mandlekar, A., Song, S.: Dexmachina: Functional retargeting for bimanual dexterous manipulation. arXiv preprint arXiv:2505.24853 (2025)

  40. [40]

    In: European Conference on Computer Vision

    Meng, H., Jin, S., Liu, W., Qian, C., Lin, M., Ouyang, W., Luo, P.: 3d interacting hand pose estimation by hand de-occlusion and removal. In: European Conference on Computer Vision. pp. 380–397. Springer (2022)

  41. [41]

    6m: A dataset and baseline for 3d interacting hand pose estimation from a single rgb image

    Moon, G., Yu, S.I., Wen, H., Shiratori, T., Lee, K.M.: Interhand2. 6m: A dataset and baseline for 3d interacting hand pose estimation from a single rgb image. In: European Conference on Computer Vision. pp. 548–564. Springer (2020)

  42. [42]

    In: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Ohkawa, T., He, K., Sener, F., Hodan, T., Tran, L., Keskin, C.: Assemblyhands: Towards egocentric activity understanding via 3d hand pose estimation. In: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 12999–13008 (2023)

  43. [43]

    arXiv preprint arXiv:2511.09484 (2025)

    Pan, C., Wang, C., Qi, H., Liu, Z., Bharadhwaj, H., Sharma, A., Wu, T., Shi, G., Malik, J., Hogan, F.: Spider: Scalable physics-informed dexterous retargeting. arXiv preprint arXiv:2511.09484 (2025)

  44. [44]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Pan, L., Yang, Z., Dou, Z., Wang, W., Huang, B., Dai, B., Komura, T., Wang, J.: Tokenhsi: Unified synthesis of physical human-scene interactions through task tokenization. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 5379–5391 (2025)

  45. [45]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A.A., Tzionas, D., Black, M.J.: Expressive body capture: 3d hands, face, and body from a single im- age. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10975–10985 (2019)

  46. [46]

    ACM Transactions On Graphics (TOG)37(4), 1–14 (2018)

    Peng, X.B., Abbeel, P., Levine, S., Van de Panne, M.: Deepmimic: Example-guided deep reinforcement learning of physics-based character skills. ACM Transactions On Graphics (TOG)37(4), 1–14 (2018)

  47. [47]

    Peng, X.B., Guo, Y., Halper, L., Levine, S., Fidler, S.: Ase: Large-scale reusable adversarialskillembeddingsforphysicallysimulatedcharacters.ACMTransactions On Graphics (TOG)41(4), 1–17 (2022)

  48. [48]

    ACM Transactions on Graphics (TOG)40(4), 1–20 (2021)

    Peng, X.B., Ma, Z., Abbeel, P., Levine, S., Kanazawa, A.: Amp: Adversarial motion priors for stylized physics-based character control. ACM Transactions on Graphics (TOG)40(4), 1–20 (2021)

  49. [49]

    In: Conference on Robot Learning

    Qi, H., Yi, B., Suresh, S., Lambeta, M., Ma, Y., Calandra, R., Malik, J.: General in-hand object rotation with vision and touch. In: Conference on Robot Learning. pp. 2549–2564. PMLR (2023) 20 W. Yu et al

  50. [50]

    IEEE Robotics and Automation Letters7(4), 10873–10881 (2022)

    Qin, Y., Su, H., Wang, X.: From one hand to multiple hands: Imitation learning for dexterous manipulation from single-camera teleoperation. IEEE Robotics and Automation Letters7(4), 10873–10881 (2022)

  51. [51]

    Proximal Policy Optimization Algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)

  52. [52]

    DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation

    Si, Z., Zhang, G., Ben, Q., Romero, B., Xian, Z., Liu, C., Gan, C.: Difftactile: A physics-based differentiable tactile simulator for contact-rich robotic manipulation. arXiv preprint arXiv:2403.08716 (2024)

  53. [53]

    In: European conference on computer vision

    Taheri, O., Ghorbani, N., Black, M.J., Tzionas, D.: Grab: A dataset of whole- body human grasping of objects. In: European conference on computer vision. pp. 581–600. Springer (2020)

  54. [54]

    arXiv preprint arXiv:2505.19086 (2025)

    Tessler, C., Jiang, Y., Coumans, E., Luo, Z., Chechik, G., Peng, X.B.: Masked- manipulator: Versatile whole-body control for loco-manipulation. arXiv preprint arXiv:2505.19086 (2025)

  55. [55]

    CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character control

    Tevet, G., Raab, S., Cohan, S., Reda, D., Luo, Z., Peng, X.B., Bermano, A.H., van de Panne, M.: Closd: Closing the loop between simulation and diffusion for multi-task character control. arXiv preprint arXiv:2410.03441 (2024)

  56. [56]

    In: European Conference on Computer Vision

    Turpin, D., Wang, L., Heiden, E., Chen, Y.C., Macklin, M., Tsogkas, S., Dickinson, S., Garg, A.: Grasp’d: Differentiable contact-rich grasp synthesis for multi-fingered hands. In: European Conference on Computer Vision. pp. 201–221. Springer (2022)

  57. [57]

    In: IEEE International Conference on Robotics and Automation (ICRA)

    Wang, R., Zhang, J., Chen, J., Xu, Y., Li, P., Liu, T., Wang, H.: Dexgraspnet: A large-scale robotic dexterous grasp dataset for general objects based on simula- tion. In: IEEE International Conference on Robotics and Automation (ICRA). pp. 11359–11366 (2023)

  58. [58]

    Wang, Y., Zhao, Q., Yu, R., Tsui, H.W., Zeng, A., Lin, J., Luo, Z., Yu, J., Li, X., Chen, Q., et al.: Skillmimic: Learning basketball interaction skills from demonstra- tions.In:ProceedingsoftheComputerVisionandPatternRecognitionConference. pp. 17540–17549 (2025)

  59. [59]

    Advances in Neural Information Processing Systems37, 46881–46907 (2024)

    Wei, Y.L., Jiang, J.J., Xing, C., Tan, X.T., Wu, X.M., Li, H., Cutkosky, M., Zheng, W.S.: Grasp as you say: Language-guided dexterous grasp generation. Advances in Neural Information Processing Systems37, 46881–46907 (2024)

  60. [60]

    In: Proceedings of the IEEE/CVF International Conference on Com- puter Vision

    Wei, Y.L., Lin, M., Lin, Y., Jiang, J.J., Wu, X.M., Zeng, L.A., Zheng, W.S.: Afford- dexgrasp: Open-set language-guided dexterous grasp with generalizable-instructive affordance. In: Proceedings of the IEEE/CVF International Conference on Com- puter Vision. pp. 11818–11828 (2025)

  61. [61]

    IEEE Robotics and Automation Letters9(12), 11834–11840 (2024)

    Weng, Z., Lu, H., Kragic, D., Lundell, J.: Dexdiffuser: Generating dexterous grasps with diffusion models. IEEE Robotics and Automation Letters9(12), 11834–11840 (2024)

  62. [62]

    ACM Trans

    Won, J., Gopinath, D., Hodgins, J.: A scalable approach to control diverse be- haviors for physically simulated characters. ACM Trans. Graph.39(4) (2020), https://doi.org/10.1145/3386569.3392381

  63. [63]

    In: Proceedings of the IEEE/CVF International Conference on Com- puter Vision

    Wu, Z., Li, J., Xu, P., Liu, C.K.: Human-object interaction from human-level in- structions. In: Proceedings of the IEEE/CVF International Conference on Com- puter Vision. pp. 11176–11186 (2025)

  64. [64]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Xu, G.H., Wei, Y.L., Zheng, D., Wu, X.M., Zheng, W.S.: Dexterous grasp trans- former. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 17933–17942 (2024)

  65. [65]

    arXiv preprint arXiv:2505.21864 (2025) WristMimic 21

    Xu, M., Zhang, H., Hou, Y., Xu, Z., Fan, L., Veloso, M., Song, S.: Dexumi: Using human hand as the universal manipulation interface for dexterous manipulation. arXiv preprint arXiv:2505.21864 (2025) WristMimic 21

  66. [66]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Xu, S., Ling, H.Y., Wang, Y.X., Gui, L.Y.: Intermimic: Towards universal whole- body control for physics-based human-object interactions. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 12266–12277 (2025)

  67. [67]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Xu, Y., Wan, W., Zhang, J., Liu, H., Shan, Z., Shen, H., Wang, R., Geng, H., Weng, Y., Chen, J., et al.: Unidexgrasp: Universal robotic dexterous grasping via learning diverse proposal generation and goal-conditioned policy. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4737–4746 (2023)

  68. [68]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Yang, L., Li, K., Zhan, X., Wu, F., Xu, A., Liu, L., Lu, C.: Oakink: A large-scale knowledge repository for understanding hand-object interaction. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 20953–20962 (2022)

  69. [69]

    arXiv preprint arXiv:2507.07356 (2025)

    Yin, K., Zeng, W., Fan, K., Dai, M., Wang, Z., Zhang, Q., Tian, Z., Wang, J., Pang, J., Zhang, W.: Unitracker: Learning universal whole-body motion tracker for humanoid robots. arXiv preprint arXiv:2507.07356 (2025)

  70. [70]

    MimicTouch: Leveraging Multi-modal Human Tactile Demonstrations for Contact-rich Manipulation

    Yu, K., Han, Y., Wang, Q., Saxena, V., Xu, D., Zhao, Y.: Mimictouch: Leveraging multi-modal human tactile demonstrations for contact-rich manipulation. arXiv preprint arXiv:2310.16917 (2023)

  71. [71]

    In: Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers

    Yu,R.,Wang,Y.,Zhao,Q.,Tsui,H.W.,Wang,J.,Tan,P.,Chen,Q.:Skillmimic-v2: Learning robust and generalizable interaction skills from sparse and noisy demon- strations. In: Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers. pp. 1–11 (2025)

  72. [72]

    HERMES: Human-to-Robot Embodied Learning from Multi-Source Motion Data for Mobile Dexterous Manipulation

    Yuan, Z., Wei, T., Gu, L., Hua, P., Liang, T., Chen, Y., Xu, H.: Hermes: Human- to-robot embodied learning from multi-source motion data for mobile dexterous manipulation. arXiv preprint arXiv:2508.20085 (2025)

  73. [73]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Zhan, X., Yang, L., Zhao, Y., Mao, K., Xu, H., Lin, Z., Li, K., Lu, C.: Oakink2: A dataset of bimanual hands-object manipulation in complex task completion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 445–456 (2024)

  74. [74]

    In: European Conference on Computer Vision

    Zhang,H.,Christen,S.,Fan,Z.,Hilliges,O.,Song,J.:Graspxl:Generatinggrasping motions for diverse objects at scale. In: European Conference on Computer Vision. pp. 386–403. Springer (2024)

  75. [75]

    arXiv preprint arXiv:2509.13833 (2025)

    Zhang, Z., Guo, J., Chen, C., Wang, J., Lin, C., Lian, Y., Xue, H., Wang, Z., Liu, M., Lyu, J., et al.: Track any motions under any disturbances. arXiv preprint arXiv:2509.13833 (2025)

  76. [76]

    arXiv preprint arXiv:2602.16710 (2026)

    Zheng, R., Niu, D., Xie, Y., Wang, J., Xu, M., Jiang, Y., Castañeda, F., Hu, F., Tan, Y.L., Fu, L., et al.: Egoscale: Scaling dexterous manipulation with diverse egocentric human data. arXiv preprint arXiv:2602.16710 (2026)

  77. [77]

    arXiv preprint arXiv:2502.20900 (2025)

    Zhong, Y., Huang, X., Li, R., Zhang, C., Chen, Z., Guan, T., Zeng, F., Lui, K.N., Ye, Y., Liang, Y., et al.: Dexgraspvla: A vision-language-action framework towards general dexterous grasping. arXiv preprint arXiv:2502.20900 (2025)

  78. [78]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Zimmermann, C., Ceylan, D., Yang, J., Russell, B., Argus, M., Brox, T.: Freihand: A dataset for markerless capture of hand pose and shape from single rgb images. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 813–822 (2019) 22 W. Yu et al. Supplementary Material A Additional Results A.1 Ablation on Finger Guidance ParaHom...