Pith. sign in

REVIEW 4 major objections 1 cited by

Matching human contact forces, not just joint paths, is what makes robot hands grasp like people.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 04:48 UTC pith:MBJ7DBFT

load-bearing objection Solid tactile HOI dataset plus a residual force-reward recipe that clearly beats pure kinematic transfer in sim; open-loop hardware leaves the physical-transfer claim only partially tested. the 4 major comments →

arxiv 2607.09190 v1 pith:MBJ7DBFT submitted 2026-07-10 cs.RO

TactiDex: A Real-World Tactile-Guided Benchmark for Human-Like Dexterous Manipulation

Suting Ni , Hanbing Zhang , Zhenyu Wei , Guo Chen , Chixuan Zhang , Ye Shi , Jingya Wang This is my paper
classification cs.RO
keywords tactile-guided transferhand-object interactiondexterous manipulationhuman-to-robot transfercontact-aware rewardbimanual manipulationsim-to-real
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Most robot skill transfer from human demos only copies joint trajectories. The result looks similar in free space but often fails at contact: fingers hover, penetrate meshes, or squeeze too hard. This paper argues that true human-like dexterity is contact-level, so it releases TactiDex, a real-world dataset that time-aligns whole-hand pressure maps with hand kinematics and object 6D poses across hundreds of single-hand and bimanual sequences. On top of that data it trains TactiSkill, whose residual policy is driven by a three-part tactile reward: a contact-initiation bonus, a force-distribution match to the human, and a safety penalty against force spikes. Experiments on 73 held-out sequences show large gains in contact F1 and tactile-aware success over pure kinematic baselines, and the same open-loop policies transfer to a dual-arm physical platform without on-robot tactile feedback.

Core claim

When human whole-hand pressure is treated as structured supervision rather than an auxiliary observation, a residual policy can simultaneously raise kinematic tracking accuracy, contact timing, and force-level human-likeness, producing more stable single-hand and bimanual manipulation than trajectory imitation alone.

What carries the argument

The tri-component tactile reward: a binary contact-guidance term that rewards matching human contact events, a tanh force-alignment term that matches per-finger force magnitudes, and an exponential safety term that penalizes forces above a human-derived limit.

Load-bearing premise

That mapping raw glove pressure into simulated rigid-body forces, then matching those forces in simulation, is accurate enough to produce real-world grasps even when the robot never senses contact during execution.

What would settle it

On the same 73-sequence test set, measure whether a pure kinematic residual policy that is given identical object meshes and friction can match or exceed TactiSkill's Contact F1 and tactile-aware success; if it can, the tactile reward is not the decisive factor.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 0 minor

Summary. The paper introduces TactiDex, a real-world HOI dataset of 757 sequences over 49 objects that synchronizes whole-hand piezoresistive tactile maps (162 taxels) with high-precision kinematics and object 6D poses, plus contact-aware metrics. Building on this, it proposes TactiSkill: residual RL (asymmetric actor-critic) with a tri-component tactile reward (contact guidance, human-like force alignment via tanh distance, and exponential safety constraints; Eqs. 1–4) that uses finger-wise sensor-to-sim mapped human forces as structured targets. On 73 evaluation sequences in Isaac Gym, full TactiSkill improves SRkin (0.82 vs 0.73), SRtac (0.65 vs 0.39), Contact F1 (0.74 vs 0.56), and safety metrics over a ManipTrans kinematic baseline and ablations (Table 2). Zero-shot open-loop deployment on dual Franka–Inspire hardware is shown qualitatively.

Significance. If the claims hold, this is a meaningful step for contact-aware human-to-robot transfer: existing HOI datasets largely lack synchronized whole-hand force (Table 1), and trajectory-only imitation is known to produce air-grasping and unstable contact. The dataset design (dual-glove + eSync, tactile-gated post-optimization) and the explicit separation of guidance/alignment/safety in the reward are useful contributions. Strengths include a clear ablation table, standardized tactile metrics (MTFE, Contact F1, SRtac, PeakSafe@3N), and an attempt at real hardware. The work would be more significant if force-level human-likeness were validated on the robot rather than only inside the same simulator that supplies Fsim.

major comments (4)
  1. Central claim vs. evidence (Secs. 4.2, 5.1–5.4, App. C.3): The paper claims physically grounded, human-like contact that transfers. Table 2’s SRtac, MTFE, Contact F1, and PeakSafe@3N are computed entirely in Isaac Gym/PhysX against the same aligned Fhuman targets used for training (via linear map M and rigid-body Fsim). Real deployment is open-loop kinematic playback of retargeted residuals; Inspire tactile is present but unused, and no hardware force, peak-force, or contact-F1 numbers are reported. Qualitative snapshots alone do not establish that sim force-matching yields transferable contact physics rather than a sim-specific regularizer that also improves kinematics. Either report on-robot contact/force metrics (or closed-loop tactile), or substantially soften claims of physical human-likeness on hardware.
  2. Table 2 / Sec. 5.1: Results lack multi-seed statistics, error bars, or confidence intervals. Gains (e.g., SRtac 0.6464 vs 0.3935) are large but unquantified for variance; residual PPO on contact-rich tasks is typically seed-sensitive. Report mean±std over ≥3 seeds (or bootstrap over sequences) so the superiority claim is statistically interpretable.
  3. Sec. 5.1 evaluation protocol: The 73 sequences are described as a “representative” split without stating selection criteria, train/eval separation relative to residual training, or whether any sequences used for reward tuning appear in the table. Clarify hold-out status and selection procedure; if the split is curated rather than fixed/random, the reported margins may not generalize across the full 757-sequence corpus.
  4. Sec. 4.2 (1) and App. B.2: Finger-wise sensor-to-sim mapping is linear (Fsim = k·(ADCraw − offset)). Piezoresistive gloves and soft fingertip–object contact are typically nonlinear and spatially distributed; rigid-body normal forces are a coarse proxy for whole-hand pressure maps. Sensitivity of Table 2 metrics to k, offset, and τ, and/or a nonlinear calibration check, is needed to support that matching Fhuman is a faithful physical prior rather than an arbitrary regularizer.

Circularity Check

0 steps flagged

No definitional circularity: human tactile is external measurement used as reward target; metrics compare sim forces to held-out human forces. Mild non-load-bearing alignment of SRtac with the training objective only.

full rationale

TactiDex/TactiSkill is an empirical systems paper (dataset + residual RL + metrics), not a first-principles derivation. Human whole-hand pressure is measured externally (piezoresistive glove, Sec. 3.1), mapped by a calibrated linear ADC-to-force function M (Sec. 4.2, App. B.2), and used as an independent target F_human in the tri-component reward (Eqs. 1–4). Evaluation metrics MTFE and Contact F1 (Eq. 6 and binarized contact) compare simulated forces to those same human targets on 73 sequences; the kinematic baseline ManipTrans [26] (external authors) is scored under identical metrics and fails them (Table 2: SRtac 0.3935 vs 0.6464; Contact F1 0.5569 vs 0.7384). That is standard train-with-reward / evaluate-on-related-metric practice, not a reduction of a claimed prediction to its fitted inputs by construction. Residual architecture and kinematic rewards follow cited ManipTrans; no uniqueness theorem or ansatz is imported from overlapping authors. Real-robot deployment is open-loop kinematic playback (Sec. 5.4, App. C.3) and does not close a force-measurement loop—that is a validity gap, not circularity. The only mild note is that SRtac is defined with MTFE≤3N and Contact F1≥0.3 thresholds that directly favor the tactile-trained policy; this does not make the result definitionally forced, because SRkin, OTEt/r, and ablations still provide independent content. Score 1 reflects that mild metric–objective proximity only; steps empty of true circular reductions.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 3 invented entities

Central claims rest on standard RL/MDP and residual imitation assumptions, domain assumptions about sim contact and sensor mapping, and several hand-chosen reward/safety thresholds. Invented entities are engineering constructs (dataset, reward bundle, metrics) rather than new physical objects; independent evidence for the reward design is the ablation table, not external theory.

free parameters (6)
  • tactile reward weights w_g, w_a, w_s = 0.6 / 1.4 / 0.01
    Set to 0.6, 1.4, 0.01 in Appendix Table 3; balance of guidance vs alignment vs safety is hand-chosen and load-bearing for reported gains.
  • contact threshold τ / τ_c = 0.3 N
    Binary contact indicators and Contact F1 use a minimal force threshold (0.3 N in Table 3); changes redefine guidance and F1.
  • alignment scale σ and safety margin δ / F_limit = F_limit=40 N (table); δ not fully specified
    Tanh force matching and exponential safety penalty depend on scaling and human-plus-margin limit (F_limit listed as 40 N in Table 3; text also uses F_human+δ).
  • SRtac thresholds MTFE≤3.0 N, Contact F1≥0.3; PeakSafe@3N = 3.0 N / 0.3 / 3 N
    Tactile-aware success and safety metrics are defined by fixed cutoffs that directly shape the headline SRtac comparison.
  • sensor-to-sim linear gain k and ADC offset = not numerically reported
    Raw ADC mapped by F_sim = k*(ADC_raw - offset); k from force-gauge calibration. Mapping quality determines whether 'human-like' forces are meaningful.
  • kinematic/task reward coefficients and retargeting weights = see Table 3 and App. C.2
    Object/fingertip/joint tracking weights and L_pos/L_anchor/L_smooth (50/20/10) are tuned; they co-determine residual learning with tactile terms.
axioms (6)
  • domain assumption Contact-aware dexterous transfer can be cast as residual RL on an MDP with human tactile as target features and residual joint actions.
    Section 4.1 formulation; standard in residual imitation but not proven necessary.
  • domain assumption Simulated rigid-body fingertip forces in Isaac Gym/PhysX are adequate supervision targets for human piezoresistive pressure maps after linear calibration.
    Section 4.2 sensor-to-sim alignment and Appendix B.2; core sim-to-real premise.
  • domain assumption Asymmetric actor-critic with privileged simulated forces for the critic improves force modulation without requiring those forces at deployment.
    Section 4.2(2); common privileged-learning assumption.
  • ad hoc to paper Gating geometric proximity by measured pressure correctly identifies true contact intervals for post-optimization.
    Section 3.2 two-stage tactile-constrained post-optimization; dataset quality depends on this filter.
  • ad hoc to paper Open-loop execution of retargeted trajectories is a valid test of physically grounded transfer when real tactile is not fed back.
    Section 5.4 and Appendix C.3 explicitly note open-loop deployment.
  • standard math PPO with GAE and listed hyperparameters yields stable optimization of the composite reward.
    Appendix B; standard RL practice.
invented entities (3)
  • TactiDex dataset no independent evidence
    purpose: Provide synchronized whole-hand tactile, kinematics, object 6D, and annotations for contact-aware transfer benchmarks.
    Primary contribution; independent evidence will depend on public release and external reuse, not yet demonstrated in-text.
  • Tri-component tactile reward (TactiSkill) no independent evidence
    purpose: Structure tactile supervision into guidance, alignment, and safety terms for residual force modulation.
    Engineering reward design; support is internal ablations, not external theory.
  • Tactile-aware metrics (MTFE, Contact F1, SRtac, PeakSafe@3N, SafeTac@3N) no independent evidence
    purpose: Quantify contact fidelity and safety beyond kinematic tracking.
    Defined in Section 5.1; useful but paper-specific thresholds.

pith-pipeline@v1.1.0-grok45 · 24604 in / 4065 out tokens · 37640 ms · 2026-07-13T04:48:18.445294+00:00 · methodology

0 comments
read the original abstract

Tactile feedback is fundamental to Hand-Object Interaction (HOI), governing contact formation, force regulation, and stable manipulation, making it essential for achieving true human-like dexterous manipulation. Yet, current human-to-robot dexterous transfer pipelines primarily rely on kinematic trajectories, resulting in motion imitation without physically grounded interaction. To address this, we introduce TactiDex, a real-world tactile-guided benchmark specifically designed to move dexterous manipulation beyond kinematic mimicry toward contact-level human-likeness. TactiDex provides a comprehensive dataset that elegantly aligns whole-hand tactile signals with multi-granularity kinematic and object states, coupled with standardized evaluation metrics. Building upon this data paradigm, we propose a tactile-driven transfer framework that effectively translates human demonstrations into physically plausible robotic execution. We introduce TactiSkill, a framework built upon a novel tri-component tactile reward that innovatively uses tactile signals as structured supervision. This reward unifies guidance, human-like alignment, and contact constraints into a single objective. Through comprehensive experiments on both single and bimanual tasks, we demonstrate that TactiSkill achieves superior performance in manipulation success and physical realism. This work lays a crucial foundation for advancing tactile-aware dexterous manipulation. Our project page at https://tactidex.github.io/.

Figures

Figures reproduced from arXiv: 2607.09190 by Chixuan Zhang, Guo Chen, Hanbing Zhang, Jingya Wang, Suting Ni, Ye Shi, Zhenyu Wei.

Figure 1
Figure 1. Figure 1: Overview of the TactiDex Data Collection and An [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the TactiSkill Pipeline. (Left) Data Collection & Preparation: Raw multi-modal signals from human [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative Results of TactiSkill. Transfer sequences for single-hand (LH/RH) and bimanual (BiH) tasks. Left: Optimized [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Real-world deployment on our bimanual Franka–Inspire platform. Left: hardware setup with dual Franka arms and [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Object Inventory of TactiDex. Rendered meshes of the 49 diverse everyday objects utilized in our data collection. The [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Task and Interaction Distribution. A Sankey diagram illustrating the hierarchical mapping among task actions, [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Custom Dual-Glove Hardware Integration. To synchronously capture high-precision kinematics and tactile signals, [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Multi-modal Human Demonstrations from the TactiDex Dataset. Representative sequences for LH (Apple), RH [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative Comparison of Contact Quality. Frame-by-frame rollouts of LH (TelephoneHand), RH (PingPang), and [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. ViTacWorld: Scaling Visuo-Tactile World Models for Contact-Rich Robot Manipulation

    cs.RO 2026-07 conditional novelty 5.0

    An action-conditioned visuo-tactile world model generates synthetic camera-plus-touch rollouts that, mixed with real demonstrations, improve downstream contact-rich manipulation policies.

Reference graph

Works this paper leans on

68 extracted references · 16 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Sridhar Pandian Arunachalam, Sneha Silwal, Ben Evans, and Lerrel Pinto. 2023. Dexterous imitation made easy: A learning-based framework for efficient dexter- ous manipulation. In2023 ieee international conference on robotics and automation (icra). IEEE, 5954–5961

    2023

  2. [2]

    Homanga Bharadhwaj, Jay Vakil, Mohit Sharma, Abhinav Gupta, Shubham Tulsiani, and Vikash Kumar. 2024. Roboagent: Generalization and efficiency in robot manipulation via semantic augmentations and action chunking. In 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 4788–4795

    2024

  3. [3]

    Bibit Bianchini, Mathew Halm, and Michael Posa. 2023. Simultaneous learning of contact and continuous dynamics. InConference on Robot Learning. PMLR, 3966–3978

    2023

  4. [4]

    Yu-Wei Chao, Wei Yang, Yu Xiang, Pavlo Molchanov, Ankur Handa, Jonathan Tremblay, Yashraj S Narang, Karl Van Wyk, Umar Iqbal, Stan Birchfield, et al. 2021. Dexycb: A benchmark for capturing hand grasping of objects. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 9044–9053

    2021

  5. [5]

    Xingyu Chen, Fu-Jen Chu, Pierre Gleize, Kevin J Liang, Alexander Sax, Hao Tang, Weiyao Wang, Michelle Guo, Thibaut Hardin, Xiang Li, et al. 2025. Sam 3d: 3dfy anything in images.arXiv preprint arXiv:2511.16624(2025)

    Pith/arXiv arXiv 2025

  6. [6]

    Yuanpei Chen, Yiran Geng, Fangwei Zhong, Jiaming Ji, Jiechuang Jiang, Zongqing Lu, Hao Dong, and Yaodong Yang. 2023. Bi-dexhands: Towards human-level bimanual dexterous manipulation.IEEE Transactions on Pattern Analysis and Machine Intelligence46, 5 (2023), 2804–2818

    2023

  7. [7]

    Yuanpei Chen, Chen Wang, Yaodong Yang, and C Karen Liu. 2024. Object-centric dexterous manipulation from human motion data.arXiv preprint arXiv:2411.04005 (2024)

    Pith/arXiv arXiv 2024

  8. [8]

    Xuxin Cheng, Jialong Li, Shiqi Yang, Ge Yang, and Xiaolong Wang. 2024. Open- television: Teleoperation with immersive active visual feedback.arXiv preprint arXiv:2407.01512(2024)

    Pith/arXiv arXiv 2024

  9. [9]

    Sudeep Dasari, Abhinav Gupta, and Vikash Kumar. 2023. Learning dexterous manipulation from exemplar object trajectories and pre-grasps. In2023 IEEE international conference on robotics and automation (ICRA). IEEE, 3889–3896

    2023

  10. [10]

    Zihan Ding, Nathan F Lepora, and Edward Johns. 2020. Sim-to-real transfer for optical tactile sensing. In2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 1639–1645

    2020

  11. [11]

    Zihan Ding, Ya-Yen Tsai, Wang Wei Lee, and Bidan Huang. 2021. Sim-to-real transfer for robotic manipulation with tactile sensory. In2021 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS). IEEE, 6778–6785

    2021

  12. [12]

    Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kauf- mann, Michael J Black, and Otmar Hilliges. 2023. ARCTIC: A dataset for dexterous bimanual hand-object manipulation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 12943–12954

    2023

  13. [13]

    Hao-Shu Fang, Chenxi Wang, Hongjie Fang, Minghao Gou, Jirong Liu, Hengxu Yan, Wenhai Liu, Yichen Xie, and Cewu Lu. 2023. Anygrasp: Robust and efficient grasp perception in spatial and temporal domains.IEEE Transactions on Robotics 39, 5 (2023), 3929–3945

    2023

  14. [14]

    Harrison Field, Max Yang, Yijiong Lin, Efi Psomopoulou, David Barton, and Nathan F Lepora. 2025. Text2Touch: Tactile In-Hand Manipulation with LLM- Designed Reward Functions.arXiv preprint arXiv:2509.07445(2025)

    Pith/arXiv arXiv 2025

  15. [15]

    Letian Fu, Huang Huang, Lars Berscheid, Hui Li, Ken Goldberg, and Sachin Chitta. 2023. Safe self-supervised learning in real of visuo-tactile feedback policies for industrial insertion. In2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 10380–10386

    2023

  16. [16]

    Beining Han, Abhishek Joshi, and Jia Deng. 2025. Zero-shot Sim2Real Transfer for Magnet-Based Tactile Sensor on Insertion Tasks.arXiv preprint arXiv:2505.02915 (2025)

    Pith/arXiv arXiv 2025

  17. [17]

    Johanna Hansen, Francois Hogan, Dmitriy Rivkin, David Meger, Michael Jenkin, and Gregory Dudek. 2022. Visuotactile-rl: Learning multimodal manipulation policies with deep reinforcement learning. In2022 International Conference on Robotics and Automation (ICRA). IEEE, 8298–8304

    2022

  18. [18]

    Yana Hasson, Gul Varol, Dimitrios Tzionas, Igor Kalevatykh, Michael J Black, Ivan Laptev, and Cordelia Schmid. 2019. Learning joint reconstruction of hands and manipulated objects. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11807–11816

    2019

  19. [19]

    Ryan Hoque, Peide Huang, David J Yoon, Mouli Sivapurapu, and Jian Zhang

  20. [20]

    Egodex: Learning dexterous manipulation from large-scale egocentric video.arXiv preprint arXiv:2505.11709(2025)

    Pith/arXiv arXiv 2025

  21. [21]

    Wenbin Hu, Bidan Huang, Wang Wei Lee, Sicheng Yang, Yu Zheng, and Zhibin Li. 2025. Dexterous in-hand manipulation of slender cylindrical objects through deep reinforcement learning with tactile sensing.Robotics and Autonomous Systems186 (2025), 104904

    2025

  22. [22]

    Binghao Huang, Yixuan Wang, Xinyi Yang, Yiyue Luo, and Yunzhu Li. 2024. 3d-vitac: Learning fine-grained manipulation with visuo-tactile sensing.arXiv preprint arXiv:2410.24091(2024)

    Pith/arXiv arXiv 2024

  23. [23]

    Yitaek Kim, Casper Hewson Rask, and Christoffer Sloth. 2025. Tac2Motion: Contact-Aware Reinforcement Learning with Tactile Feedback for Robotic Hand Manipulation.arXiv preprint arXiv:2509.17812(2025)

    arXiv 2025

  24. [24]

    Taein Kwon, Bugra Tekin, Jan Stühmer, Federica Bogo, and Marc Pollefeys. 2021. H2o: Two hands manipulating objects for first person interaction recognition. In Proceedings of the IEEE/CVF international conference on computer vision. 10138– 10148

    2021

  25. [25]

    Arjun S Lakshmipathy, Jessica K Hodgins, and Nancy S Pollard. 2025. Kinematic motion retargeting for contact-rich anthropomorphic manipulations.ACM Transactions on Graphics44, 2 (2025), 1–20

    2025

  26. [26]

    Jiaman Li, Jiajun Wu, and C Karen Liu. 2023. Object motion guided human motion synthesis.ACM Transactions on Graphics (TOG)42, 6 (2023), 1–11

    2023

  27. [27]

    Kailin Li, Puhao Li, Tengyu Liu, Yuyang Li, and Siyuan Huang. 2025. Maniptrans: Efficient dexterous bimanual manipulation transfer via residual learning. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6991–7003

    2025

  28. [28]

    Sizhe Li, Zhiao Huang, Tao Chen, Tao Du, Hao Su, Joshua B Tenenbaum, and Chuang Gan. 2023. Dexdeform: Dexterous deformable object manipulation with human demonstrations and differentiable physics.arXiv preprint arXiv:2304.03223 (2023)

    Pith/arXiv arXiv 2023

  29. [29]

    Yuchao Li, Ziqi Jin, Jin Liu, and Daolin Ma. 2025. Visuo-tactile feedback policies for terminal assembly facilitated by reinforcement learning.Frontiers in Robotics and AI12 (2025), 1660244

    2025

  30. [30]

    Wenyu Liang, Fen Fang, Cihan Acar, Wei Qi Toh, Ying Sun, Qianli Xu, and Yan Wu. 2022. Visuo-tactile manipulation planning using reinforcement learning with affordance representation.arXiv preprint arXiv:2207.06608(2022)

    Pith/arXiv arXiv 2022

  31. [31]

    Yijiong Lin, Alex Church, Max Yang, Haoran Li, John Lloyd, Dandan Zhang, and Nathan F Lepora. 2023. Bi-touch: Bimanual tactile manipulation with sim-to-real deep reinforcement learning.IEEE Robotics and Automation Letters8, 9 (2023), 5472–5479

    2023

  32. [32]

    Qingtao Liu, Yu Cui, Zhengnan Sun, Gaofeng Li, Jiming Chen, and Qi Ye. 2025. VTDexmanip: A dataset and benchmark for visual-tactile pretraining and dexter- ous manipulation with reinforcement learning. InThe Thirteenth International Conference on Learning Representations

    2025

  33. [33]

    Xueyi Liu, Kangbo Lyu, Jieqiong Zhang, Tao Du, and Li Yi. 2024. Parameterized quasi-physical simulators for dexterous manipulations transfer. InEuropean Conference on Computer Vision. Springer, 164–182

    2024

  34. [34]

    Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, and Li Yi. 2022. Hoi4d: A 4d egocentric dataset for category-level human-object interaction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 21013–21022

    2022

  35. [35]

    Yun Liu, Haolin Yang, Xu Si, Ling Liu, Zipeng Li, Yuxiang Zhang, Yebin Liu, and Li Yi. 2024. Taco: Benchmarking generalizable bimanual tool-action-object understanding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 21740–21751

    2024

  36. [36]

    Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. 2021. Isaac gym: High performance gpu-based physics simulation for robot learning.arXiv preprint arXiv:2108.10470(2021)

    Pith/arXiv arXiv 2021

  37. [37]

    Elle Miller, Trevor McInroe, David Abel, Oisin Mac Aodha, and Sethu Vijayaku- mar. 2025. Enhancing Tactile-based Reinforcement Learning for Robotic Control. arXiv preprint arXiv:2510.21609(2025)

    arXiv 2025

  38. [38]

    Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. 2019. Deepsdf: Learning continuous signed distance functions for shape representation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 165–174

    2019

  39. [39]

    Haozhi Qi, Ashish Kumar, Roberto Calandra, Yi Ma, and Jitendra Malik. 2023. In- hand object rotation via rapid motor adaptation. InConference on Robot Learning. PMLR, 1722–1732

    2023

  40. [40]

    Yuzhe Qin, Yueh-Hua Wu, Shaowei Liu, Hanwen Jiang, Ruihan Yang, Yang Fu, and Xiaolong Wang. 2022. Dexmv: Imitation learning for dexterous manipulation from human videos. InEuropean Conference on Computer Vision. Springer, 570– 587

    2022

  41. [41]

    Yuzhe Qin, Wei Yang, Binghao Huang, Karl Van Wyk, Hao Su, Xiaolong Wang, Yu-Wei Chao, and Dieter Fox. 2023. Anyteleop: A general vision-based dexterous robot arm-hand teleoperation system.arXiv preprint arXiv:2307.04577(2023)

    Pith/arXiv arXiv 2023

  42. [42]

    Javier Romero, Dimitrios Tzionas, and Michael J Black. 2022. Embodied hands: Modeling and capturing hands and bodies together.arXiv preprint arXiv:2201.02610(2022)

    Pith/arXiv arXiv 2022

  43. [43]

    Yuxin Ray Song, Jinzhou Li, Rao Fu, Devin Murphy, Kaichen Zhou, Rishi Shiv, Yaqi Li, Haoyu Xiong, Crystal Elaine Owens, Yilun Du, et al . 2025. OPEN- TOUCH: Bringing Full-Hand Touch to Real-World Interaction.arXiv preprint arXiv:2512.16842(2025)

    arXiv 2025

  44. [44]

    Entong Su, Chengzhe Jia, Yuzhe Qin, Wenxuan Zhou, Annabella Macaluso, Binghao Huang, and Xiaolong Wang. 2024. Sim2real manipulation on unknown objects with tactile-based reinforcement learning. In2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 9234–9241

    2024

  45. [45]

    Omid Taheri, Nima Ghorbani, Michael J Black, and Dimitrios Tzionas. 2020. GRAB: A dataset of whole-body human grasping of objects. InEuropean confer- ence on computer vision. Springer, 581–600. Ni et al

    2020

  46. [46]

    Jiaxian Tang, Xiaogang Yuan, and Shaodong Li. 2025. Visual–Tactile Fusion and SAC-Based Learning for Robot Peg-in-Hole Assembly in Uncertain Environments. Machines13, 7 (2025), 605

    2025

  47. [47]

    Jie Tian, Ran Ji, Lingxiao Yang, Suting Ni, Yuexin Ma, Lan Xu, Jingyi Yu, Ye Shi, and Jingya Wang. 2024. Gaze-guided hand-object interaction synthesis: Dataset and method.arXiv preprint arXiv:2403.16169(2024)

    arXiv 2024

  48. [48]

    Chen Wang, Haochen Shi, Weizhuo Wang, Ruohan Zhang, Li Fei-Fei, and C Karen Liu. 2024. Dexcap: Scalable and portable mocap data collection system for dexterous manipulation.arXiv preprint arXiv:2403.07788(2024)

    Pith/arXiv arXiv 2024

  49. [49]

    Ruicheng Wang, Jialiang Zhang, Jiayi Chen, Yinzhen Xu, Puhao Li, Tengyu Liu, and He Wang. 2022. Dexgraspnet: A large-scale robotic dexterous grasp dataset for general objects based on simulation.arXiv preprint arXiv:2210.02697(2022)

    Pith/arXiv arXiv 2022

  50. [50]

    Youzhuo Wang, Jiayi Ye, Chuyang Xiao, Yiming Zhong, Heng Tao, Hang Yu, Yumeng Liu, Jingyi Yu, and Yuexin Ma. 2025. DexH2R: A benchmark for dynamic dexterous grasping in human-to-robot handover. InProceedings of the IEEE/CVF International Conference on Computer Vision. 12702–12712

    2025

  51. [51]

    Bowen Wen, Jonathan Tremblay, Valts Blukis, Stephen Tyree, Thomas Müller, Alex Evans, Dieter Fox, Jan Kautz, and Stan Birchfield. 2023. Bundlesdf: Neural 6-dof tracking and 3d reconstruction of unknown objects. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 606–617

    2023

  52. [52]

    Bowen Wen, Wei Yang, Jan Kautz, and Stan Birchfield. 2024. Foundationpose: Unified 6d pose estimation and tracking of novel objects. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 17868–17879

    2024

  53. [53]

    Ziwei Xia, Zhen Deng, Bin Fang, Yiyong Yang, and Fuchun Sun. 2022. A review on sensory perception for dexterous robotic manipulation.International Journal of Advanced Robotic Systems19, 2 (2022), 17298806221095974

    2022

  54. [54]

    Chendong Xin, Mingrui Yu, Yongpeng Jiang, Zhefeng Zhang, and Xiang Li

  55. [55]

    Analyzing Key Objectives in Human-to-Robot Retargeting for Dexterous Manipulation.IEEE Robotics and Automation Practice(2026)

    2026

  56. [56]

    Yinzhen Xu, Weikang Wan, Jialiang Zhang, Haoran Liu, Zikang Shan, Hao Shen, Ruicheng Wang, Haoran Geng, Yijia Weng, Jiayi Chen, et al. 2023. Unidexgrasp: Universal robotic dexterous grasping via learning diverse proposal generation and goal-conditioned policy. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition. 4737–4746

    2023

  57. [57]

    Lixin Yang, Kailin Li, Xinyu Zhan, Fei Wu, Anran Xu, Liu Liu, and Cewu Lu

  58. [58]

    InProceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Oakink: A large-scale knowledge repository for understanding hand-object interaction. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 20953–20962

  59. [59]

    Jessica Yin, Haozhi Qi, Youngsun Wi, Sayantan Kundu, Mike Lambeta, William Yang, Changhao Wang, Tingfan Wu, Jitendra Malik, and Tess Hellebrekers. 2025. OSMO: Open-Source Tactile Glove for Human-to-Robot Skill Transfer.arXiv preprint arXiv:2512.08920(2025)

    arXiv 2025

  60. [60]

    Kelin Yu, Yunhai Han, Qixian Wang, Vaibhav Saxena, Danfei Xu, and Ye Zhao

  61. [61]

    Mimictouch: Leveraging multi-modal human tactile demonstrations for contact-rich manipulation.arXiv preprint arXiv:2310.16917(2023)

    Pith/arXiv arXiv 2023

  62. [62]

    Xinyu Zhan, Lixin Yang, Yifei Zhao, Kangrui Mao, Hanlin Xu, Zenan Lin, Kailin Li, and Cewu Lu. 2024. Oakink2: A dataset of bimanual hands-object manipulation in complex task completion. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 445–456

    2024

  63. [63]

    Shenao Zhang, Wanxin Jin, and Zhaoran Wang. 2023. Adaptive barrier smoothing for first-order policy gradient with contact dynamics. InInternational Conference on Machine Learning. PMLR, 41219–41243

    2023

  64. [64]

    Zifan Zhao, Siddhant Haldar, Jinda Cui, Lerrel Pinto, and Raunaq Bhirangi

  65. [65]

    Touch begins where vision ends: Generalizable policies for contact-rich manipulation.arXiv preprint arXiv:2506.13762(2025). TactiDex: A Real-World Tactile-Guided Benchmark for Human-Like Dexterous Manipulation A TactiDex Details A.1 Data Scale and Inventory The TactiDex dataset features over 700 meticulously recorded inter- action sequences, specifically ...

    Pith/arXiv arXiv 2025

  66. [66]

    • Joint States (36-dim):Current joint positionsq ∈R 12, and their trigonometric encodings sin(q),cos( q) ∈R 24 to ensure continuity in rotational space

    Proprioceptive State ( 𝑆𝑝𝑟𝑜𝑝 ∈R 46):Contains the internal kine- matic status of the robotic hand. • Joint States (36-dim):Current joint positionsq ∈R 12, and their trigonometric encodings sin(q),cos( q) ∈R 24 to ensure continuity in rotational space. • Wrist Base State (10-dim):The 6D pose (quaternion) and linear/angular velocities of the wrist base, excl...

  67. [67]

    Target Reference (𝑆𝑡𝑎𝑟𝑔𝑒𝑡 ∈R 330):Provides dense spatial and temporal cues from the human demonstration to guide the imitation process. • Tactile & Geometry Prior (133-dim):A Basis Point Set (BPS) encoding of the object’s point cloud (128-dim) cou- pled with the ground-truth target tactile distances for the fingertips (5-dim). • Future Kinematic Trajector...

  68. [68]

    Ni et al

    Privileged Information ( 𝑆𝑝𝑟𝑖𝑣 ∈R 49):Accessible exclusively to the Critic during simulation training to accurately estimate the value function. Ni et al. Figure 5: Object Inventory of TactiDex. Rendered meshes of the 49 diverse everyday objects utilized in our data collection. The collection spans a wide range of geometries, physical scales, and function...