Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learning
Pith reviewed 2026-06-27 09:48 UTC · model grok-4.3
The pith
Tactile-only policies learned via Real2Sim calibration enable blind dexterous grasping at 27% real-world success without vision.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
By constructing a contact-calibrated digital-twin simulator, pretraining a layout-aware tactile encoder on sensor geometry, and aggregating trajectories from object-specific reinforcement learning experts into a tactile-conditioned Diffusion Policy, the method achieves a 27% grasp success rate on a physical multi-fingered hand across 20 objects using only tactile observations.
What carries the argument
The three-component Real2Sim2Real framework: contact calibration for simulation fidelity, self-supervised layout-aware tactile encoding, and diffusion policy aggregation from RL experts.
Load-bearing premise
The contact-calibrated digital-twin simulator reproduces real tactile signals with sufficient accuracy for policy transfer.
What would settle it
Direct comparison of tactile signal patterns from the same contact events in the calibrated simulator versus the physical hand showing substantial mismatch would indicate the calibration fails to bridge the gap.
Figures
read the original abstract
Blind grasping with a dexterous hand is a crucial manipulation capability. Nevertheless, learning such tactile-only policies for real robots remains challenging due to the tactile sim-to-real gap and the limited expressiveness of sparse tactile signals. To bridge this gap, we propose a framework for tactile-only blind grasping that is deployable on a physical multi-fingered robotic hand. Our approach combines three key components. First, we introduce a Real2Sim tactile calibration pipeline that constructs a contact-calibrated digital-twin simulator capable of reproducing real tactile signals. Second, we improve the expressiveness of sparse tactile observations using a layout-aware tactile encoder, which incorporates sensor-geometry priors through self-supervised pretraining. Third, to improve generalization to unseen objects, we train object-specific reinforcement-learning experts in the calibrated simulator and aggregate their successful grasp trajectories into a tactile-conditioned Diffusion Policy. We evaluate our method on a physical LEAP Hand equipped with distributed tactile sensing across 10 seen and 10 unseen objects. The deployed policy achieves a 27\% real-world grasp success rate across all 20 objects, without real-world grasping demonstrations or visual input. Simulation ablations show that layout-aware tactile pretraining improves grasping performance, while sensing-level evaluations confirm that Real2Sim calibration increases the consistency of tactile contact events between simulation and hardware. Together, these results suggest that contact-event calibration, geometry-aware tactile representation learning, and diffusion-based policy aggregation provide an effective path toward tactile-only blind grasping on real dexterous robotic hands. Project page:Dex-Blind-Grasp.github.io.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a Real2Sim2Real framework for tactile-only blind dexterous grasping on a multi-fingered hand. Key components are a contact-calibrated digital-twin simulator via Real2Sim tactile calibration, a layout-aware tactile encoder using sensor-geometry priors and self-supervised pretraining, and aggregation of successful trajectories from object-specific RL experts into a tactile-conditioned Diffusion Policy. On a physical LEAP Hand with distributed tactile sensing, the policy achieves 27% real-world grasp success across 10 seen and 10 unseen objects with no real grasping demonstrations or visual input. Simulation ablations confirm benefits of layout-aware pretraining, and sensing-level checks show improved sim-hardware tactile consistency.
Significance. If the empirical results hold under full experimental reporting, the work demonstrates a concrete, falsifiable path for closing the tactile sim-to-real gap in dexterous manipulation without vision or real demos. The contact-event calibration, geometry-aware representation, and diffusion aggregation together address sparse-signal expressiveness and generalization, offering reusable techniques for tactile robotics that could be extended to other contact-rich tasks.
major comments (1)
- [Abstract and Evaluation] Abstract and Evaluation section: the central claim of 27% real-world success across all 20 objects is presented without trial counts per object, explicit success criteria, baseline comparisons on hardware, or statistical tests. These details are load-bearing for assessing whether the result supports the framework's effectiveness.
minor comments (1)
- [Abstract] The project page URL is given in abbreviated form; expand to a complete https link for accessibility.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback and the recommendation of minor revision. We agree that clearer reporting of experimental details is necessary to support the central claims and will revise the abstract and evaluation sections accordingly.
read point-by-point responses
-
Referee: [Abstract and Evaluation] Abstract and Evaluation section: the central claim of 27% real-world success across all 20 objects is presented without trial counts per object, explicit success criteria, baseline comparisons on hardware, or statistical tests. These details are load-bearing for assessing whether the result supports the framework's effectiveness.
Authors: We acknowledge that the current presentation lacks sufficient detail on these points. In the revised manuscript we will: (1) report the exact number of trials conducted per object (typically 10 trials each for the 20 objects); (2) provide an explicit definition of success (stable lift of the object for at least 5 seconds without dropping or excessive slip); (3) add hardware baseline comparisons, including a random-action policy and a simple heuristic tactile-threshold policy, evaluated under identical conditions; and (4) include statistical measures such as 95% confidence intervals on the success rates and, where appropriate, paired statistical tests across conditions. These additions will be placed in both the abstract and the main evaluation section. We note that because the method uses no real-world grasping demonstrations, learned baselines from prior work are not directly comparable, but the non-learning baselines will still be reported. revision: yes
Circularity Check
No significant circularity identified
full rationale
The paper reports an empirical engineering framework evaluated via physical robot deployment (27% real-world success on 20 objects with no real demos or vision). The three components (Real2Sim calibration, layout-aware encoder, diffusion aggregation) are validated through simulation ablations and hardware sensing consistency checks rather than any derivation chain. No equations, fitted parameters renamed as predictions, self-citations as load-bearing uniqueness theorems, or ansatzes smuggled via prior work appear in the abstract or description. The result is a concrete, externally falsifiable outcome from hardware experiments, making the work self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Roberto, O
C. Roberto, O. Andrew, J. Dinesh, L. Justin, Y . Wenzhen, M. Jitendra, E. H. Adelson, and L. Sergey. More than a feeling: Learning to grasp and regrasp using vision and touch.IEEE Robotics & Automation Letters, 3(4):3300–3307, 2018
2018
-
[2]
Ward-Cherrier, N
B. Ward-Cherrier, N. Pestell, L. Cramphorn, B. Winstone, M. E. Giannaccini, J. Rossiter, and N. F. Lepora. The tactip family: Soft optical tactile sensors with 3d-printed biomimetic morphologies.Soft Robotics, pages 216–227, 2018
2018
-
[3]
Z.-H. Yin, B. Huang, Y . Qin, Q. Chen, and X. Wang. Rotating without seeing: Towards in-hand dexterity through touch, 2023. URLhttps://arxiv.org/abs/2303.10880
arXiv 2023
-
[4]
H. Dang, J. Weisz, and P. K. Allen. Blind grasping: Stable robotic grasping using tactile feedback and hand kinematics. InICRA, pages 5917–5922. IEEE, 2011. URLhttp://dblp. uni-trier.de/db/conf/icra/icra2011.html#DangWA11
2011
-
[5]
Z. Kappassov, J.-A. Corrales, and V . Perdereau. Tactile sensing in dexterous robot hands - review.Robot. Auton. Syst., 74(PA):195–220, Dec. 2015. ISSN 0921-8890. doi:10.1016/j. robot.2015.07.015. URLhttps://doi.org/10.1016/j.robot.2015.07.015
work page doi:10.1016/j 2015
-
[6]
Church, J
A. Church, J. Lloyd, raia hadsell, and N. F. Lepora. Tactile sim-to-real policy transfer via real-to-sim image translation. In5th Annual Conference on Robot Learning, 2021. URL https://openreview.net/forum?id=2NcPgLa7yqD
2021
-
[7]
Garcia-Garcia, B
A. Garcia-Garcia, B. S. Zapata-Impata, S. Orts-Escolano, P. Gil, and J. Garcia-Rodriguez. Tactilegcn: A graph convolutional network for predicting grasp stability with tactile sensors,
-
[8]
URLhttps://arxiv.org/abs/1901.06181
Pith/arXiv arXiv 1901
-
[9]
Y . Li, W. Du, C. Yu, P. Li, Z. Zhao, T. Liu, C. Jiang, Y . Zhu, and S. Huang. Taccel: Scaling up vision-based tactile robotics via high-performance GPU simulation. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URLhttps://openreview. net/forum?id=PtGMadeONU
2025
-
[10]
Si and W
Z. Si and W. Yuan. Taxim: An example-based simulation model for gelsight tactile sensors,
-
[11]
URLhttps://arxiv.org/abs/2109.04027
-
[12]
C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion, 2024. URLhttps://arxiv.org/ abs/2303.04137
Pith/arXiv arXiv 2024
-
[13]
X. Hu, A. Venkatesh, Y . Wan, G. Zheng, N. Jawale, N. Kaur, X. Chen, and P. Birkmeyer. Learning to detect slip through tactile estimation of the contact force field and its entropy,
-
[14]
URLhttps://arxiv.org/abs/2303.00935
-
[15]
E. Holweg, H. Hoeve, W. Jongkind, L. Marconi, C. Melchiorri, and C. Bonivento. Slip de- tection by tactile sensors: algorithms and experimental results. InProceedings of IEEE Inter- national Conference on Robotics and Automation, volume 4, pages 3234–3239 vol.4, 1996. doi:10.1109/ROBOT.1996.509205. 9
- [16]
-
[17]
Y . Yuan, H. Che, Y . Qin, B. Huang, Z.-H. Yin, K.-W. Lee, Y . Wu, S.-C. Lim, and X. Wang. Robot synesthesia: In-hand manipulation with visuotactile sensing, 2024. URLhttps:// arxiv.org/abs/2312.01853
arXiv 2024
-
[18]
P. Lin, Y . Huang, W. Li, J. Ma, C. Xiao, and Z. Jiao. Pp-tac: Paper picking using tactile feedback in dexterous robotic hands.arXiv preprint arXiv:2504.16649, 2025
arXiv 2025
- [19]
-
[20]
W. Yuan, R. Li, M. A. Srinivasan, and E. H. Adelson. Measurement of shear and slip with a gelsight tactile sensor. In2015 IEEE International Conference on Robotics and Automation (ICRA), pages 304–311, 2015. doi:10.1109/ICRA.2015.7139016
- [21]
-
[22]
A. Yamaguchi and C. G. Atkeson. Implementing tactile behaviors using fingervision. In2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), pages 241– 248, 2017. doi:10.1109/HUMANOIDS.2017.8246881
-
[23]
Z. Ding, Y .-Y . Tsai, W. W. Lee, and B. Huang. Sim-to-real transfer for robotic manipulation with tactile sensory, 2021. URLhttps://arxiv.org/abs/2103.00410
arXiv 2021
-
[24]
L. Lach, R. Haschke, D. Tateo, J. Peters, H. J. Ritter, J. B. Sol, and C. Torras. To- wards transferring tactile-based continuous force control policies from simulation to robot. ArXiv, abs/2311.07245, 2023. URLhttps://api.semanticscholar.org/CorpusID: 265149679
arXiv 2023
-
[25]
W. Chen, J. Xu, F. Xiang, X. Yuan, H. Su, and R. Chen. General-purpose sim2real protocol for learning contact-rich manipulation with marker-based visuotactile sensors.IEEE Transactions on Robotics, 40:1509–1526, 2024. URLhttps://api.semanticscholar.org/CorpusID: 267005473
2024
- [26]
-
[27]
Church, J
A. Church, J. Lloyd, R. Hadsell, and N. F. Lepora. Tactile sim-to-real policy transfer via real-to-sim image translation. InConference on Robot Learning, 2021. URLhttps://api. semanticscholar.org/CorpusID:240354208
2021
-
[28]
K.-W. Lee, Y . Qin, X. Wang, and S.-C. Lim. Dextouch: Learning to seek and manipulate objects with tactile dexterity.IEEE Robotics and Automation Letters, 9(12):10772–10779, Dec. 2024. ISSN 2377-3774. doi:10.1109/lra.2024.3478571. URLhttp://dx.doi.org/ 10.1109/LRA.2024.3478571
-
[29]
Z.-H. Yin, B. Huang, Y . Qin, Q. Chen, and X. Wang. Rotating without seeing: Towards in-hand dexterity through touch.Robotics: Science and Systems, 2023
2023
-
[30]
T. Liu, Z. Liu, Z. Jiao, Y . Zhu, and S.-C. Zhu. Synthesizing diverse and physically stable grasps with arbitrary hand structures using differentiable force closure estimator.IEEE Robotics and Automation Letters, 7(1):470–477, Jan. 2022. ISSN 2377-3774. doi:10.1109/lra.2021. 3129138. URLhttp://dx.doi.org/10.1109/LRA.2021.3129138. 10
-
[31]
J. Chen, Y . Ke, and H. Wang. Bodex: Scalable and efficient robotic dexterous grasp synthesis using bilevel optimization, 2025. URLhttps://arxiv.org/abs/2412.16490
arXiv 2025
-
[32]
J. Chen, Y . Ke, L. Peng, and H. Wang. Dexonomy: Synthesizing all dexterous grasp types in a grasp taxonomy.Robotics: Science and Systems, 2025
2025
-
[33]
X. Fei, Z. Xu, H. Fang, T. Zhang, and L. Shao. T(r,o) grasp: Efficient graph diffusion of robot- object spatial transformation for cross-embodiment dexterous grasping, 2025. URLhttps: //arxiv.org/abs/2510.12724
arXiv 2025
-
[34]
M. Yang, C. Lu, A. Church, Y . Lin, C. Ford, H. Li, E. Psomopoulou, D. A. W. Barton, and N. F. Lepora. Anyrotate: Gravity-invariant in-hand object rotation with sim-to-real touch,
-
[35]
URLhttps://arxiv.org/abs/2405.07391
- [36]
-
[37]
H. Li, S. Dikhale, S. Iba, and N. Jamali. Vihope: Visuotactile in-hand object 6d pose estimation with shape completion.IEEE Robotics and Automation Letters, 8(11):6963–6970, 2023. doi: 10.1109/LRA.2023.3313941
- [38]
- [39]
-
[40]
K. Shaw, A. Agarwal, and D. Pathak. Leap hand: Low-cost, efficient, and anthropomorphic hand for robot learning, 2023. URLhttps://arxiv.org/abs/2309.06440
arXiv 2023
-
[41]
X. Huang, Z. Xu, and C. Xiao. Twintac: A wide-range, highly sensitive tactile sensor with real-to-sim digital twin sensor model. In2025 IEEE/RSJ International Conference on Intelli- gent Robots and Systems (IROS), pages 12286–12292, 2025. doi:10.1109/IROS60139.2025. 11247002
-
[42]
Fuji Tsang, M
C. Fuji Tsang, M. Shugrina, J. F. Lafleche, T. Takikawa, J. Wang, and C. e. a. Loop. Kaolin: A pytorch library for accelerating 3d deep learning research.https://github. com/NVIDIAGameWorks/kaolin, 2022
2022
-
[43]
O. I. Vinogradova and F. Feuillebois. Interaction of elastic bodies via surface forces: 2. expo- nential decay.Journal of colloid and interface science, 268(2):464–475, 2003
2003
-
[44]
J. Xu, S. Kim, T. Chen, A. R. Garcia, P. Agrawal, W. Matusik, and S. Sueda. Efficient tactile simulation with differentiability for robotic manipulation. In6th Annual Conference on Robot Learning, 2022. URLhttps://openreview.net/forum?id=6BIffCl6gsM
2022
-
[45]
Z. Zhao, W. Li, Y . Li, T. Liu, B. Li, M. Wang, K. Du, H. Liu, Y . Zhu, Q. Wang, K. Althoefer, and S.-C. Zhu. Embedding high-resolution touch across robotic hands enables adaptive human- like grasping.Nature Machine Intelligence, 7(6):889–900, 2025. ISSN 2522-5839. doi:10. 1038/s42256-025-01053-3. URLhttp://dx.doi.org/10.1038/s42256-025-01053-3
-
[46]
T. Z. Zhao, V . Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware, 2023. URLhttps://arxiv.org/abs/2304.13705
Pith/arXiv arXiv 2023
- [47]
-
[48]
I. Loshchilov and F. Hutter. Decoupled weight decay regularization, 2019. URLhttps: //arxiv.org/abs/1711.05101. 11 Appendix Overview This appendix provides additional details for hardware design, evaluation protocol, failure analysis, ablations, expert data generation, tactile encoder pretraining, and policy deployment. A. Hardware and Tactile Sensing Sys...
Pith/arXiv arXiv 2019
-
[49]
(A) Paired Real--Sim Calibration Setup TwinTac FSR TwinTac FSR Real
Learn tactile alignment parameters from paired sequences 00-4ce6-bc21-530309f9327g.png Sphere Cylinder (B) Calibration Fixtures and Objects (C) Real2Sim Tactile Alignment Calibration improves tactile event consistency between simulated and real sensors Real Sim Real Sim Aligned trajectories in real and simulation ensure rich, diverse contact on many tacti...
-
[50]
Collect controlled-contact trajectories
-
[51]
Replay hand qpos in simulation contact region 3D-printed fixture for repeatable object placement Real Sim Sim Aligned Sim Figure 5:Real2Sim tactile alignment system.(A) Paired calibration trajectories are collected in the real setup and replayed in simulation under matched object poses. During real-world collection, the hand interacts with fixed calibrati...
2000
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.