Pith. sign in

REVIEW 4 major objections 5 minor 57 references

NeuralTouch claims that a tactile reinforcement learning policy conditioned on neural descriptor fields can refine vision-based grasping poses to sub-millimeter accuracy and transfer zero-shot to real robots.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 08:26 UTC pith:ZPCPZAKG

load-bearing objection NeuralTouch shows that NDF-conditioned tactile RL is a genuine improvement for precise grasping, but the real-world zero-shot evidence only covers cylindrical features—the abstract's 'general framework' language outruns the data. the 4 major comments →

arxiv 2510.20390 v2 pith:ZPCPZAKG submitted 2025-10-23 cs.RO

NeuralTouch: Neural Descriptors for Precise Sim-to-Real Tactile Robot Control

Yijiong Lin , Bowen Deng , Keju Pu , Chenghua Lu , Max Yang , Efi Psomopoulou , Nathan F. Lepora This is my paper
classification cs.RO
keywords neural descriptor fieldstactile servoingsim-to-real transferreinforcement learningrobot graspingpeg-in-holemultimodal manipulationtactile sensing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

NeuralTouch combines visual Neural Descriptor Fields (NDF) with tactile reinforcement learning to refine robot grasping poses. The paper claims that conditioning a tactile RL policy on NDF pose descriptors lets a single policy grasp different contact features—rims, handles, bottle necks, bolt heads—without predefined contact geometry. In simulation, grasp position errors drop from 11–14 mm with NDF alone to under 1 mm. The same policy transfers zero-shot to real robots, improving bottle-lid opening from 30–45% to 85–90% success and enabling peg-out/in-hole insertions that the vision-only baseline almost never completes. The aim is a general framework for precise, contact-rich manipulation that avoids task-specific contact models.

Core claim

The central claim is that neural descriptor fields can serve as the conditioning signal for a tactile servoing policy, replacing the hand-specified contact geometry used by prior tactile RL. The policy takes tactile images, proprioception, and an NDF descriptor of the target pose, and outputs a 7D action (end-effector twist and gripper width). Because NDF descriptors are SE(3)-equivariant and encode geometric correspondence across objects in a category, the same trained policy can servo to different target features on unseen objects, and even switch targets online by updating the input descriptor. The authors show that this raises simulated grasping accuracy to roughly a millimeter position

What carries the argument

The key object is the neural pose descriptor field: a function that maps a 6D pose and an object point cloud to a descriptor vector, built from an SE(3)-equivariant occupancy network. It defines a target contact pose implicitly, so the RL policy can read the desired geometric relationship without a categorical contact type. The policy is a deep RL network trained with a policy-gradient algorithm, conditioned on the descriptor along with tactile images and proprioception. The descriptor does the work of disambiguating which feature is the target—something tactile feedback alone cannot do, as the ablation with a tactile-only RL baseline shows.

Load-bearing premise

The entire zero-shot sim-to-real claim rests on the real-to-sim tactile image translation being accurate enough—and the paper itself reports that this transfer fails on curved surfaces and on very light contacts, so if that translation is inaccurate for a new geometry or contact force, the policy cannot transfer.

What would settle it

Take a previously unseen object with a curved or non-cylindrical contact surface (e.g., a mug handle or a spherical knob), run the NeuralTouch pipeline with the same trained policy and translation model, and measure the achieved grasp pose error and task success. If the success rate collapses or errors exceed a few millimeters, the claim that the framework generalizes beyond cylindrical features is falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Grasping accuracy in simulation improves to sub-millimeter position error (0.7–1.0 mm across six target features), compared to 9.5–13.6 mm for NDF alone.
  • A single policy, without retraining, can servo to different target contact features and switch targets online by changing the input descriptor.
  • Zero-shot sim-to-real transfer works for cylindrical features: bottle-lid opening success rises to 85–90% and peg-out/in-hole reaches 55% for a bolt with 2 mm clearance.
  • The framework removes the need for predefined contact-geometry types in tactile servoing, extending from flat edges and surfaces to curved surfaces and handles.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the descriptor continues to generalize, combining NeuralTouch with closed-loop insertion policies—which the authors note are complementary—could yield full assembly tasks that are robust to pose errors.
  • The tactile real-to-sim transfer is the bottleneck; improving its fidelity for light contacts and curved surfaces, perhaps with physics-based simulators or better domain randomization, would likely extend the method beyond cylindrical features.
  • Because the descriptor can be swapped online, the same policy architecture could be trained once on many categories and then used for few-shot adaptation to new tasks by providing one or a few demonstrations of the target pose descriptor.
  • The failure mode of tactile aliasing on light contacts suggests that adding force or shear information, or temporal accumulation of contacts, may be necessary for sub-millimeter real-world accuracy.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes NeuralTouch, a two-stage manipulation framework that combines Neural Descriptor Fields (NDF) for coarse, category-level grasp pose generation with a deep RL policy that refines the pose using tactile images, proprioception, and NDF target descriptors. The fine-phase policy is trained in simulation with PPO and transferred zero-shot to a real Franka arm with TacTip sensors via a pix2pix-based real-to-sim tactile translation. In simulation, NeuralTouch reduces position errors from 9.5-13.6 mm (NDF) to 0.7-1.0 mm and outperforms an RL-touch baseline lacking descriptors. Real-world experiments on bottle-lid opening and peg-out/in-hole with cylindrical objects report higher success rates than NDF alone. The paper claims this is a general framework for precise contact-rich manipulation.

Significance. If the reported results hold, NeuralTouch is a valuable contribution: it shows that conditioning a tactile RL policy on neural descriptors can substantially improve precision over vision-only NDF while retaining category-level generalization, and it demonstrates zero-shot sim-to-real for cylindrical features. The simulation ablation is well motivated, the online target-switching result (Fig. 6) is compelling, and the authors are transparent about the tactile-transfer limitations on curved surfaces and light contacts. However, the absence of statistical rigor and the narrow real-world geometry scope mean the evidence does not yet support the strong 'general framework' claim.

major comments (4)
  1. [Sec. V-A / Tables I-IV] The central quantitative claims are reported without error bars, standard deviations, or trial counts. Table I gives 'average' errors with no variance despite the training curves in Fig. 5 being over 10 seeds; Table II gives success rates for 60 trials without confidence intervals; Tables III-IV give success rates without stating the number of real-world trials. This makes it impossible to assess whether the large improvements (e.g., 0.8 mm vs 13.6 mm in Table I) are significant or stable. Please report per-seed/per-trial statistics, confidence intervals, and trial counts for all tables.
  2. [Sec. IV-C2 / V-A / Eq. (4)] The training initial poses are sampled uniformly from [±20 mm, ±20°] rather than from the NDF optimizer, because 'initial grasping poses are not sampled from the NDF optimizer during training' (Sec. IV-C2). The Table I comparison with the NDF baseline is therefore not an end-to-end comparison from the same coarse-pose distribution: the NDF row reports errors of the NDF-generated pose, while the NeuralTouch row reports errors after refinement from random poses within a fixed box. To substantiate the claim that NeuralTouch improves NDF-based grasping, the evaluation should initialize the policy from the NDF output (or a held-out set of NDF failures) and report the resulting final errors, or at least report the distribution of NDF initial errors and show the policy is robust over that distribution.
  3. [Sec. IV-C3 / V-C1 / Abstract] The zero-shot sim-to-real evidence is restricted to cylindrical features. The authors state that the tactile real-to-sim transfer 'performed unsatisfactorily with the curved surfaces of the mugs and handles' (Sec. IV-C3), so real-world experiments were limited to cylinders; they additionally report failures on light contacts due to tactile aliasing (Sec. V-C1). Consequently, the real-world results support the method for cylinder-like geometries with relatively firm contacts only. The abstract's 'general framework for precise, contact-rich robotic manipulation' and the phrase 'various objects' overstate the demonstrated scope. Either add real-world experiments on non-cylindrical features or revise the claims to specify cylindrical-contact geometry.
  4. [Sec. IV-C3 / Fig. 2] The real-to-sim tactile image transfer is a load-bearing component of the zero-shot claim, but it is evaluated only qualitatively. No quantitative validation of the pix2pix transfer accuracy (e.g., pose-estimation error on a held-out set of real contacts) is provided, and the paper notes that it 'performed unsatisfactorily' for curved surfaces. Given that the policy depends on this transfer, please report quantitative transfer accuracy and, ideally, analyze how transfer error compares to the policy's tolerance. This would clarify the boundary conditions of the otherwise promising results.
minor comments (5)
  1. [Sec. II-C] Typo: 'relared' should be 'related'.
  2. [Sec. IV-C1 / Eq. (4)] The number of target descriptor samples per feature is fixed at 12, but the sampling procedure is not described; please specify how the 12 descriptors are drawn (e.g., from demonstrations, perturbed poses) and whether this choice affects performance.
  3. [Fig. 5] The caption labels 'Without Neural Descriptors' and 'With Neural Descriptors' without indicating which curve corresponds to which color; please clarify in the caption or legend.
  4. [Table I] The heading 'average error' is ambiguous because the rows aggregate over six target features; please define how the average is computed and report per-feature results with variance.
  5. [Sec. III-C / Fig. 6] The online target-switching demo is interesting, but the quantitative success of switching (e.g., number of trials) is not reported; please add at least a brief evaluation or clarify that it is a qualitative illustration.

Circularity Check

0 steps flagged

No significant circularity: NeuralTouch's improvements are empirical results from an external NDF representation and a goal-conditioned RL policy.

full rationale

The paper's central claim is an empirical system result, not a derivation that reduces to its own inputs. The coarse grasping pose is obtained from the external Neural Descriptor Fields work (Simeonov et al., [4]) via Eq. (4), which is a pose regression objective, not a fitted prediction of the paper's own evaluation outcomes. The RL policy is trained with PPO in simulation, conditioned on target NDF descriptors, tactile images, and proprioception; the descriptors are observations specifying the goal, not parameters fitted to the test results. The reported simulation and real-world success rates are measured outcomes of executing the policy, and there is no equation in the paper where the predicted quantity is identical by construction to a fitted input. The real-to-sim tactile transfer uses a pix2pix GAN trained on paired simulated/real tactile images, following prior work [26], [27]; this is a domain-transfer component and is not a claim that the downstream task success is statistically forced by that training data. Self-citations appear (e.g., Tactile Gym 2.0 [27], real-to-sim translation [26], tactile aliasing [57]), but they are supporting tools or explanatory references, not load-bearing uniqueness theorems or the sole justification for the central improvement claim. The manuscript explicitly reports limitations—curved-surface transfer failure (Sec. IV-C3) and light-contact aliasing (Sec. V-C1)—which narrow the scope of the 'general framework' claim but do not indicate circular reasoning. The central derivation chain is therefore self-contained with respect to the paper's inputs.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 0 invented entities

The central claim rests on several domain assumptions about contact geometry, simulator fidelity, and transfer. No new physical entities are introduced. The main free parameters are design choices in the training setup that are not derived from theory.

free parameters (2)
  • Initial pose perturbation range = ±20 mm, ±20° in each translation/rotation axis
    Used to sample training starting poses for the RL policy (Sec. IV-C2). Chosen by hand to mimic NDF errors, but not validated against the actual NDF error distribution, which could affect policy performance at test time.
  • Number of target descriptor samples per feature = 12
    Number of NDF vectors collected per target feature to train the policy (Sec. IV-C1). This is a manual choice; using more samples might improve descriptor reliability but also increase complexity.
axioms (5)
  • domain assumption The target contact is assumed to be where tactile sensors are perpendicular to the local surface (normal contact assumption).
    Used to unify different contact geometries (Sec. III-C). If optimal grasping does not require normal contact, the unified target pose G_τ may be misspecified.
  • domain assumption The pretrained occupancy network on ShapeNet categories (bottles, mugs, bolts) generalizes to unseen real-world objects in these categories.
    The method relies on this for coarse pose generation (Sec. IV-C1). If the categories differ geometrically, the NDF descriptors may be inaccurate.
  • domain assumption The pix2pix GAN trained on paired real/sim images transfers real tactile images to simulated ones with sufficient fidelity for the policy.
    The paper notes this failed for curved surfaces and light contacts (Sec. IV-C3, V-C1), so this is a fragile assumption affecting zero-shot transfer.
  • domain assumption PyBullet with modelled TacTip sensors accurately simulates contact dynamics needed for policy training.
    The policy is trained entirely in simulation (Sec. IV-A). Any mismatch between simulated and real contact dynamics affects sim-to-real transfer.
  • domain assumption Uniformly sampled initial poses in the range [±20mm, ±20°] represent the distribution of NDF optimizer errors at test time.
    The paper acknowledges training does not use actual NDF outputs (Sec. IV-C2). If actual errors are larger or differently correlated, the refinement policy may fail.

pith-pipeline@v1.3.0-alltime-deepseek · 14057 in / 12080 out tokens · 107599 ms · 2026-08-04T08:26:09.479497+00:00 · methodology

0 comments
read the original abstract

Grasping accuracy is a critical prerequisite for precise object manipulation, often requiring careful alignment between the robot hand and object. Neural Descriptor Fields (NDF) offer a promising vision-based method to generate grasping poses that generalize across object categories. However, NDF alone can produce inaccurate poses due to imperfect camera calibration, incomplete point clouds, and object variability. Meanwhile, tactile sensing enables more precise contact, but existing approaches typically learn policies limited to simple, predefined contact geometries. In this work, we introduce NeuralTouch, a multimodal framework that integrates NDF and tactile sensing to enable accurate, generalizable grasping through gentle physical interaction. Our approach leverages NDF to implicitly represent the target contact geometry, from which a deep reinforcement learning (RL) policy is trained to refine the grasp using tactile feedback. This policy is conditioned on the neural descriptors and does not require explicit specification of contact types. We validate NeuralTouch through ablation studies in simulation and zero-shot transfer to real-world manipulation tasks--such as peg-out-in-hole and bottle lid opening--without additional fine-tuning. Results show that NeuralTouch significantly improves grasping accuracy and robustness over baseline methods, offering a general framework for precise, contact-rich robotic manipulation.

Figures

Figures reproduced from arXiv: 2510.20390 by Bowen Deng, Chenghua Lu, Efi Psomopoulou, Keju Pu, Max Yang, Nathan F. Lepora, Yijiong Lin.

Figure 1
Figure 1. Figure 1: With just a few demonstrations of a manipulation task in simulation, our method [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the NeuralTouch: In simulation, we first pre-train an occupancy network which is the core component of the Neural Pose Descriptor [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: The objects used in the two proposed manipulation tasks for real [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Examples of online adaptability of NeuralTouch RL policy with dynamic target changes demonstrated in grasping tasks. (a) The top row illustrates [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The snapshots of the robot performing four different tasks in simulation with three methods: (a) NeuralTouch, (b) NDF (first two rows) and [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: (a) Robot arm equipped with a tactile gripper performing two real-world manipulation tasks requiring high accuracy. Top 3 rows: bottle-lid opening. [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 10 linked inside Pith

  1. [1]

    Adelson, and Sergey Levine

    Roberto Calandra, Andrew Owens, Dinesh Jayaraman, Justin Lin, Wen- zhen Yuan, Jitendra Malik, Edward H. Adelson, and Sergey Levine. More than a feeling: Learning to grasp and regrasp using vision and touch.IEEE Robotics and Automation Letters, 3(4):3300–3307, 2018

    2018

  2. [2]

    Simple, a visuotactile method learned in simulation to precisely pick, localize, regrasp, and place objects.Science Robotics, 9(91):eadi8808, 2024

    Maria Bauza, Antonia Bronars, Yifan Hou, Ian Taylor, Nikhil Chavan- Dafle, and Alberto Rodriguez. Simple, a visuotactile method learned in simulation to precisely pick, localize, regrasp, and place objects.Science Robotics, 9(91):eadi8808, 2024

    2024

  3. [3]

    Planning visual-tactile precision grasps via complementary use of vision and touch.IEEE Robotics and Automation Letters, 8(2):768–775, 2023

    Martin Matak and Tucker Hermans. Planning visual-tactile precision grasps via complementary use of vision and touch.IEEE Robotics and Automation Letters, 8(2):768–775, 2023

    2023

  4. [4]

    Neural descriptor fields: Se (3)-equivariant object representations for manipu- lation

    Anthony Simeonov, Yilun Du, Andrea Tagliasacchi, Joshua B Tenen- baum, Alberto Rodriguez, Pulkit Agrawal, and Vincent Sitzmann. Neural descriptor fields: Se (3)-equivariant object representations for manipu- lation. In2022 International Conference on Robotics and Automation (ICRA), pages 6394–6400. IEEE, 2022

    2022

  5. [5]

    N. Lepora. Soft Biomimetic Optical Tactile Sensing With the TacTip: A Review.IEEE Sensors Journal, 21(19):21131–21143, October 2021

    2021

  6. [6]

    A. C. Abad and A. Ranasinghe. Visuotactile sensors with emphasis on gelsight sensor: A review.IEEE Sensors Journal, 20(14):7628–7638, July 2020

    2020

  7. [7]

    Chenghua Lu, Kailuan Tang, Max Yang, Tianqi Yue, Haoran Li, and Nathan F. Lepora. Dexitac: Soft dexterous tactile gripping.IEEE/ASME Transactions on Mechatronics, 30(1):333–344, 2025

    2025

  8. [8]

    Rotipbot: Robotic handling of thin and flexible objects using rotatable tactile sensors.IEEE Transactions on Robotics, pages 1–19, 2025

    Jiaqi Jiang, Xuyang Zhang, Daniel Fernandes Gomes, Thanh-Toan Do, and Shan Luo. Rotipbot: Robotic handling of thin and flexible objects using rotatable tactile sensors.IEEE Transactions on Robotics, pages 1–19, 2025

    2025

  9. [9]

    Representation reinforcement learning-based dense control for point following with state sparse sensing of 3-d snake robots

    Lixing Liu, Jiashun Liu, Xian Guo, Wei Huang, Yongchun Fang, and Jianye Hao. Representation reinforcement learning-based dense control for point following with state sparse sensing of 3-d snake robots. IEEE/ASME Transactions on Mechatronics, 30(2):851–861, 2025. 10

    2025

  10. [10]

    Anyrotate: Gravity-invariant in-hand object rotation with sim-to-real touch.arXiv preprint arXiv:2405.07391, 2024

    Max Yang, Chenghua Lu, Alex Church, Yijiong Lin, Chris Ford, Haoran Li, Efi Psomopoulou, David AW Barton, and Nathan F Lepora. Anyrotate: Gravity-invariant in-hand object rotation with sim-to-real touch.arXiv preprint arXiv:2405.07391, 2024

    Pith/arXiv arXiv 2024

  11. [11]

    Church, J

    A. Church, J. Lloyd, R. Hadsell, and N. F Lepora. Deep reinforcement learning for tactile robotics: Learning to type on a braille keyboard. IEEE Robotics and Automation Letters, 5(4):6145–6152, July 2019

    2019

  12. [12]

    Amadio, A

    F. Amadio, A. Colomé, and C. Torras. Exploiting symmetries in reinforcement learning of bimanual robotic tasks.IEEE Robotics and Automation Letters, 4(2):1838–1845, 2019

    2019

  13. [13]

    Where shall i touch? vision-guided tactile poking for transparent object grasping.IEEE/ASME Transactions on Mechatronics, 28(1):233– 244, 2023

    Jiaqi Jiang, Guanqun Cao, Aaron Butterworth, Thanh-Toan Do, and Shan Luo. Where shall i touch? vision-guided tactile poking for transparent object grasping.IEEE/ASME Transactions on Mechatronics, 28(1):233– 244, 2023

    2023

  14. [14]

    W. Yuan, S. Dong, and E. H. Adelson. Gelsight: High-resolution robot tactile sensors for estimating geometry and force.Sensors, 17(12):2762, November 2017

    2017

  15. [15]

    Learning generaliz- able vision-tactile robotic grasping strategy for deformable objects via transformer.IEEE/ASME Transactions on Mechatronics, 30(1):554–566, 2025

    Yunhai Han, Kelin Yu, Rahul Batra, Nathan Boyd, Chaitanya Mehta, Tuo Zhao, Yu She, Seth Hutchinson, and Ye Zhao. Learning generaliz- able vision-tactile robotic grasping strategy for deformable objects via transformer.IEEE/ASME Transactions on Mechatronics, 30(1):554–566, 2025

    2025

  16. [16]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017

    2017

  17. [17]

    Lepora and J

    N. Lepora and J. Lloyd. Pose-Based Tactile Servoing: Controlled Soft Touch Using Deep Learning.IEEE Robotics & Automation Magazine, 28(4):43–55, December 2021

    2021

  18. [18]

    Lepora, Alex Church, Conrad de Kerckhove, Raia Hadsell, and John Lloyd

    Nathan F. Lepora, Alex Church, Conrad de Kerckhove, Raia Hadsell, and John Lloyd. From pixels to percepts: Highly robust edge perception and contour following using deep learning and an optical biomimetic tactile sensor.IEEE Robotics and Automation Letters, 4(2):2101–2107, 2019

    2019

  19. [19]

    N. F. Lepora, Y . Lin, B. Money-Coomes, and J. Lloyd. Digitac: A digit- tactip hybrid tactile sensor for comparing low-cost high-resolution robot touch.IEEE Robotics and Automation Letters, 7(4):9382–9388, 2022

    2022

  20. [20]

    Lloyd and N

    J. Lloyd and N. Lepora. Goal-Driven Robotic Pushing Using Tactile and Proprioceptive Feedback.IEEE Transactions on Robotics, 38(2):1201– 1212, April 2022

    2022

  21. [21]

    John Lloyd and Nathan F. Lepora. Pose-and-shear-based tac- tile servoing.The International Journal of Robotics Research, 0(0):02783649231225811, 0

  22. [22]

    Cable routing and assembly using tactile-driven motion primitives.arXiv preprint arXiv:2303.11765, 2023

    Achu Wilson, Helen Jiang, Wenzhao Lian, and Wenzhen Yuan. Cable routing and assembly using tactile-driven motion primitives.arXiv preprint arXiv:2303.11765, 2023

    Pith/arXiv arXiv 2023

  23. [23]

    Cable manipulation with a tactile-reactive gripper

    Yu She, Shaoxiong Wang, Siyuan Dong, Neha Sunil, Alberto Rodriguez, and Edward Adelson. Cable manipulation with a tactile-reactive gripper. The International Journal of Robotics Research, 40(12-14):1385–1401, 2021

    2021

  24. [24]

    Skill generalization of tubular object manipulation with tactile sensing and sim2real learning.Robotics and Autonomous Systems, 160:104321, 2023

    Yongqiang Zhao, Xingshuo Jing, Kun Qian, Daniel Fernandes Gomes, and Shan Luo. Skill generalization of tubular object manipulation with tactile sensing and sim2real learning.Robotics and Autonomous Systems, 160:104321, 2023

    2023

  25. [25]

    Visual-tactile multimodal- ity for following deformable linear objects using reinforcement learning

    Leszek Pecyna, Siyuan Dong, and Shan Luo. Visual-tactile multimodal- ity for following deformable linear objects using reinforcement learning. In2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3987–3994. IEEE, 2022

    2022

  26. [26]

    Church, J

    A. Church, J. Lloyd, R. Hadsell, and N. Lepora. Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation. InProceedings of the 5th Conference on Robot Learning, pages 1–9. PMLR, October 2021

    2021

  27. [27]

    Y . Lin, J. Lloyd, A. Church, and N. Lepora. Tactile gym 2.0: Sim-to- real deep reinforcement learning for comparing low-cost high-resolution robot touch. volume 7 ofProceedings of Machine Learning Research, pages 10754–10761. IEEE, August 2022

    2022

  28. [28]

    Yijiong Lin, Alex Church, Max Yang, Haoran Li, John Lloyd, Dandan Zhang, and Nathan F. Lepora. Bi-touch: Bimanual tactile manipulation with sim-to-real deep reinforcement learning.IEEE Robotics and Automation Letters, 8(9):5472–5479, 2023

    2023

  29. [29]

    Neural fields in visual computing and beyond

    Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, and Srinath Sridhar. Neural fields in visual computing and beyond. In Computer Graphics Forum, volume 41, pages 641–676. Wiley Online Library, 2022

    2022

  30. [30]

    Touchsdf: A deepsdf approach for 3d shape reconstruction using vision-based tactile sensing.IEEE Robotics and Automation Letters, 2024

    Mauro Comi, Yijiong Lin, Alex Church, Alessio Tonioni, Laurence Aitchison, and Nathan F Lepora. Touchsdf: A deepsdf approach for 3d shape reconstruction using vision-based tactile sensing.IEEE Robotics and Automation Letters, 2024

    2024

  31. [31]

    Snap-it, tap-it, splat-it: Tactile-informed 3d gaussian splatting for reconstructing challenging surfaces.arXiv preprint arXiv:2403.20275, 2024

    Mauro Comi, Alessio Tonioni, Max Yang, Jonathan Tremblay, Valts Blukis, Yijiong Lin, Nathan F Lepora, and Laurence Aitchison. Snap-it, tap-it, splat-it: Tactile-informed 3d gaussian splatting for reconstructing challenging surfaces.arXiv preprint arXiv:2403.20275, 2024

    Pith/arXiv arXiv 2024

  32. [32]

    Se (3)-equivariant relational rearrangement with neural descriptor fields

    Anthony Simeonov, Yilun Du, Yen-Chen Lin, Alberto Rodriguez Garcia, Leslie Pack Kaelbling, Tomás Lozano-Pérez, and Pulkit Agrawal. Se (3)-equivariant relational rearrangement with neural descriptor fields. In Conference on Robot Learning, pages 835–846. PMLR, 2023

    2023

  33. [33]

    Shelving, stacking, hanging: Relational pose diffusion for multi-modal rearrangement.arXiv preprint arXiv:2307.04751, 2023

    Anthony Simeonov, Ankit Goyal, Lucas Manuelli, Lin Yen-Chen, Alina Sarmiento, Alberto Rodriguez, Pulkit Agrawal, and Dieter Fox. Shelving, stacking, hanging: Relational pose diffusion for multi-modal rearrangement.arXiv preprint arXiv:2307.04751, 2023

    Pith/arXiv arXiv 2023

  34. [34]

    Local neural descriptor fields: Locally conditioned object representations for manipulation

    Ethan Chun, Yilun Du, Anthony Simeonov, Tomas Lozano-Perez, and Leslie Kaelbling. Local neural descriptor fields: Locally conditioned object representations for manipulation. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 1830–1836. IEEE, 2023

    2023

  35. [35]

    D3fields: Dynamic 3d descriptor fields for zero-shot generalizable robotic manipulation.arXiv preprint arXiv:2309.16118, 2023

    Yixuan Wang, Zhuoran Li, Mingtong Zhang, Katherine Driggs- Campbell, Jiajun Wu, Li Fei-Fei, and Yunzhu Li. D3fields: Dynamic 3d descriptor fields for zero-shot generalizable robotic manipulation.arXiv preprint arXiv:2309.16118, 2023

    Pith/arXiv arXiv 2023

  36. [36]

    Center-of-mass- based object regrasping: A reinforcement learning approach and the ef- fects of perception modality.IEEE/ASME Transactions on Mechatronics, 30(2):1356–1365, 2025

    Renpeng Wang, Yu Xie, Houde Liu, and Wei Zhou. Center-of-mass- based object regrasping: A reinforcement learning approach and the ef- fects of perception modality.IEEE/ASME Transactions on Mechatronics, 30(2):1356–1365, 2025

    2025

  37. [37]

    Tactile-based insertion for dense box-packing

    Siyuan Dong and Alberto Rodriguez. Tactile-based insertion for dense box-packing. In2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7953–7960. IEEE, 2019

    2019

  38. [38]

    Jha, Diego Romeres, Sangwoon Kim, Daniel Nikovski, and Alberto Rodriguez

    Siyuan Dong, Devesh K. Jha, Diego Romeres, Sangwoon Kim, Daniel Nikovski, and Alberto Rodriguez. Tactile-rl for insertion: Generalization to objects of unknown geometry. In2021 IEEE International Conference on Robotics and Automation (ICRA), pages 6437–6443, 2021

    2021

  39. [39]

    Sims- hear: Sim-to-real shear-based tactile servoing.arXiv preprint arXiv:2508.20561, 2025

    Kipp McAdam Freud, Yijiong Lin, and Nathan F Lepora. Sims- hear: Sim-to-real shear-based tactile servoing.arXiv preprint arXiv:2508.20561, 2025

    Pith/arXiv arXiv 2025

  40. [40]

    Perceiving extrinsic contacts from touch improves learning insertion policies.arXiv preprint arXiv:2309.16652, 2023

    Carolina Higuera, Joseph Ortiz, Haozhi Qi, Luis Pineda, Byron Boots, and Mustafa Mukadam. Perceiving extrinsic contacts from touch improves learning insertion policies.arXiv preprint arXiv:2309.16652, 2023

    Pith/arXiv arXiv 2023

  41. [41]

    Tac2pose: Tactile object pose estimation from the first touch.The International Journal of Robotics Research, 42(13):1185–1209, 2023

    Maria Bauza, Antonia Bronars, and Alberto Rodriguez. Tac2pose: Tactile object pose estimation from the first touch.The International Journal of Robotics Research, 42(13):1185–1209, 2023

    2023

  42. [42]

    Tactile object pose estimation from the first touch with geometric contact rendering

    Maria Bauza Villalonga, Alberto Rodriguez, Bryan Lim, Eric Valls, and Theo Sechopoulos. Tactile object pose estimation from the first touch with geometric contact rendering. InConference on Robot Learning, pages 1015–1029. PMLR, 2021

    2021

  43. [43]

    Coarse-to-fine imitation learning: Robot manipulation from a single demonstration

    Edward Johns. Coarse-to-fine imitation learning: Robot manipulation from a single demonstration. In2021 IEEE International Conference on Robotics and Automation (ICRA), pages 4613–4619, 2021

    2021

  44. [44]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space.Advances in neural information processing systems, 30, 2017

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space.Advances in neural information processing systems, 30, 2017

    2017

  45. [45]

    Vector neurons: A general framework for so (3)-equivariant networks

    Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas J Guibas. Vector neurons: A general framework for so (3)-equivariant networks. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 12200– 12209, 2021

    2021

  46. [46]

    Occupancy networks: Learning 3d reconstruction in function space

    Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4460– 4470, 2019

    2019

  47. [47]

    Optimal deep learning for robot touch: Training accurate pose models of 3d surfaces and edges.IEEE Robotics & Automation Magazine, 27(2):66–77, 2020

    Nathan F Lepora and John Lloyd. Optimal deep learning for robot touch: Training accurate pose models of 3d surfaces and edges.IEEE Robotics & Automation Magazine, 27(2):66–77, 2020

    2020

  48. [48]

    Pybullet, a python module for physics simulation for games, robotics and machine learning, 2016

    Erwin Coumans and Yunfei Bai. Pybullet, a python module for physics simulation for games, robotics and machine learning, 2016

    2016

  49. [49]

    Fots: A fast optical tactile simulator for sim2real learning of tactile-motor robot manipulation skills.IEEE Robotics and Automation Letters, 2024

    Yongqiang Zhao, Kun Qian, Boyi Duan, and Shan Luo. Fots: A fast optical tactile simulator for sim2real learning of tactile-motor robot manipulation skills.IEEE Robotics and Automation Letters, 2024

    2024

  50. [50]

    Efficient tactile simulation with differentiability for robotic manipulation

    Jie Xu, Sangwoon Kim, Tao Chen, Alberto Rodriguez Garcia, Pulkit Agrawal, Wojciech Matusik, and Shinjiro Sueda. Efficient tactile simulation with differentiability for robotic manipulation. In6th Annual Conference on Robot Learning, 2022

    2022

  51. [51]

    Neural contact fields: Tracking extrinsic contact with tactile sensing

    Carolina Higuera, Siyuan Dong, Byron Boots, and Mustafa Mukadam. Neural contact fields: Tracking extrinsic contact with tactile sensing. 11 In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 12576–12582, 2023

    2023

  52. [52]

    Active extrinsic contact sensing: Application to general peg-in-hole insertion

    Sangwoon Kim and Alberto Rodriguez. Active extrinsic contact sensing: Application to general peg-in-hole insertion. In2022 International Conference on Robotics and Automation (ICRA), pages 10241–10247, 2022

    2022

  53. [53]

    Shapenet: An information-rich 3d model repository.arXiv preprint arXiv:1512.03012, 2015

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository.arXiv preprint arXiv:1512.03012, 2015

    Pith/arXiv arXiv 2015

  54. [54]

    Schulman, F

    J. Schulman, F. Wolski, A. Radford, and O. Klimov. Proximal policy optimization algorithms.arXiv preprint:1707.06347, 2017

    Pith/arXiv arXiv 2017

  55. [55]

    Raffin, A

    A. Raffin, A. Hill, M. Ernestus, A. Gleave, A. Kanervisto, and N. Dor- mann. Stable baselines3, 2019

    2019

  56. [56]

    Isola, J

    P. Isola, J. Y . Zhu, T. Zhou, and A. A Efros. Image-to-image translation with conditional adversarial networks. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1125–1134, July 2017

    2017

  57. [57]

    Probabilistic discrim- inative models address the tactile perceptual aliasing problem.arXiv preprint arXiv:2106.02125, 2021

    John Lloyd, Yijiong Lin, and Nathan F Lepora. Probabilistic discrim- inative models address the tactile perceptual aliasing problem.arXiv preprint arXiv:2106.02125, 2021

    Pith/arXiv arXiv 2021