REVIEW 12 cited by
Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Teaching dexterity to multi-fingered robots has been a longstanding challenge in robotics. Most prominent work in this area focuses on learning controllers or policies that either operate on visual observations or state estimates derived from vision. However, such methods perform poorly on fine-grained manipulation tasks that require reasoning about contact forces or about objects occluded by the hand itself. In this work, we present T-Dex, a new approach for tactile-based dexterity, that operates in two phases. In the first phase, we collect 2.5 hours of play data, which is used to train self-supervised tactile encoders. This is necessary to bring high-dimensional tactile readings to a lower-dimensional embedding. In the second phase, given a handful of demonstrations for a dexterous task, we learn non-parametric policies that combine the tactile observations with visual ones. Across five challenging dexterous tasks, we show that our tactile-based dexterity models outperform purely vision and torque-based models by an average of 1.7X. Finally, we provide a detailed analysis on factors critical to T-Dex including the importance of play data, architectures, and representation learning.
Forward citations
Cited by 12 Pith papers
-
LocoTouch: Learning Dynamic Quadrupedal Transport with Tactile Sensing
LocoTouch trains a quadrupedal policy that uses a 221-taxel tactile back to balance and transport unsecured cylindrical objects, transferring zero-shot to a real Unitree Go1.
-
Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills
A taxonomy of robot learning on a weights-versus-skills axis, with a five-rung self-improvement ladder whose top cell (feedback plus memory plus search) holds only a few recent systems.
-
FA-RDP: A Frequency-Adaptive Reactive Diffusion Policy for Contact-Rich Manipulation
A shared visual-force diffusion policy with a multimodality indicator and manifold consistency distillation raises contact-rich task success to 81.7% while keeping diverse pre-contact modes.
-
OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies
A policy-agnostic two-stage real-world RL method learns tactile residual corrections on frozen visual policies, lifting contact-rich task success from 5–40% to 85–100% in under 80 minutes.
-
Self-Supervised Multisensory Pretraining for Contact-Rich Robot Reinforcement Learning
MSDP pre-trains a transformer encoder with masked multisensory autoencoding, then uses an asymmetric actor-critic bridge (cross-attention for critic, pooling for actor) to accelerate and robustify contact-rich RL acro...
-
Touch begins where vision ends: Generalizable policies for contact-rich manipulation
A localize-then-execute policy that combines vision-language reaching, semantic background augmentation, and residual reinforcement learning with tactile sensing reaches about 90% success on millimeter-precision manip...
-
Self-supervised perception for tactile skin covered dexterous hands
A self-supervised encoder pretrained on four hours of hand-object play produces tactile representations that improve downstream robot touch tasks by over 40% versus prior methods.
-
Adaptive Wiping: Adaptive contact-rich manipulation through few-shot imitation learning with Force-Torque feedback and pre-trained object representations
A closed-loop force-torque feedback extension of few-shot imitation learning lets a robot adapt its wiping force to unseen surface heights and sponge properties, reaching 96% of reference force in real-world tests.
-
Robust Model-Based In-Hand Manipulation with Integrated Real-Time Motion-Contact Planning and Tracking
A hierarchical, model-based, contact-implicit controller with tactile tracking completes five real-world in-hand manipulation tasks without any learning, under disturbances.
-
CordViP: Correspondence-based Visuomotor Policy for Dexterous Manipulation in Real-World
CordViP achieves strong real-world dexterous manipulation by feeding a diffusion policy with pose-tracked 3D object models and hand point clouds, pretrained on contact maps and arm-hand coordination.
-
AdaDexGrasp: Adaptive Dexterous Grasping via 3D Visuo-Tactile Representation Fusion
AdaDexGrasp learns to fuse point clouds with finger-level tactile labels to generate, judge, and correct dexterous grasps, reporting 91%/82%/83% success on seen, unseen-object, and unseen-category sets in simulation.
-
Detecting Reading-Induced Confusion Using EEG and Eye Tracking
Multimodal EEG plus eye tracking classifies reading-induced confusion at 77.3% average weighted accuracy, beating unimodal models by 4-22%, in an 11-participant study.
Discussion (0). Continue with ORCID to comment.