REVIEW 4 cited by
Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing
read the original abstract
Executing contact-rich manipulation tasks necessitates the fusion of tactile and visual feedback. However, the distinct nature of these modalities poses significant challenges. In this paper, we introduce a system that leverages visual and tactile sensory inputs to enable dexterous in-hand manipulation. Specifically, we propose Robot Synesthesia, a novel point cloud-based tactile representation inspired by human tactile-visual synesthesia. This approach allows for the simultaneous and seamless integration of both sensory inputs, offering richer spatial information and facilitating better reasoning about robot actions. The method, trained in a simulated environment and then deployed to a real robot, is applicable to various in-hand object rotation tasks. Comprehensive ablations are performed on how the integration of vision and touch can improve reinforcement learning and Sim2Real performance. Our project page is available at https://yingyuan0414.github.io/visuotactile/ .
Forward citations
Cited by 4 Pith papers
-
Multi-Modal Manipulation via Multi-Modal Policy Consensus
A policy that factorizes into modality-specific diffusion models combined by a learned router network for adaptive multi-modal robotic manipulation.
-
Closing the Reality Gap: Zero-Shot Sim-to-Real Deployment for Dexterous Force-Based Grasping and Manipulation
Zero-shot sim-to-real RL policies on a five-finger hand achieve commandable grasp-force tracking and in-hand reorientation using dense tactile simulation, current-to-torque calibration, and actuator randomization.
-
Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learning
Real2Sim tactile calibration, layout-aware encoder pretraining, and diffusion policy aggregation from object-specific RL experts enable 27% real-world success in blind grasping on a LEAP Hand for 10 seen and 10 unseen...
-
FlexiTac: A Low-Cost, Open-Source, Scalable Tactile Sensing Solution for Robotic Systems
FlexiTac is a scalable piezoresistive tactile sensing system with flexible FPC-Velostat-FPC pads and a 100 Hz multi-channel readout board that mounts on rigid or soft grippers and supports visuo-tactile learning.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.