REVIEW 27 cited by
3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing
read the original abstract
Tactile and visual perception are both crucial for humans to perform fine-grained interactions with their environment. Developing similar multi-modal sensing capabilities for robots can significantly enhance and expand their manipulation skills. This paper introduces \textbf{3D-ViTac}, a multi-modal sensing and learning system designed for dexterous bimanual manipulation. Our system features tactile sensors equipped with dense sensing units, each covering an area of 3$mm^2$. These sensors are low-cost and flexible, providing detailed and extensive coverage of physical contacts, effectively complementing visual information. To integrate tactile and visual data, we fuse them into a unified 3D representation space that preserves their 3D structures and spatial relationships. The multi-modal representation can then be coupled with diffusion policies for imitation learning. Through concrete hardware experiments, we demonstrate that even low-cost robots can perform precise manipulations and significantly outperform vision-only policies, particularly in safe interactions with fragile items and executing long-horizon tasks involving in-hand manipulation. Our project page is available at \url{https://binghao-huang.github.io/3D-ViTac/}.
Forward citations
Cited by 27 Pith papers
-
FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-Rich Manipulation
FTP-1 is the first foundation tactile policy pretrained on ~3000 hours of data from 26 sources across 21 sensors that improves performance on seen setups by 17.2% and transfers to unseen sensors with 31% success rate gain.
-
HapTile: A Haptic-Informed Vision-Tactile-Language-Action Dataset for Contact-Rich Imitation Learning
HapTile introduces a visuotactile dataset with haptic-informed teleoperation for language-conditioned contact-rich manipulation tasks and provides baseline policy benchmarks.
-
Force Policy: Learning Hybrid Force-Position Control Policy under Interaction Frame for Contact-Rich Manipulation
Force Policy learns a global vision policy for free space and a local force-feedback policy that recovers an interaction frame to execute stable hybrid force-position control in contact-rich manipulation.
-
TouchGuide: Inference-Time Steering of Visuomotor Policies via Touch Guidance
TouchGuide improves contact-rich robot manipulation by steering diffusion or flow-matching visuomotor policies with tactile feasibility scores from a contrastively trained Contact Physical Model.
-
FA-RDP: A Frequency-Adaptive Reactive Diffusion Policy for Contact-Rich Manipulation
A shared visual-force diffusion policy with a multimodality indicator and manifold consistency distillation raises contact-rich task success to 81.7% while keeping diverse pre-contact modes.
-
TactiDex: A Real-World Tactile-Guided Benchmark for Human-Like Dexterous Manipulation
A tactile-rich HOI dataset plus a tri-component force reward improves contact fidelity and success of human-to-robot dexterous transfer over kinematic imitation alone.
-
WristMimic: Full-Body Humanoid Control with Wrist-Guided Manipulation
Wrist-guided whole-body RL retargets human–object interactions without finger pose supervision, matching supervised methods and generalizing across hand morphologies in simulation.
-
WristMimic: Full-Body Humanoid Control with Wrist-Guided Manipulation
WristMimic achieves comparable or superior object manipulation retargeting by supervising wrist kinematics while letting finger behavior emerge from object and contact dynamics.
-
SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects
Success-only metrics overstate deformable-manipulation performance; tactile sensing raises Safety Success (e.g. 21.4%→35.6% on Object-Soft) while Goal Success stays comparable.
-
OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies
A policy-agnostic two-stage real-world RL method learns tactile residual corrections on frozen visual policies, lifting contact-rich task success from 5–40% to 85–100% in under 80 minutes.
-
Current as Touch: Proprioceptive Contact Feedback for Compliant Dexterous Manipulation
Motor current plus joint state predicts compliance reference positions that let standard PD control produce stable, contact-aware grasping without external tactile or force sensors.
-
TACO: TActile World Model as a Self-COrrector forScalable VLA Post-Training
A tactile-aware world model recognizes failure-adjacent contact states, imagines local visuo-tactile corrections, and post-trains VLAs with knowledge insulation, raising average success by 44% over the base policy.
-
UniTacVLA: Unified Tactile Understanding and Prediction in Vision Language Action Models
UniTacVLA builds a state-aware and dynamics-aware tactile prior via unified latent space, tactile chain-of-thought, and mixed real/predicted feedback controller to boost dexterous manipulation performance.
-
TACTFUL: Tactile-Driven Exploration For Object Localization and Identification in Confined Environments
TACTFUL introduces a vision-free tactile policy for robotic exploration and object identification in confined workspaces, trained on real hardware and achieving 77% success with 0.015 m reconstruction error.
-
Tac-DINO: Learning Vision-Tactile Features with Patch Alignment
Tac-DINO constructs a large tactile dataset and Vis-Tac Holographic Matching Benchmark, then proposes Vision-Tactile Patch Alignment (VTPA) methods that outperform non-aligned baselines on local-to-global feature matching.
-
Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learning
Real2Sim tactile calibration, layout-aware encoder pretraining, and diffusion policy aggregation from object-specific RL experts enable 27% real-world success in blind grasping on a LEAP Hand for 10 seen and 10 unseen...
-
RGB-S: Image-Aligned Tactile Saliency for Robust Dexterous Manipulation
RGB-S projects tactile contacts onto images as force-modulated Gaussian saliency maps via kinematics and zero-initialized conditioning, raising real-world occluded dexterous manipulation success by 26.7 percentage poi...
-
RealDexUMI: A Wearable Universal Manipulation Interface for Dexterous Robot Learning
A wearable interface with a shared dexterous hand module enables retargeting-free teleoperation and matched data collection, yielding policies with 88.75% average success across eight real-robot tasks that generalize ...
-
TacO: Benchmarking Tactile Sensors for Object Manipulation
The paper provides a task-driven benchmark comparing visual, acoustic, magnetic, and resistive tactile sensors on three manipulation tasks and concludes that sensor utility depends on modality, material friction, and ...
-
TA-VLA: Elucidating the Design Space of Torque-aware Vision-Language-Action Models
Feeding torque history as a single decoder token and adding torque prediction as an auxiliary objective improves pretrained VLA success rates on contact-rich manipulation, with large gains on button pushing and charge...
-
Towards Trustworthy Embodied Intelligence: A Systems Framework and Graded Trustworthiness Levels
A four-layer systems framework and T0–T5 hierarchy for grading and maintaining bounded trustworthiness claims in embodied AI systems.
-
Seeing Touch from Motion: A Unified Modality-Aware Visuo-Tactile Policy with Tactile Motion Correlation
A visuo-tactile policy learning method that exploits tactile motion correlation for contact state distinction and Mixture-of-Transformers for cross-modal fusion.
-
HiPi: Reproducible High-Fidelity Piezoresistive Sensors for Robotic Manipulation
HiPi integrates a compact readout PCB, STM32 MCU, optimized comms, and FPCB layers to deliver 220 Hz readout on 2048-taxel bimanual arrays while raising contact-geometry IoU from 0.428 to 0.797 versus a reproducible baseline.
-
Learning Versatile Humanoid Manipulation with Touch Dreaming
HTD, a multimodal transformer policy trained with behavioral cloning and touch dreaming to predict future tactile latents, achieves a 90.9% relative success rate improvement over baselines on five real-world contact-r...
-
TacRefineNet: Goal-Conditioned Tactile Grasp Refinement for Edge-Prominent Objects
A robot hand uses fingertip pressure images to iteratively re-grasp thin objects, aligning them to a demonstrated target pose within a few millimeters using no vision.
-
Towards Robotic Dexterous Hand Intelligence: A Survey
A structured survey of dexterous robotic hand research that reviews hardware, control methods, data resources, and benchmarks while identifying major limitations and future directions.
-
Robotic Affection -- Opportunities of AI-based haptic interactions to improve social robotic touch through a multi-deep-learning approach
A position paper proposes decomposing affective robotic touch into multiple specialized deep learning models for better social human-robot interaction.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.