SABER provides 44.8K multi-representation action samples from unscripted retail environments that raise a VLA model's mean success rate on ten manipulation tasks from 13.4% to 29.3%.
Robotic telekinesis: Learning a robotic hand imita- tor by watching humans on youtube
11 Pith papers cite this work. Polarity classification is still indexing.
11
Pith papers citing it
abstract
We build a system that enables any human to control a robot hand and arm, simply by demonstrating motions with their own hand. The robot observes the human operator via a single RGB camera and imitates their actions in real-time. Human hands and robot hands differ in shape, size, and joint structure, and performing this translation from a single uncalibrated camera is a highly underconstrained problem. Moreover, the retargeted trajectories must effectively execute tasks on a physical robot, which requires them to be temporally smooth and free of self-collisions. Our key insight is that while paired human-robot correspondence data is expensive to collect, the internet contains a massive corpus of rich and diverse human hand videos. We leverage this data to train a system that understands human hands and retargets a human video stream into a robot hand-arm trajectory that is smooth, swift, safe, and semantically similar to the guiding demonstration. We demonstrate that it enables previously untrained people to teleoperate a robot on various dexterous manipulation tasks. Our low-cost, glove-free, marker-free remote teleoperation system makes robot teaching more accessible and we hope that it can aid robots in learning to act autonomously in the real world. Videos at https://robotic-telekinesis.github.io/
citation-role summary
background 3
citation-polarity summary
fields
cs.RO 11roles
background 3polarities
background 3representative citing papers
POMDAR is a taxonomy-grounded benchmark that quantifies dexterity as task throughput across vertical, horizontal, rotation, and grasping configurations with mechanical constraints for unambiguous measurement.
Task-agnostic RL play pretraining on diverse objects yields a reusable dexterous prior that makes sparse-reward assembly learning ~33× more sample-efficient and enables zero-shot sim-to-real transfer on tight insertions and multi-part tasks.
Video2Sim2Real turns a single human video into a deployable robot manipulation skill by reconstructing a digital twin, anchoring motions to object-centric simulator configurations, and bridging sim-to-real gaps with imitation learning and residual RL.
SERNF fine-tunes dexterous manipulation policies on real hardware by pairing normalizing-flow policies with action-chunked critics and conservative off-policy RL.
RIGVid shows that filtered AI-generated videos can serve as effective supervision for complex robotic manipulation tasks without any real demonstrations.
Sampling-Based Retargeter (SBR) delivers lower-jitter real-time kinematic hand retargeting and higher task success with less operator fatigue than gradient-based baselines in an 18-person study.
A dual-arm teleoperation system combines a graph-based motion retargeting network with VLM-informed MPC force control to achieve cross-platform motion mapping and adaptive grasping across multiple robots and objects.
ZeroDex grounds VLM outputs into 3D keypoints via multi-view triangulation and ray voting to enable zero-shot long-horizon dexterous manipulation with closed-loop replanning.
TopoRetarget uses a sparse interaction graph and distance-weighted Laplacian deformation optimization with kinematic and penetration constraints to retarget human demonstrations to dexterous hands while preserving task-relevant contacts.
A structured survey of dexterous robotic hand research that reviews hardware, control methods, data resources, and benchmarks while identifying major limitations and future directions.
citing papers explorer
-
SABER: A Scalable Action-Based Embodied Dataset for Real-World VLA Adaptation
SABER provides 44.8K multi-representation action samples from unscripted retail environments that raise a VLA model's mean success rate on ten manipulation tasks from 13.4% to 29.3%.
-
A Benchmark of Dexterity for Anthropomorphic Robotic Hands
POMDAR is a taxonomy-grounded benchmark that quantifies dexterity as task throughput across vertical, horizontal, rotation, and grasping configurations with mechanical constraints for unambiguous measurement.
-
Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?
Task-agnostic RL play pretraining on diverse objects yields a reusable dexterous prior that makes sparse-reward assembly learning ~33× more sample-efficient and enables zero-shot sim-to-real transfer on tight insertions and multi-part tasks.
-
Video2Sim2Real: Full-Stack Autonomous Dexterous Skill Acquisition from a Single Human Video
Video2Sim2Real turns a single human video into a deployable robot manipulation skill by reconstructing a digital twin, anchoring motions to object-centric simulator configurations, and bridging sim-to-real gaps with imitation learning and residual RL.
-
SERNF: Sample-Efficient Real-World Dexterous Policy Fine-Tuning via Action-Chunked Critics and Normalizing Flows
SERNF fine-tunes dexterous manipulation policies on real hardware by pairing normalizing-flow policies with action-chunked critics and conservative off-policy RL.
-
Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrations
RIGVid shows that filtered AI-generated videos can serve as effective supervision for complex robotic manipulation tasks without any real demonstrations.
-
Smooth Operator: A Real-Time Sampling-Based Algorithm for Kinematic Hand Retargeting
Sampling-Based Retargeter (SBR) delivers lower-jitter real-time kinematic hand retargeting and higher task success with less operator fatigue than gradient-based baselines in an 18-person study.
-
DexTele: A Dual-Arm Dexterous Teleoperation System Based on Motion Retargeting and Adaptive Force Control
A dual-arm teleoperation system combines a graph-based motion retargeting network with VLM-informed MPC force control to achieve cross-platform motion mapping and adaptive grasping across multiple robots and objects.
-
ZeroDex: Zero-Shot Long-Horizon Dexterous Manipulation via Multi-View 3D-Grounded VLM Reasoning
ZeroDex grounds VLM outputs into 3D keypoints via multi-view triangulation and ray voting to enable zero-shot long-horizon dexterous manipulation with closed-loop replanning.
-
TopoRetarget: Interaction-Preserving Retargeting for Dexterous Manipulation
TopoRetarget uses a sparse interaction graph and distance-weighted Laplacian deformation optimization with kinematic and penetration constraints to retarget human demonstrations to dexterous hands while preserving task-relevant contacts.
-
Towards Robotic Dexterous Hand Intelligence: A Survey
A structured survey of dexterous robotic hand research that reviews hardware, control methods, data resources, and benchmarks while identifying major limitations and future directions.