REVIEW 8 cited by
Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtube
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We build a system that enables any human to control a robot hand and arm, simply by demonstrating motions with their own hand. The robot observes the human operator via a single RGB camera and imitates their actions in real-time. Human hands and robot hands differ in shape, size, and joint structure, and performing this translation from a single uncalibrated camera is a highly underconstrained problem. Moreover, the retargeted trajectories must effectively execute tasks on a physical robot, which requires them to be temporally smooth and free of self-collisions. Our key insight is that while paired human-robot correspondence data is expensive to collect, the internet contains a massive corpus of rich and diverse human hand videos. We leverage this data to train a system that understands human hands and retargets a human video stream into a robot hand-arm trajectory that is smooth, swift, safe, and semantically similar to the guiding demonstration. We demonstrate that it enables previously untrained people to teleoperate a robot on various dexterous manipulation tasks. Our low-cost, glove-free, marker-free remote teleoperation system makes robot teaching more accessible and we hope that it can aid robots in learning to act autonomously in the real world. Videos at https://robotic-telekinesis.github.io/
Forward citations
Cited by 8 Pith papers
-
Teleopit: A Full-Embodiment Humanoid Teleoperation System
Teleopit combines VR body, hand, and head tracking with a learned whole-body tracker and a cross-hand retargeter, and teleop-collected demos train ACT and GR00T policies to around 90 to 95 percent success on a humanoi...
-
DexDirect: Direct Kinesthetic Arm Guidance for Efficient Dexterous Demonstration Collection
A hybrid kinesthetic-arm-plus-webcam-hand teleoperation interface achieved 17x/3x higher demonstration throughput than vision baselines and trained a 90%-success pick-and-place policy in a ten-person study.
-
Smooth Operator: A Real-Time Sampling-Based Algorithm for Kinematic Hand Retargeting
Sampling-Based Retargeter (SBR) delivers lower-jitter real-time kinematic hand retargeting and higher task success with less operator fatigue than gradient-based baselines in an 18-person study.
-
Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?
Task-agnostic RL play pretraining on diverse objects yields a reusable dexterous prior that makes sparse-reward assembly learning ~33× more sample-efficient and enables zero-shot sim-to-real transfer on tight insertio...
-
SERNF: Sample-Efficient Real-World Dexterous Policy Fine-Tuning via Action-Chunked Critics and Normalizing Flows
SERNF fine-tunes dexterous manipulation policies on real hardware by pairing normalizing-flow policies with action-chunked critics and conservative off-policy RL.
-
DexTele: A Dual-Arm Dexterous Teleoperation System Based on Motion Retargeting and Adaptive Force Control
A dual-arm teleoperation system combines a graph-based motion retargeting network with VLM-informed MPC force control to achieve cross-platform motion mapping and adaptive grasping across multiple robots and objects.
-
ObjRetarget: An Object-Aware Motion Retargeting Framework with Anthropomorphic Arm Constraints and Polyhedral Hand Modeling
Decoupled arm–hand retargeting with anthropomorphic arm-plane constraints and polyhedral contact invariants raises real-robot dexterous-task success to 75.8% versus 61.6% and 50.8% for OKAMI and ORION.
-
CRAFT: A Tendon-Driven Hand with Hybrid Hard-Soft Compliance
Hybrid hard-soft tendon hand with rolling joints improves strength and fragile teleop over a rigid baseline, covers 33/33 Feix grasps, and costs under $600 open-source.
Discussion (0). Sign in to comment.