REVIEW 33 cited by
AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System
read the original abstract
Vision-based teleoperation offers the possibility to endow robots with human-level intelligence to physically interact with the environment, while only requiring low-cost camera sensors. However, current vision-based teleoperation systems are designed and engineered towards a particular robot model and deploy environment, which scales poorly as the pool of the robot models expands and the variety of the operating environment increases. In this paper, we propose AnyTeleop, a unified and general teleoperation system to support multiple different arms, hands, realities, and camera configurations within a single system. Although being designed to provide great flexibility to the choice of simulators and real hardware, our system can still achieve great performance. For real-world experiments, AnyTeleop can outperform a previous system that was designed for a specific robot hardware with a higher success rate, using the same robot. For teleoperation in simulation, AnyTeleop leads to better imitation learning performance, compared with a previous system that is particularly designed for that simulator. Project page: https://yzqin.github.io/anyteleop/.
Forward citations
Cited by 33 Pith papers
-
TouchGuide: Inference-Time Steering of Visuomotor Policies via Touch Guidance
TouchGuide improves contact-rich robot manipulation by steering diffusion or flow-matching visuomotor policies with tactile feasibility scores from a contrastively trained Contact Physical Model.
-
Towards Human-level Dexterous Teleoperation
A single-stage RL co-tracking controller trained on consecutive human-derived hand–object subgoals achieves ~75% real-robot success on long-horizon dexterous teleoperation where baselines fail.
-
TactiDex: A Real-World Tactile-Guided Benchmark for Human-Like Dexterous Manipulation
A tactile-rich HOI dataset plus a tri-component force reward improves contact fidelity and success of human-to-robot dexterous transfer over kinematic imitation alone.
-
DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation
A modular benchmark of 100 dexterous manipulation tasks across 3 arms and 6 hands with 3,180 demonstrations reveals that current policies (Diffusion Policy, DP3, OpenVLA, π0.5) achieve only 34% mean success, exposing ...
-
Smooth Operator: A Real-Time Sampling-Based Algorithm for Kinematic Hand Retargeting
A sampling-based retargeter reduces hand teleoperation jitter and improves task success rates and operator workload compared to gradient-based baselines.
-
Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?
Task-agnostic RL play pretraining on diverse objects yields a reusable dexterous prior that makes sparse-reward assembly learning ~33× more sample-efficient and enables zero-shot sim-to-real transfer on tight insertio...
-
Do as I Do: Dexterous Manipulation Data from Everyday Human Videos
DO AS I DO reconstructs and retargets hand-object interactions from in-the-wild monocular RGB videos to produce dexterous robot manipulation trajectories, outperforming prior methods on ground-truth and online video datasets.
-
InDex: Empowering VLA Models with Intent-Conditioned Arm-Hand Coordination for Dexterous Manipulation
InDex adapts parallel-gripper VLA policies to dexterous hands by conditioning a diffusion hand policy on a normalized scalar grasp intent, raising average simulated task success from 50.3% to 85.8%.
-
Video2Sim2Real: Full-Stack Autonomous Dexterous Skill Acquisition from a Single Human Video
Video2Sim2Real turns a single human video into a deployable robot manipulation skill by reconstructing a digital twin, anchoring motions to object-centric simulator configurations, and bridging sim-to-real gaps with i...
-
LIMMT: Less is More for Motion Tracking
A data-centric approach shows that less than 3% of AMASS motion data, filtered by physics feasibility, diversity, and complexity, yields better humanoid tracking policies than the full dataset.
-
GraspGen-X: Cross-Embodiment 6-DOF Diffusion-based Grasping
GraspGen-X extends diffusion 6-DOF grasping to cross-embodiment via swept-volume gripper encoding, trained on procedural grippers and 2B grasps, claiming best zero-shot generalization to novel grippers in sim and real tests.
-
FingerViP: Learning Real-World Dexterous Manipulation with Fingertip Visual Perception
FingerViP equips each finger with a miniature camera and trains a multi-view diffusion policy that achieves 80.8% success on real-world dexterous tasks previously limited by wrist-camera occlusion.
-
WARPED: Wrist-Aligned Rendering for Robot Policy Learning from Egocentric Human Demonstrations
WARPED synthesizes realistic wrist-view observations from monocular egocentric human videos via foundation models, hand-object tracking, retargeting, and Gaussian Splatting to train visuomotor policies that match tele...
-
TeleGate: Whole-Body Humanoid Teleoperation via Gated Expert Selection with Motion Prior
TeleGate achieves high-precision real-time whole-body teleoperation of humanoid robots by dynamically gating between expert policies and using a VAE motion prior to infer future intent from history, outperforming dist...
-
Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulation
A 10,300-demonstration, 260-task multimodal humanoid manipulation dataset with baseline policy evaluations and a cloud evaluation platform.
-
3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations
DP3 uses compact 3D representations from sparse point clouds inside diffusion policies to learn generalizable visuomotor skills from few demonstrations, reporting 24% gains in simulation and 85% success on real robots.
-
Smooth Operator: A Real-Time Sampling-Based Algorithm for Kinematic Hand Retargeting
Sampling-Based Retargeter (SBR) delivers lower-jitter real-time kinematic hand retargeting and higher task success with less operator fatigue than gradient-based baselines in an 18-person study.
-
MEVION: Low-Cost Open-Source Data Collection System for Powerful and High-Speed Dual-Arm Manipulation
MEVION is an open-source dual-arm teleoperation platform with 60 Nm joint torque, built from e-commerce parts for about $14,000 per four-arm system, enabling heavier, faster manipulation data collection.
-
WARP: Whole-Body Retargeting for Learning from Offline Human Demonstrations
WARP is an offline retargeting method using a SEW geometric solver to produce consistent whole-body robot trajectories from human demonstrations for zero-shot mobile manipulation.
-
Not All Actions Are Equal: Rethinking Conditioning for Dexterous World Model
DexAC-WM improves FID, FVD, and PCK in high-DoF action-conditioned video prediction via structured action modeling and semantic grounding on EgoDex and EgoVerse.
-
Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?
Play2Perfect uses task-agnostic RL play pretraining on diverse objects to build reusable manipulation priors, then fine-tunes for assembly, yielding 33x sample efficiency gains and 60% success on 0.5mm-clearance inser...
-
Scalable Multi-Task Data Generation via Reinforcement Learning for Language-Conditioned Bimanual Dexterous Manipulation
An RL data generation pipeline with generalizable rewards and language annotations produces diverse synthetic datasets that improve multi-task policy generalization on three bimanual manipulation tasks.
-
TopoRetarget: Interaction-Preserving Retargeting for Dexterous Manipulation
TopoRetarget uses a sparse interaction graph and distance-weighted Laplacian deformation optimization with kinematic and penetration constraints to retarget human demonstrations to dexterous hands while preserving tas...
-
InDex: Empowering VLA Models with Intent-Conditioned Arm-Hand Coordination for Dexterous Manipulation
InDex adapts VLA models to high-DoF dexterous manipulation via intent-conditioned fine-tuning and a decoupled diffusion head, outperforming monolithic baselines in simulation tasks with minimal data.
-
PLUME: Probabilistic Latent Unified World Modeling and Parameter Estimation for Multi-Finger Manipulation
PLUME jointly models parameter beliefs and conditioned dynamics in a latent space for dexterous manipulation, enabling zero-shot sim-to-real transfer that outperforms offline RL and behavior cloning baselines on turni...
-
DexPIE: Stable Dexterous Policy Improvement from Real-World Experience
DexPIE improves dexterous manipulation success rates by 37% over demo policies via real-world experience collection with adapted intervention, multi-stage DAgger, asynchronous relative-action inference, and optimality...
-
Humanoid-GPT: Scaling Data and Structure for Zero-Shot Motion Tracking
Humanoid-GPT is a causal Transformer pre-trained on a unified billion-scale motion dataset that tracks dynamic behaviors with zero-shot generalization to unseen motions and tasks.
-
ConTrack: Constrained Hand Motion Tracking with Adaptive Trade-off Control
ConTrack introduces a constrained RL method with online dual-variable adaptation and adaptive resets for improved long-horizon hand tracking in simulation and on real robots.
-
AVI-HT: Adaptive Vision-IMU Fusion for 3D Hand Tracking
AVI-HT adaptively fuses vision and IMU data via attention to cut 3D hand keypoint error by 16.1% (24.2% wrist-aligned) on a new 100K+ sample DexGloveHOI dataset in occluded hand-object scenarios.
-
ImMimic: Cross-Domain Imitation from Human Videos via Mapping and Interpolation
A co-training framework that maps retargeted human hand trajectories to robot demonstrations with dynamic time warping and MixUp interpolation improves robot manipulation success rates and smoothness across four embodiments.
-
Scalable Multi-Task Data Generation via Reinforcement Learning for Language-Conditioned Bimanual Dexterous Manipulation
RL pipeline with generalizable rewards and domain randomization generates datasets that improve generalization of language-conditioned bimanual policies on three manipulation tasks.
-
EaDex: A Cross-Embodiment Dexterous Manipulation Framework from Low-Cost Demonstrations
EaDex combines single-camera RGB-D capture, MANO retargeting, and dynamic demonstration annealing to achieve 55.3% relative improvement over baseline on nine cross-embodiment dexterous object-opening tasks across three hands.
-
Robotic Affection -- Opportunities of AI-based haptic interactions to improve social robotic touch through a multi-deep-learning approach
A position paper proposes decomposing affective robotic touch into multiple specialized deep learning models for better social human-robot interaction.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.