REVIEW 6 cited by
Holo-Dex: Teaching Dexterity with Immersive Mixed Reality
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Holo-Dex: Teaching Dexterity with Immersive Mixed Reality
read the original abstract
A fundamental challenge in teaching robots is to provide an effective interface for human teachers to demonstrate useful skills to a robot. This challenge is exacerbated in dexterous manipulation, where teaching high-dimensional, contact-rich behaviors often require esoteric teleoperation tools. In this work, we present Holo-Dex, a framework for dexterous manipulation that places a teacher in an immersive mixed reality through commodity VR headsets. The high-fidelity hand pose estimator onboard the headset is used to teleoperate the robot and collect demonstrations for a variety of general-purpose dexterous tasks. Given these demonstrations, we use powerful feature learning combined with non-parametric imitation to train dexterous skills. Our experiments on six common dexterous tasks, including in-hand rotation, spinning, and bottle opening, indicate that Holo-Dex can both collect high-quality demonstration data and train skills in a matter of hours. Finally, we find that our trained skills can exhibit generalization on objects not seen in training. Videos of Holo-Dex are available at https://holo-dex.github.io.
Forward citations
Cited by 6 Pith papers
-
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
Low-cost imprecise robots achieve 80-90% success on six fine bimanual manipulation tasks using imitation learning with a new Action Chunking with Transformers algorithm trained on only 10 minutes of demonstrations.
-
Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?
Task-agnostic RL play pretraining on diverse objects yields a reusable dexterous prior that makes sparse-reward assembly learning ~33× more sample-efficient and enables zero-shot sim-to-real transfer on tight insertio...
-
DexTwist: Dexterous Hand Retargeting for Twist Motion via Mixed Reality-based Teleoperation
DexTwist detects tripod pinches, estimates the intended screw axis and twist magnitude, then applies real-time joint refinement to track turning progress while stabilizing the robot's tripod geometry.
-
Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?
Play2Perfect uses task-agnostic RL play pretraining on diverse objects to build reusable manipulation priors, then fine-tunes for assembly, yielding 33x sample efficiency gains and 60% success on 0.5mm-clearance inser...
-
Manipulation Planning for Construction Activities with Repetitive Tasks
A robot generalizes one demonstration of placing a single brick to build walls of arbitrary length and layout by approximating motions as screw sequences and using ScLERP/RMRC for planning.
-
VR-DAgger: Immersive VR for Dexterous Data Collection and Uncertainty-Guided On-Policy Correction
VR-DAgger is a VR-centered human-in-the-loop framework that applies MC dropout uncertainty to select and correct failure segments in diffusion policy rollouts, yielding up to 23 percentage point gains over behavioral ...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.