REVIEW 7 cited by
ARCap: Collecting High-quality Human Demonstrations for Robot Learning with Augmented Reality Feedback
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent progress in imitation learning from human demonstrations has shown promising results in teaching robots manipulation skills. To further scale up training datasets, recent works start to use portable data collection devices without the need for physical robot hardware. However, due to the absence of on-robot feedback during data collection, the data quality depends heavily on user expertise, and many devices are limited to specific robot embodiments. We propose ARCap, a portable data collection system that provides visual feedback through augmented reality (AR) and haptic warnings to guide users in collecting high-quality demonstrations. Through extensive user studies, we show that ARCap enables novice users to collect robot-executable data that matches robot kinematics and avoids collisions with the scenes. With data collected from ARCap, robots can perform challenging tasks, such as manipulation in cluttered environments and long-horizon cross-embodiment manipulation. ARCap is fully open-source and easy to calibrate; all components are built from off-the-shelf products. More details and results can be found on our website: https://stanford-tml.github.io/ARCap
Forward citations
Cited by 7 Pith papers
-
HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alone
Robot-free HiFi-UMI demonstrations can replace teleoperated real-robot data in post-training: three policy backbones matched in-domain teleoperation within 3.1 percentage points, including 85% success on a precision i...
-
Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration
Dexplore learns dexterous robotic hand control from human MoCap demonstrations by treating them as soft, adaptively shrinking spatial references, then distills the policy into a vision-based controller.
-
Learning human-to-robot handovers through 3D scene reconstruction
A handover policy trained only on images rendered from a sparse-view Gaussian Splatting scene can deploy on a real robot without real-robot training data.
-
RwoR: Generating Robot Demonstrations from Human Hand Collection for Policy Learning without Robot
A generative model and wrist camera turn human hand videos into robot gripper demonstrations that train manipulation policies at success rates close to those trained on real gripper data.
-
TypeTele: Releasing Dexterity in Teleoperation by Dexterous Manipulation Types
A type-guided teleoperation system that selects predefined dexterous hand poses with a language model outperforms retargeting-based teleoperation on nine real-world tasks and improves imitation learning success.
-
Where Do Humans Look When Demonstrating to Robots? Human Gaze Behavior in Pick-and-Place Tasks Across Demonstration Devices
Demonstration devices that emulate a robot's body or camera view shift human gaze away from task objects toward the end-effector, and gaze from natural wearable recording improves imitation learning robustness.
-
TacPrint: A Wearable Fingertip Tactile Sensor for Human-to-Robot Contact Reproduction
A low-cost wearable fingertip sensor estimates dense contact-depth maps from 24 capacitive channels and uses them to substantially improve robot grasping and wiping in human-to-robot replay.
Discussion (0). Sign in to comment.