REVIEW 8 cited by
MimicDroid: In-Context Learning for Humanoid Robot Manipulation from Human Play Videos
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
MimicDroid: In-Context Learning for Humanoid Robot Manipulation from Human Play Videos
read the original abstract
We aim to enable humanoid robots to efficiently solve new manipulation tasks from a few video examples. In-context learning (ICL) is a promising framework for achieving this goal due to its test-time data efficiency and rapid adaptability. However, current ICL methods rely on labor-intensive teleoperated data for training, which restricts scalability. We propose using human play videos -- continuous, unlabeled videos of people interacting freely with their environment -- as a scalable and diverse training data source. We introduce MimicDroid, which enables humanoids to perform ICL using human play videos as the only training data. MimicDroid extracts trajectory pairs with similar manipulation behaviors and trains the policy to predict the actions of one trajectory conditioned on the other. Through this process, the model acquired ICL capabilities for adapting to novel objects and environments at test time. To bridge the embodiment gap, MimicDroid first retargets human wrist poses estimated from RGB videos to the humanoid, leveraging kinematic similarity. It also applies random patch masking during training to reduce overfitting to human-specific cues and improve robustness to visual differences. To evaluate few-shot learning for humanoids, we introduce an open-source simulation benchmark with increasing levels of generalization difficulty. MimicDroid outperformed state-of-the-art methods and achieved nearly twofold higher success rates in the real world. Additional materials can be found on: ut-austin-rpl.github.io/MimicDroid
Forward citations
Cited by 8 Pith papers
-
WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time
A frozen world-action model can be steered to new tasks by adapting a lightweight memory from unlabeled human video via test-time training.
-
In-Context World Modeling for Robotic Control
ICWM frames system identification as in-context adaptation so VLA policies can infer dynamics from self-generated interactions and handle novel configurations without parameter updates.
-
Bimanual Robot Manipulation via Multi-Agent In-Context Learning
BiCICLe frames bimanual robot control as a multi-agent leader-follower problem with Arms' Debate and an LLM judge, achieving up to 71.1% success on 13 TWIN benchmark tasks without fine-tuning.
-
X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrations
X-Diffusion adapts Ambient Diffusion to selectively train on noised human actions for cross-embodiment robot policies, yielding 16% higher average success rates than naive co-training or manual filtering across five r...
-
In-Context World Modeling for Robotic Control
ICWM reframes system identification as in-context adaptation, letting VLA policies capture current world dynamics from task-agnostic interaction histories to generalize to novel configurations.
-
Learning of Robot Safety Policies via Adversarial Synthetic Scenarios
Proposes a Red Team-Blue Team adversarial gamification architecture to generate synthetic hazardous scenarios for learning robot safety policies.
-
HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos
HumanEgo reports 92.5% average success on four real robot tasks using only 15-30 minutes of human video per task and zero robot data, with zero-shot transfer to new robots and cameras.
-
Bimanual Robot Manipulation via Multi-Agent In-Context Learning
HYMN is a multi-sensor, time-synchronized indoor-outdoor positioning dataset combining five radio technologies (GNSS, UWB, BLE, WiFi, 5G) with total-station ground truth across 48 reference points in an industrial hall.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.