CoorDex distills privileged body and hand motion teachers into proprioceptive latent priors and composes them via shared-context residual RL heads to enable continuous high-DoF dexterous loco-manipulation.
Learning human-to-humanoid real-time whole-body teleoperation
10 Pith papers cite this work, alongside 3 external citations. Polarity classification is still indexing.
10
Pith papers citing it
3
external citations · Pith
abstract
We present Human to Humanoid (H2O), a reinforcement learning (RL) based framework that enables real-time whole-body teleoperation of a full-sized humanoid robot with only an RGB camera. To create a large-scale retargeted motion dataset of human movements for humanoid robots, we propose a scalable "sim-to-data" process to filter and pick feasible motions using a privileged motion imitator. Afterwards, we train a robust real-time humanoid motion imitator in simulation using these refined motions and transfer it to the real humanoid robot in a zero-shot manner. We successfully achieve teleoperation of dynamic whole-body motions in real-world scenarios, including walking, back jumping, kicking, turning, waving, pushing, boxing, etc. To the best of our knowledge, this is the first demonstration to achieve learning-based real-time whole-body humanoid teleoperation.
citation-role summary
background 3
citation-polarity summary
roles
background 3representative citing papers
CEER proposes a compliant end-effector and root control interface that unifies loco-manipulation for humanoids via a distilled low-level policy and hierarchical planners.
Recasts sampling-based nonconvex optimization as smoothed gradient descent to obtain non-asymptotic convergence guarantees and introduces the DIDA annealed algorithm that converges to the global optimum.
BifrostUMI enables robot-free human demonstration capture via VR and wrist cameras to train visuomotor policies that predict keypoint trajectories for transfer to humanoid whole-body control through retargeting.
A single learned controller called MHC enables real humanoid robots to execute diverse whole-body behaviors from multi-modal inputs via masked target trajectories.
Humanoid-DART bootstraps goal-conditioned humanoid loco-manipulation policies from sparse demonstrations via diffusion trajectory generation plus RL tracking.
OMG is a diffusion model for omni-modal whole-body humanoid motion generation that uses language, audio, and reference motions after large-scale data curation to achieve state-of-the-art performance and adaptation.
Diffusion-generated terrain-aware references plus closed-loop RL tracking enable whole-body humanoid locomotion on a Unitree G1 with onboard perception.
HTD, a multimodal transformer policy trained with behavioral cloning and touch dreaming to predict future tactile latents, achieves a 90.9% relative success rate improvement over baselines on five real-world contact-rich humanoid loco-manipulation tasks.
Novice operators achieved 80% success on object manipulation and 70% on social cube-passing using a VR-and-LLM-assisted humanoid teleoperation framework on a Unitree H1.
citing papers explorer
-
CoorDex: Coordinating Body and Hand Priors for Continuous Dexterous Humanoid Loco-Manipulation
CoorDex distills privileged body and hand motion teachers into proprioceptive latent priors and composes them via shared-context residual RL heads to enable continuous high-DoF dexterous loco-manipulation.
-
CEER: Compliant End-Effector and Root Control as a Unified Interface for Hierarchical Humanoid Loco-Manipulation
CEER proposes a compliant end-effector and root control interface that unifies loco-manipulation for humanoids via a distilled low-level policy and hierarchical planners.
-
Global Convergence of Sampling-Based Nonconvex Optimization through Diffusion-Style Smoothing
Recasts sampling-based nonconvex optimization as smoothed gradient descent to obtain non-asymptotic convergence guarantees and introduces the DIDA annealed algorithm that converges to the global optimum.
-
BifrostUMI: Bridging Robot-Free Demonstrations and Humanoid Whole-Body Manipulation
BifrostUMI enables robot-free human demonstration capture via VR and wrist cameras to train visuomotor policies that predict keypoint trajectories for transfer to humanoid whole-body control through retargeting.
-
Learning Multi-Modal Whole-Body Control for Real-World Humanoid Robots
A single learned controller called MHC enables real humanoid robots to execute diverse whole-body behaviors from multi-modal inputs via masked target trajectories.
-
Humanoid-DART: Humanoid Loco-Manipulation using Diffusion-guided Augmentation through Relabeling and Tracking
Humanoid-DART bootstraps goal-conditioned humanoid loco-manipulation policies from sparse demonstrations via diffusion trajectory generation plus RL tracking.
-
OMG: Omni-Modal Motion Generation for Generalist Humanoid Control
OMG is a diffusion model for omni-modal whole-body humanoid motion generation that uses language, audio, and reference motions after large-scale data curation to achieve state-of-the-art performance and adaptation.
-
Learning Whole-Body Humanoid Locomotion via Motion Generation and Motion Tracking
Diffusion-generated terrain-aware references plus closed-loop RL tracking enable whole-body humanoid locomotion on a Unitree G1 with onboard perception.
-
Learning Versatile Humanoid Manipulation with Touch Dreaming
HTD, a multimodal transformer policy trained with behavioral cloning and touch dreaming to predict future tactile latents, achieves a 90.9% relative success rate improvement over baselines on five real-world contact-rich humanoid loco-manipulation tasks.
-
Immersive Social Interaction with VR and LLM-Assisted Humanoids
Novice operators achieved 80% success on object manipulation and 70% on social cube-passing using a VR-and-LLM-assisted humanoid teleoperation framework on a Unitree H1.