REVIEW 40 cited by
OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We present OmniH2O (Omni Human-to-Humanoid), a learning-based system for whole-body humanoid teleoperation and autonomy. Using kinematic pose as a universal control interface, OmniH2O enables various ways for a human to control a full-sized humanoid with dexterous hands, including using real-time teleoperation through VR headset, verbal instruction, and RGB camera. OmniH2O also enables full autonomy by learning from teleoperated demonstrations or integrating with frontier models such as GPT-4. OmniH2O demonstrates versatility and dexterity in various real-world whole-body tasks through teleoperation or autonomy, such as playing multiple sports, moving and manipulating objects, and interacting with humans. We develop an RL-based sim-to-real pipeline, which involves large-scale retargeting and augmentation of human motion datasets, learning a real-world deployable policy with sparse sensor input by imitating a privileged teacher policy, and reward designs to enhance robustness and stability. We release the first humanoid whole-body control dataset, OmniH2O-6, containing six everyday tasks, and demonstrate humanoid whole-body skill learning from teleoperated datasets.
Forward citations
Cited by 40 Pith papers
-
Learning Panorama-Aware VLA for Mobile Manipulation with Whole-Body Teleoperation
Adding a panoramic camera feed to a vision-language-action policy raises end-to-end success on four real-world mobile two-arm tasks from 30% to 73%.
-
Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills
A taxonomy of robot learning on a weights-versus-skills axis, with a five-rung self-improvement ladder whose top cell (feedback plus memory plus search) holds only a few recent systems.
-
GenTrack: Physical Alignment for Robot-Native Motion Generation and Zero-Shot Humanoid Tracking
Online co-training of a text-to-motion generator and a humanoid tracker on simulated G1 improves generator executability and zero-shot tracker coverage beyond static replay or one-way filtering.
-
First Deployable Dynamic-CoM: A Unified Policy and Method-Agnostic Benchmark for Humanoid Single-Leg Balance
A support-relative dynamic capture-point observation, reconstructible without base linear velocity, lets a humanoid policy hold clean single-leg balance at 86/90 in simulation and deploy on a Unitree G1 without distillation.
-
DexDirect: Direct Kinesthetic Arm Guidance for Efficient Dexterous Demonstration Collection
A hybrid kinesthetic-arm-plus-webcam-hand teleoperation interface achieved 17x/3x higher demonstration throughput than vision baselines and trained a 90%-success pick-and-place policy in a ten-person study.
-
What Matters in Humanoid General Motion Tracking? An Empirical Study
A controlled ablation of humanoid motion-tracking pipelines shows that explicit reference joint velocities and a short observation history improve tracking, while residual actions and teacher-student training yield on...
-
Behavior Foundations for Quadruped Robots: ABot-C0 Technical Report
A multi-source 16,074-clip quadruped motion library plus a flow-matching generalist tracker shows empirical data scaling and zero-shot unseen tracking, integrated with all-terrain locomotion and real-robot deployment.
-
WristMimic: Full-Body Humanoid Control with Wrist-Guided Manipulation
WristMimic achieves comparable or superior object manipulation retargeting by supervising wrist kinematics while letting finger behavior emerge from object and contact dynamics.
-
ThorArena: Benchmarking Humanoid Physical Interaction with Human Motion-Force Demonstrations
A force-aware humanoid benchmark pairs synchronized human motion-force data with simulation-based force replay to evaluate whole-body control policies under realistic physical disturbances.
-
Kinodynamic Motion Retargeting for Humanoid Locomotion via Multi-Contact Whole-Body Trajectory Optimization
A physics-aware motion-retargeting pipeline that uses ground-reaction-force-derived heel-toe contacts produces dynamically feasible humanoid references and improves downstream imitation learning.
-
Thor: Towards Human-Level Whole-Body Reactions for Intense Contact-Rich Environments
A decoupled whole-body RL policy with a force-based lean reward enables a Unitree G1 humanoid to pull with up to 167.7 N, beating prior controllers by 69–75%.
-
PHUMA: Physically Reliable Humanoid Locomotion Dataset
PHUMA is a curated 73-hour humanoid locomotion corpus whose physical-reliability metrics are partly defined by the same losses used to optimize it, and whose imitation success claims are confounded by in-distribution ...
-
Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulation
A 10,300-demonstration, 260-task multimodal humanoid manipulation dataset with baseline policy evaluations and a cloud evaluation platform.
-
UMI-on-Air: Embodiment-Aware Guidance for Embodiment-Agnostic Visuomotor Policies
Embodiment-Aware Diffusion Policy steers a UMI-trained diffusion policy with controller tracking-cost gradients at inference time, improving aerial manipulation success in simulation and real flights.
-
A Scalable Whole-body Motion Transfer via Implicit Kinodynamic Motion Retargeting
A neural retargeting pipeline maps human motion to humanoid robot motion at 5000+ frames per second using a shared latent space and physics-based fine-tuning, filtering noise and producing physically feasible trajectories.
-
Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactions
Claimed first large-scale egocentric and multi-view dataset of human-object-human assistance (11.4 hours, 1.2M frames) with three benchmarks; only the abstract was assessable because the submitted body text is a diffe...
-
TOP: Time Optimization Policy for Stable and Accurate Standing Manipulation with Humanoid Robots
A reinforcement-learned time optimization policy that adaptively slows upper-body motion clips improves stability and precision of humanoid standing manipulation at a modest time cost.
-
A3FR: Agile 3D Gaussian Splatting with Incremental Gaze Tracked Foveated Rendering in Virtual Reality
A3FR parallelizes CPU gaze tracking with GPU 3D Gaussian Splatting rendering using incremental early-exit gaze predictions, cutting end-to-end foveated rendering latency by up to 2x without measured quality loss.
-
RwoR: Generating Robot Demonstrations from Human Hand Collection for Policy Learning without Robot
A generative model and wrist camera turn human hand videos into robot gripper demonstrations that train manipulation policies at success rates close to those trained on real gripper data.
-
TypeTele: Releasing Dexterity in Teleoperation by Dexterous Manipulation Types
A type-guided teleoperation system that selects predefined dexterous hand poses with a language model outperforms retargeting-based teleoperation on nine real-world tasks and improves imitation learning success.
-
Learning Motion Skills with Adaptive Assistive Curriculum Force in Humanoid Robots
A2CF uses an adaptive assistive-force agent to guide humanoid robots through training, yielding faster convergence and robust policies that work without the external force.
-
GMT: General Motion Tracking for Humanoid Whole-Body Control
GMT trains a single unified humanoid policy using adaptive sampling and mixture-of-experts, achieving lower tracking errors than a re-implemented ExBody2 across diverse whole-body motions.
-
SkillBlender: Towards Versatile Humanoid Whole-Body Loco-Manipulation via Skill Blending
SkillBlender pretrains reusable goal-conditioned skills and blends them with softmax per-joint weights to solve simulated humanoid loco-manipulation tasks with one or two reward terms.
-
MoRE: Mixture of Residual Experts for Humanoid Lifelike Gaits Learning on Complex Terrains
A two-stage reinforcement learning pipeline with a mixture of latent residual experts gives a Unitree G1 humanoid multiple commanded human-like gaits over complex terrains.
-
Understanding and Mitigating Network Latency Effect on Teleoperated-Robot with Extended Reality
TeleXR decouples robot control and XR visualization from network delays by having each side reconstruct the other's state from local sensor data.
-
Hold My Beer: Learning Gentle Humanoid Locomotion and End-Effector Stabilization Control
A slow-fast two-agent reinforcement learning architecture with separate upper- and lower-body policies reduces end-effector shaking during humanoid locomotion.
-
Towards a Generalizable Bimanual Foundation Policy via Flow-based Video Prediction
CogRobot uses optical flow as an intermediate variable to fine-tune a text-to-video model for predicting bimanual robot trajectories, then maps those predictions to actions with a goal-conditioned diffusion policy.
-
H2-COMPACT: Human-Humanoid Co-Manipulation via Adaptive Contact Trajectory Policies
A hierarchical framework that maps wrist force/torque into velocity commands and then into stable leg motions lets a humanoid robot carry loads cooperatively with a human using only haptic cues.
-
ComplexMimic: Human-Scene Interaction Imitation in Complex 3D Environments
Dual-expert RL plus difficulty-aware multi-teacher distillation improves physics-based human–scene interaction imitation under complex 3D geometry versus prior single-policy baselines.
-
ZeroWBC: Learning Natural Whole-Body Humanoid Interaction from Human Egocentric Data
An open-loop generation-then-tracking system maps one egocentric image plus language into Unitree G1 whole-body interactions using only human egocentric motion data.
-
RobotDancing: Residual-Action Reinforcement Learning Enables Robust Long-Horizon Humanoid Motion Tracking
Residual-action reinforcement learning, with selective corrections on hip and knee pitch joints, enables zero-shot long-horizon dance tracking on real humanoid robots.
-
GBC: Generalized Behavior-Cloning Framework for Whole-Body Humanoid Imitation
GBC unifies MoCap retargeting and imitation learning into one framework that trains whole-body humanoid policies across multiple robot morphologies in simulation.
-
Towards Immersive Human-X Interaction: A Real-Time Framework for Physically Plausible Motion Synthesis
Human-X jointly predicts actions and reactions in real time to produce physically plausible human-machine interaction motion.
-
Hierarchical Vision-Language Planning for Multi-Step Humanoid Manipulation
A three-layer hierarchical system using a VLM planner and VLM skill monitor with imitation-learned skills and an RL tracking policy achieved 73% success on a real humanoid pick-and-place task.
-
Whole-Body Conditioned Egocentric Video Prediction
An autoregressive conditional diffusion transformer predicts future egocentric video from whole-body 3D pose sequences, trained on Nymeria, with atomic action and long-horizon evaluations.
-
Multi-Loco: Unifying Multi-Embodiment Legged Locomotion via Reinforcement Learning Augmented Diffusion
A single diffusion-plus-residual-RL policy, trained on four robot morphologies using zero-padded observations and actions, outperforms per-robot PPO baselines in simulation and transfers to real robots.
-
Multi-Embodiment Robotic Retargeting via Guided Diffusion Model
A graph-conditioned diffusion model retargets motions across heterogeneous robot embodiments without needing target-robot motion data, yet lacks baseline comparisons and error bars in its validation.
-
From Motion to Behavior: Hierarchical Modeling of Humanoid Generative Behavior Control
A new 124K-clip dataset with hierarchical text annotations, plus a pipeline that couples an LLM planner, a text-to-pose VAE, diffusion in-betweening, and physics control to generate long-horizon human behaviors.
-
Data Pyramid for Embodied Manipulation
Embodied training data form a five-layer pyramid—real-robot, UMI, ego/exo, simulation, general V–L—ordered by the trade-off between scale and robot alignment, and model capabilities track how those layers are mixed.
-
Feature-Based vs. GAN-Based Learning from Demonstrations: When and Why
Feature-based and GAN-based imitation learning should be selected by task priorities (fidelity, diversity, interpretability, adaptability), not by paradigm loyalty.
Discussion (0). Sign in to comment.