Pith. sign in

REVIEW 3 major objections 5 minor 33 cited by

OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction

T0 review · 3 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Interaction-preserving retargeting turns human demonstrations into kinematically feasible references that let a minimal RL policy run 30-second loco-manipulation and parkour on a real humanoid with zero-shot transfer.

desk verdict Interaction-preserving retargeting works; the paper deserves a real referee, but the flagship long-horizon claim is under-tested and the 'no curriculum' framing needs an asterisk. read the letter →

arxiv 2509.26633 v3 pith:NTGW3B7P submitted 2025-09-30 cs.RO cs.AIcs.LGcs.SYeess.SY

Lujie Yang , Xiaoyu Huang , Zhen Wu , Angjoo Kanazawa , Pieter Abbeel , Carmelo Sferrazza , C. Karen Liu , Rocky Duan
show 1 more author
Guanya Shi
This is my paper
classification cs.ROcs.AIcs.LGcs.SYeess.SY MSC 68T40
keywords motionretargetinghumanoidwhole-bodycontrolloco-manipulationinteractionmeshreinforcementlearningsim-to-realtransferdataaugmentationkinematicconstraints
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

OmniRetarget sets out to prove that the main obstacle to natural humanoid whole-body control is not reward engineering but the quality of kinematic reference data. Its claim: if human demonstrations are retargeted while explicitly preserving spatial and contact relationships between the robot, objects, and terrain—and while enforcing hard collision, joint, velocity, and foot-stance constraints—the resulting trajectories are clean enough that a simple proprioceptive RL policy, trained with only five shared reward terms and no curriculum, can execute long-horizon loco-manipulation and parkour on a physical humanoid with zero-shot sim-to-real transfer. The engine represents these relationships with an interaction mesh and minimizes its Laplacian deformation under hard constraints, framing retargeting as data generation rather than as an approximation to be patched later. This matters because it suggests a scalable route to diverse humanoid skills: one demonstration can be augmented into many object poses, shapes, and terrains, and downstream policies inherit robustness without per-task tuning.

What carries the argument

The key machinery is the interaction mesh: a Delaunay tetrahedralization built from user-specified key joints on the body together with densely sampled points on manipulated objects and terrain. Each vertex's Laplacian coordinate encodes its position relative to its neighbors, and retargeting minimizes the deformation of these coordinates between the human demonstration and the robot configuration. The per-frame optimization enforces hard kinematic constraints—collision avoidance via signed distance, joint and velocity limits, and zero horizontal velocity for stance feet—and is solved with a sequential SOCP that linearizes constraints around the previous iterate, uses a trust region, and war

What would settle it

Measure cumulative penetration volume and foot-skating distance over an entire 30-second retargeted sequence rather than per-frame maxima; if violations grow with horizon length, or an independently implemented exact-constraint solver yields substantially different trajectories, the kinematic-guarantee claim fails. Alternatively, evaluate policies trained on these references under object shapes and terrain perturbations slightly outside the augmentation ranges; a sharp drop in success would indicate that the interaction preservation is less transferable than claimed.

Watch

Extended reading notes

Core claim

OmniRetarget claims that interaction-preserving kinematic retargeting is sufficient to unlock complex, long-horizon whole-body behaviors that previously required extensive reward shaping. Concretely, it constructs an interaction mesh whose vertices are the robot's key joints plus densely sampled object and terrain points, then solves, frame by frame, a constrained optimization that minimizes the Laplacian deformation energy between the source human mesh and the target robot mesh subject to hard constraints: signed-distance collision avoidance, joint and velocity limits, and fixed stance feet. The optimization is solved by a sequential second-order cone program with linearized constraints and

Load-bearing premise

The load-bearing premise is that the sequential SOCP, with its linearly approximated collision and foot constraints, produces trajectories that genuinely satisfy the advertised hard kinematic guarantees; residual penetration up to about 1.34 cm shows the guarantee is only approximate, and if such violations accumulate over long multi-stage motions, downstream RL and sim-to-real transfer could degrade.

Editorial extensions

If this is right

  • Cleaner references shrink reward engineering: five shared reward terms replace task-specific contact schedules, foot-flight regularizers, and curriculum tuning.
  • A single demonstration can be expanded into hours of kinematically feasible trajectories covering varied object poses, sizes, and terrain, lowering data collection cost.
  • Retargeting quality directly transfers to downstream RL: higher contact preservation and lower penetration translate into higher success rates and lower variance across motions.
  • Zero-shot sim-to-real transfer becomes attainable for agile whole-body skills, including object carrying, platform climbing, slope crawling, and a dynamic wall-flip.
  • Extending the frame-wise solver to joint whole-trajectory optimization (flagged by the paper as future work) could make the pipeline robust to noisier sources like video.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the results position reference-data quality as a primary bottleneck for humanoid imitation; other learning pipelines could likely gain as much from interaction-preserving data generation as from better RL algorithms.
  • Editorial inference: since the mesh is built from keypoint correspondences and a collision model, the same engine should transfer to other humanoid embodiments cheaply; the paper demonstrates this on three robots but does not quantify cross-embodiment policy transfer.
  • Editorial inference: the object-frame mesh construction suggests the method may extend to deformable objects or multi-object scenes, though the paper only evaluates rigid objects and static terrain.
  • Editorial inference: a direct ablation separating the hard-constraint enforcement from the interaction-preservation objective is not reported; such an ablation would clarify which component drives the RL gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. OmniRetarget is an offline motion retargeting pipeline that transfers human demonstrations to humanoid robots while preserving interactions with objects and terrain. The method builds an interaction mesh over body keypoints and sampled object/terrain points, and solves a per-frame constrained optimization (Eq. 3) that minimizes Laplacian deformation subject to collision-avoidance, joint/velocity limits, and foot-sticking constraints. The paper reports that the resulting trajectories are kinematically feasible and interaction-preserving, and that a single demonstration can be augmented into diverse object/terrain variations. These references are then used to train proprioceptive RL policies with only five reward terms and shared domain randomization; the authors report successful long-horizon (30 s) loco-manipulation and parkour on a Unitree G1, with zero-shot sim-to-real transfer. Quantitative comparisons against PHC, GMR, and VideoMimic on kinematic quality and downstream RL success are provided in Table II.

Significance. If the claims hold, the paper makes a valuable contribution to a recognized data bottleneck in humanoid whole-body control. The optimization formulation is specified in detail, the kinematic-quality benchmark covers multiple data sources, and the downstream RL results—including hardware demonstrations—are substantially better than the baselines. The interaction-mesh-based augmentation scheme is a promising idea for expanding demonstration coverage. The paper promises to release code, datasets, and policies, which would further increase its impact. However, the advertised 'hard kinematic constraint' guarantee is only approximate because the constraints are linearized in a trust-region SOCP, and the long-horizon feasibility claim is not directly evaluated. These issues are load-bearing for the central claim, so the current significance is conditional on addressing them.

major comments (3)
  1. [§III-A and Appendix D, Eq. (15)] The text repeatedly calls (3b)–(3e) 'hard kinematic constraints' and the abstract states that OmniRetarget 'generates kinematically feasible trajectories.' However, the solver linearizes these constraints within a sequential SOCP with a trust region (Eq. 15) and terminates after at most 10 iterations. Table II shows nonzero max penetration (1.34 cm for robot-object, 1.37 cm for robot-terrain), and Sec. V-B.a explicitly attributes this to linearization of constraint (3b). Thus the output satisfies only linearized constraints at the iterate, and the guarantee is approximate, not hard. Please either (a) verify exact constraint satisfaction at convergence over all frames—including the 30 s parkour sequence—and report worst-case violations, or (b) revise the abstract and contribution claims to say 'approximately feasible' and state clearly how this differs from soft-penalty methods. This matt
  2. [Sec. III-A / Sec. V-B.a, Table II] The foot-skating metric is defined relative to the 'desired foot sticking length,' but the stance-phase label is exactly the same 1 cm/s horizontal-velocity threshold used to impose constraint (3e). Consequently, a foot that slides in the source motion at, say, 0.8 cm/s is labeled stance, forced to stick by the optimizer, and then counted as correctly stuck in the evaluation. The reported '0' foot-skating duration is therefore not independent evidence of contact quality. An independent foot-contact annotation, or at least a sensitivity analysis of the 1 cm/s threshold, is needed before the kinematic-quality comparison can be taken at face value.
  3. [Sec. VI (Conclusion) and Eq. (3)] The frame-by-frame formulation has no mechanism preventing constraint violations from accumulating over long horizons. Table II reports averages over individual motions; the 30 s multi-stage parkour sequence (Sec. V-A.a) is not included in the quantitative kinematic metrics. The conclusion acknowledges that 'future work could explore jointly optimizing the entire trajectory to enhance the framework's robustness,' but the abstract and contributions present the output as generally kinematically feasible. Please report worst-case penetration and foot-skating over the full long-horizon trajectory, or explicitly qualify the feasibility claim to per-frame/per-motion metrics.
minor comments (5)
  1. [Sec. V-A.a] The wall-flip result required relaxing the termination condition and removing a foot-joint orientation tracking term. This should be stated at the point where the 'only 5 reward terms' and 'without hyperparameter tuning' claims are made, to avoid overgeneralization.
  2. [Abstract / Sec. V-B] The paper states that all code, retargeted datasets, and trained policies will be publicly released, but no repository or dataset identifiers are given, and the in-house MoCap dataset is not described in enough detail to be reproduced. Please provide availability details and dataset statistics.
  3. [Appendix D, Eq. (15)] The trust-region radius ε=0.2 and the 10-iteration cap are introduced without a convergence criterion. A brief sensitivity study for ε and for the free parameters W, τ_p, and τ_θ would strengthen practical guidance.
  4. [Table II] In the LAFAN1 row, OmniRetarget reports max depth 1.07±0.00 and zero variance. Please clarify whether this is a single motion or why the variance is exactly zero.
  5. [Fig. 2] The figure caption contains stray text (e.g., 'min)!') and the equations in the Appendix are not all referenced from the main text. Please clean up the caption and add cross-references.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the retargeting output is not an input by construction, and the headline metrics are either independently evaluated or direct constraint checks rather than fitted predictions.

full rationale

OmniRetarget's core derivation is the optimization in Eq. (3): minimize interaction-mesh Laplacian deformation subject to hard kinematic constraints. The headline claims—kinematic feasibility, interaction preservation, and downstream RL success—are not equivalent to this objective by construction. Contact preservation is measured as contact duration, not directly enforced as an objective; penetration residuals (max depth 1.34 cm, Table II) show that the collision constraint is only approximately satisfied due to linearization, so the feasibility metrics are not forced to zero. The foot-skating metric is directly tied to the stance-foot equality constraint (3e), so a zero value is a consistency check of the solver rather than an independent prediction; this is not a circular derivation. Downstream RL success and zero-shot sim-to-real transfer provide external, task-level validation that is not reducible to the optimization inputs. The use of reward weights from BeyondMimic [33] is a stated adoption of prior work, not a self-citation chain that determines the main result, and the LAFAN1 comparison against the Unitree dataset [34] supplies an external baseline. The acknowledged limitation about per-frame vs. joint trajectory optimization is a numerical-robustness concern, not circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

Everything the central claim rests on that isn't derived in the paper: the interaction-mesh proxy, the scaling/correspondence mapping, the stance labeling threshold, the per-frame SOCP feasibility, and the transfer of BeyondMimic's minimal-reward assumption to this setting.

free parameters (5)
  • Anchor weight W (Eq. 4) = not given
    The lower-body anchor penalty in augmentation is introduced to prevent trivial rigid transformations and shapes the augmented data; no value or selection procedure is reported.
  • Temporal smoothness cost matrix Q (Eq. 3a) = not specified
    Controls the smoothness vs interaction-deformation tradeoff; chosen by hand and not reported numerically.
  • Trust-region radius epsilon (Eq. 15g) = 0.2
    Step-size bound in sequential SOCP; arbitrary and affects convergence and feasibility of the linearized constraints.
  • Stance-foot velocity threshold = 1 cm/s
    Defines which foot is in stance and triggers the foot-stick constraint; directly determines the foot-skating measurement.
  • Object augmentation time constants tau_p, tau_theta (Eq. 14) = not specified
    Decay rates for object pose perturbations; chosen by hand and controls the augmentation distribution.
assumptions (5)
  • domain assumption Minimizing Laplacian deformation of interaction mesh (Eq. 2) preserves task-relevant spatial and contact relationships between robot, object, and terrain.
    Central premise inherited from Ho et al. [14] and IMMA [22]; the paper does not prove it and relies on it to define 'interaction-preserving' retargeting.
  • domain assumption Source human keypoints can be mapped to the robot by a single global height-ratio scale factor alpha (Eq. 7) with semantically consistent point correspondences.
    Used for all datasets; if per-bone scaling is needed, results may differ. Paper admits per-bone scaling is possible but not used.
  • ad hoc to paper A foot is in stance iff its horizontal velocity in source motion is below 1 cm/s (Sec. III-A); this label drives the foot-stick hard constraint and foot-skating evaluation.
    Threshold chosen by hand; no contact-force ground truth; a mislabeled stance would either create skating or false sticking.
  • ad hoc to paper Sequential SOCP with linearized constraints and warm start converges to a practically feasible robot trajectory at every frame (Eq. 15).
    No convergence or feasibility guarantee is provided; paper acknowledges residual penetration from linearization.
  • domain assumption A clean kinematic reference is sufficient for proprioceptive RL with five shared rewards to learn robust dynamics and transfer zero-shot (extrapolated from BeyondMimic [33]).
    The entire minimal-RL claim rests on this transfer of BeyondMimic's result to interaction-rich references.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction." pith.science (2026). https://pith.science/paper/NTGW3B7P

@misc{pith2026250926633,
  author       = {Pith},
  title        = {Pith review of: OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NTGW3B7P}},
  note         = {Machine review of arXiv:2509.26633}
}
read the original abstract

A dominant paradigm for teaching humanoid robots complex skills is to retarget human motions as kinematic references to train reinforcement learning (RL) policies. However, existing retargeting pipelines often struggle with the significant embodiment gap between humans and robots, producing physically implausible artifacts like foot-skating and penetration. More importantly, common retargeting methods neglect the rich human-object and human-environment interactions essential for expressive locomotion and loco-manipulation. To address this, we introduce OmniRetarget, an interaction-preserving data generation engine based on an interaction mesh that explicitly models and preserves the crucial spatial and contact relationships between an agent, the terrain, and manipulated objects. By minimizing the Laplacian deformation between the human and robot meshes while enforcing kinematic constraints, OmniRetarget generates kinematically feasible trajectories. Moreover, preserving task-relevant interactions enables efficient data augmentation, from a single demonstration to different robot embodiments, terrains, and object configurations. We comprehensively evaluate OmniRetarget by retargeting motions from OMOMO, LAFAN1, and our in-house MoCap datasets, generating over 8-hour trajectories that achieve better kinematic constraint satisfaction and contact preservation than widely used baselines. Such high-quality data enables proprioceptive RL policies to successfully execute long-horizon (up to 30 seconds) parkour and loco-manipulation skills on a Unitree G1 humanoid, trained with only 5 reward terms and simple domain randomization shared by all tasks, without any learning curriculum.

Figures

Figures reproduced from arXiv: 2509.26633 by the authors.

Figure 1
Figure 1. OMNIRETARGET enables reinforcement learning policies to learn complex, long-horizon loco-manipulation skills in challenging environments that transfer zero-shot from simulation to a Unitree G1 humanoid. Thanks to the high-quality interaction-preserving motion retargeting, these policies are trained and deployed in a minimal and unified way: it involves only 5 rewards, 4 robot domain randomization terms, and a purely… view at source ↗
Figure 2
Figure 2. OMNIRETARGET overview. Human demonstrations are retargeted to the robot via interaction-mesh–based constrained optimization. Each spatial and shape augmentation is solved as a new optimization, producing diverse trajectories that serve as references for RL training with minimal reward design and domain randomization, enabling zero-shot transfer to real-world humanoids. reward formulation like BeyondMimic [33] achiev… view at source ↗
Figure 3
Figure 3. Cross-embodiment robot-object-terrain interaction. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: OMNIRETARGET generates systematic variations of (a) terrain height, (b) object initial pose, and (c) object shape from a single human demonstration, with optimized motions in simulation (top) transferring consistently to hardware (bottom). parts naturally follow the ob…
Figure 5
Figure 5. Figure 5: Additional hardware results showing diverse, agile and human-like behaviors. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Hardware results showing a high-dynamic wall-flip motion. The robot reaches a maximum linear velocity of 3 [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Artifacts resulting from the retargeting baselines. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: The Laplacian coorinate should stay the same when [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: The actual target (left) and source (right) interaction [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Histograms from the downstream RL evaluation [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 33 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Light-Loco-Parkour: Versatile Perceptive Whole-Body Locomotion via Multi-Skill Distillation

    cs.RO 2026-08 conditional novelty 7.0 of 10

    A single neural-network policy, trained in simulation, makes a humanoid climb, vault, and traverse uneven terrain from onboard depth and a velocity command, with no skill labels or runtime motion graphs.

  2. Handroid: Bridging Dexterous Hand and Humanoid

    cs.RO 2026-07 conditional novelty 7.0 of 10

    A single 27-DoF body doubles as an anthropomorphic dexterous hand and a 0.33 m desktop humanoid, with a unified control stack for manipulation, locomotion, and embodiment switching.

  3. MotionDisco: Motion Discovery for Extreme Humanoid Loco-Manipulation

    cs.RO 2026-06 unverdicted novelty 7.0 of 10

    MotionDisco discovers long-horizon humanoid loco-manipulation motions from scratch via LLM-guided evolutionary search, trajectory optimization, and pruning, then transfers them to real robots with RL policies.

  4. Extreme-RGMT: Continual Learning of Highly Dynamic Skills for Robust Generalist Humanoid Control

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A two-stage continual-learning framework lets a generalist humanoid tracking policy acquire highly dynamic acrobatic skills while preserving its general-purpose motion capabilities.

  5. Scaling Behavior Foundation Model for Humanoid Robots

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A scaling recipe for humanoid behavior foundation models—global-frame motion tracking, on-policy data quantity plus reference diversity, and a transformer with hyperspherical latents—cuts global tracking error by roug...

  6. EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal

    cs.RO 2026-07 conditional novelty 6.0 of 10

    EgoHTR is a 55-sequence, 150k-frame egocentric 4D human-terrain dataset with a reconstruction pipeline, MoCap-validated benchmark, and perceptive locomotion policies deployed on a Unitree G1.

  7. ContactMimic: Humanoid Object Interaction via Contact Control

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A humanoid tracking policy is trained with contact-following rewards and trajectory augmentation to decouple physical contact from keypoint geometry, enabling runtime contact control.

  8. ThorArena: Benchmarking Humanoid Physical Interaction with Human Motion-Force Demonstrations

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A force-aware humanoid benchmark pairs synchronized human motion-force data with simulation-based force replay to evaluate whole-body control policies under realistic physical disturbances.

  9. SceneBot: Contact-Prompted General Humanoid Whole Body Tracking with Scene-Interaction

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    SceneBot conditions a humanoid tracking policy on motion references and contact labels, using reconstructed scene-interaction data to unify free-space locomotion with contact-rich manipulation and terrain tasks.

  10. PressMimic: Pressure-Guided Motion Capture and Control for Humanoid Robot Imitation

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    PressMimic fuses RGB and pressure for pose estimation via FRAPPE++ and uses pressure signals in RL policy PSP, backed by the MotionPRO dataset, to achieve physically consistent humanoid motion imitation.

  11. TaskNPoint: How to Teach Your Humanoid to Hit a Backhand in Minutes

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    TaskNPoint lets humanoid robots learn dynamic skills such as tennis backhands from single short human video demonstrations plus under one hour of single-GPU simulation training, achieving zero-shot generalization to n...

  12. OmniContact: Chaining Meta-Skills via Contact Flow for Generalizable Humanoid Loco-Manipulation

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    OmniContact introduces contact flow as a shared representation of body trajectories and contact signals to learn and chain loco-manipulation meta-skills, reporting 98.7% success on box carrying and 76.5% on push-stack tasks.

  13. PhysDrift: Bridging the Embodiment Gap in Humanoid Co-Speech Motion Generation

    cs.AI 2026-06 unverdicted novelty 6.0 of 10

    PhysDrift generates executable humanoid co-speech motions directly from speech via robot-native data curated by IK-EER, claiming better alignment and plausibility than human-centric retargeting.

  14. GenHOI: Contact-Aware Humanoid-Object Interaction by Imitating Generated Videos without Task-Specific Training

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    GenHOI reconstructs robot-object scenes, generates task videos from language and first-frame images, extracts contact constraints, optimizes reference trajectories, and executes them via closed-loop control for zero-s...

  15. Video2Sim2Real: Full-Stack Autonomous Dexterous Skill Acquisition from a Single Human Video

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    Video2Sim2Real turns a single human video into a deployable robot manipulation skill by reconstructing a digital twin, anchoring motions to object-centric simulator configurations, and bridging sim-to-real gaps with i...

  16. Perceptive Behavior Foundation Model: Adapting Human Motion Priors to Robot-Centric Terrain

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    Perceptive BFM grounds human motion priors in robot terrain perception via terrain-conformal reference synthesis and teacher-student transfer from adapted to raw-reference tracking.

  17. LIMMT: Less is More for Motion Tracking

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    A data-centric approach shows that less than 3% of AMASS motion data, filtered by physics feasibility, diversity, and complexity, yields better humanoid tracking policies than the full dataset.

  18. PHASOR: Phase-Anchored Universal Action Representations for Humanoid Embodiments

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    PHASOR factorizes motion into an FFT-based phase manifold and pose branch with semantic distillation to produce a cross-embodiment, human-anchored action embedding space for humanoid robots.

  19. New Scheme Adaption Strategy for Hyperbolic Conservation Laws

    math.NA 2026-04 unverdicted novelty 6.0 of 10

    Continuously varying an SBM-type limiter parameter yields a smooth rough-to-smooth transition that improves resolution and cuts dissipation versus threshold-based adaptive schemes for Euler equations.

  20. HERO: Learning Humanoid End-Effector Control for Visual Whole-Body Open-Vocabulary Object Grasping

    cs.RO 2026-02 conditional novelty 6.0 of 10

    HERO achieves 2.44 cm end-effector tracking error on a Unitree G1 humanoid and uses it, with open-vocabulary perception, to grasp novel objects at up to 90% success in diverse real scenes.

  21. EgoHumanoid: Unlocking In-the-Wild Loco-Manipulation with Robot-Free Egocentric Demonstration

    cs.RO 2026-02 conditional novelty 6.0 of 10

    Co-training a vision-language-action humanoid policy on aligned egocentric human demonstrations plus limited robot data improves real-world loco-manipulation success by 20% in-domain and 51% in environments the robot ...

  22. ObjRetarget: An Object-Aware Motion Retargeting Framework with Anthropomorphic Arm Constraints and Polyhedral Hand Modeling

    cs.RO 2026-07 conditional novelty 5.0 of 10

    Decoupled arm–hand retargeting with anthropomorphic arm-plane constraints and polyhedral contact invariants raises real-robot dexterous-task success to 75.8% versus 61.6% and 50.8% for OKAMI and ORION.

  23. TopoRetarget: Interaction-Preserving Retargeting for Dexterous Manipulation

    cs.RO 2026-06 unverdicted novelty 5.0 of 10

    TopoRetarget uses a sparse interaction graph and distance-weighted Laplacian deformation optimization with kinematic and penetration constraints to retarget human demonstrations to dexterous hands while preserving tas...

  24. OMG: Omni-Modal Motion Generation for Generalist Humanoid Control

    cs.RO 2026-06 unverdicted novelty 5.0 of 10

    OMG is a diffusion model for omni-modal whole-body humanoid motion generation that uses language, audio, and reference motions after large-scale data curation to achieve state-of-the-art performance and adaptation.

  25. VAIC: Vision-Guided Humanoid Agile Object Interaction Control via Decoupled Commands

    cs.RO 2026-06 unverdicted novelty 5.0 of 10

    VAIC distills a teacher policy into a vision-and-proprioception student policy using recurrent adaptation and decoupled commands, enabling diverse real-robot tasks like box carrying and skateboarding that outperform b...

  26. HANDOFF: Humanoid Agentic Task-Space Whole-Body Control via Distilled Complementary Teachers

    cs.RO 2026-06 unverdicted novelty 5.0 of 10

    HANDOFF is a distilled mixture-of-experts humanoid whole-body controller that follows a compact task-space interface, matches SOTA velocity tracking, provides large manipulation workspace on Unitree G1, and supports V...

  27. LadderMan: Learning Humanoid Perceptive Ladder Climbing

    cs.RO 2026-06 unverdicted novelty 5.0 of 10

    A hybrid motion-tracking and imitation-reinforcement pipeline produces a depth-based visuomotor policy that lets humanoids climb varied ladders zero-shot on hardware and perform teleoperated manipulation while climbing.

  28. Humanoid-GPT: Scaling Data and Structure for Zero-Shot Motion Tracking

    cs.RO 2026-06 unverdicted novelty 5.0 of 10

    Humanoid-GPT is a causal Transformer pre-trained on a unified billion-scale motion dataset that tracks dynamic behaviors with zero-shot generalization to unseen motions and tasks.

  29. Constrained Whole-Body Tracking for Humanoid Robots

    cs.RO 2026-05 unverdicted novelty 5.0 of 10

    ConstrainedMimic integrates operational space control and control barrier functions into RL tracking policies to enforce arbitrary runtime constraints on humanoid kinematics and dynamics while preserving contact modes...

  30. HumanoidMimicGen: Data Generation for Loco-Manipulation via Whole-Body Planning

    cs.RO 2026-05 unverdicted novelty 5.0 of 10

    HumanoidMimicGen automatically generates large loco-manipulation datasets from few source demonstrations using whole-body planning, enabling visuomotor policies that outperform real-data-only training by 20% on a new ...

  31. Imitation Learning from Human Motion Alone Does Not Guarantee Biomechanically Plausible Gait Kinetics

    cs.RO 2026-03 conditional novelty 5.0 of 10

    Motion-only imitation learning reproduces walking kinematics but produces inaccurate ground reaction forces and joint moments; adding GRF and center-of-pressure rewards brings simulated kinetics closer to inverse dynamics.

  32. Consensus-based optimization (CBO): Towards Global Optimality in Robotics

    cs.RO 2026-02 conditional novelty 5.0 of 10

    On long-horizon, underactuated, and high-dimensional simulated robot planning tasks, consensus-based optimization finds lower-cost trajectories than MPPI, CEM, and CMA-ES.

  33. SplitAdapter: Load-Aware Humanoid Loco-Manipulation via Factorized Adaptation

    cs.RO 2026-06 unverdicted novelty 4.0 of 10

    SplitAdapter factorizes adaptation into load-aware and dynamics-aware encoders using split world-model objectives, GRL regularization, and hierarchical FiLM, reporting higher full-task success than baselines across 2-...

Reference graph

Works this paper leans on

63 extracted references · 2 linked inside Pith · cited by 33 Pith papers

  1. [1]

    Object motion guided human motion synthesis,

    J. Li, J. Wu, and C. K. Liu, “Object motion guided human motion synthesis,”ACM Transactions on Graphics (TOG), 2023

    2023

  2. [2]

    Robust motion in-betweening,

    F. G. Harvey, M. Yurick, D. Nowrouzezahrai, and C. Pal, “Robust motion in-betweening,” vol. 39, no. 4, 2020

    2020

  3. [3]

    Learning quadrupedal locomotion over challenging terrain,

    J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning quadrupedal locomotion over challenging terrain,”Science robotics, 2020

    2020

  4. [4]

    Deepmimic: Example-guided deep reinforcement learning of physics-based char- acter skills,

    X. B. Peng, P. Abbeel, S. Levine, and M. Van de Panne, “Deepmimic: Example-guided deep reinforcement learning of physics-based char- acter skills,”ACM Transactions On Graphics (TOG), 2018

    2018

  5. [5]

    Parc: Physics-based augmentation with reinforcement learning for character controllers,

    M. Xu, Y . Shi, K. Yin, and X. B. Peng, “Parc: Physics-based augmentation with reinforcement learning for character controllers,” inProceedings of the SIGGRAPH Conference Papers, 2025

    2025

  6. [6]

    Human-object interaction from human-level instructions,

    Z. Wu, J. Li, P. Xu, and C. K. Liu, “Human-object interaction from human-level instructions,”arXiv preprint arXiv:2406.17840, 2024

    arXiv 2024

  7. [7]

    Humanplus: Humanoid shadowing and imitation from humans,

    Z. Fu, Q. Zhao, Q. Wu, G. Wetzstein, and C. Finn, “Humanplus: Humanoid shadowing and imitation from humans,”CoRL, 2024

    2024

  8. [8]

    Omnih2o: Universal and dexterous human-to- humanoid whole-body teleoperation and learning,

    T. He, Z. Luo, X. He, W. Xiao, C. Zhang, W. Zhang, K. M. Kitani, C. Liu, and G. Shi, “Omnih2o: Universal and dexterous human-to- humanoid whole-body teleoperation and learning,” inCoRL, 2025

    2025

Show all 63 references
  1. [9]

    Twist: Teleoperated whole-body imitation system,

    Y . Ze, Z. Chen, J. P. Ara ˜Aˇsjo, Z.-a. Cao, X. B. Peng, J. Wu, and C. K. Liu, “Twist: Teleoperated whole-body imitation system,”CoRL, 2025

    2025

  2. [10]

    Perpetual humanoid control for real-time simulated avatars,

    Z. Luo, J. Cao, A. W. Winkler, K. Kitani, and W. Xu, “Perpetual humanoid control for real-time simulated avatars,” inICCV, 2023

    2023

  3. [11]

    Visual imitation enables contextual humanoid control,

    A. Allshire, H. Choi, J. Zhang, D. McAllister, A. Zhang, C. M. Kim, T. Darrell, P. Abbeel, J. Malik, and A. Kanazawa, “Visual imitation enables contextual humanoid control,”CoRL, 2025

    2025

  4. [12]

    Hub: Learning extreme humanoid balance,

    T. Zhang, B. Zheng, R. Nai, Y . Hu, Y .-J. Wang, G. Chen, F. Lin, J. Li, C. Hong, K. Sreenathet al., “Hub: Learning extreme humanoid balance,”CoRL, 2025

    2025

  5. [13]

    Asap: Aligning simulation and real-world physics for learning agile humanoid whole-body skills,

    T. He, J. Gao, W. Xiao, Y . Zhang, Z. Wang, J. Wang, Z. Luo, G. He, N. Sobanbabu, C. Pan, Z. Yi, G. Qu, K. Kitani, J. Hodgins, L. J. Fan, Y . Zhu, C. Liu, and G. Shi, “Asap: Aligning simulation and real-world physics for learning agile humanoid whole-body skills,”arXiv, 2025

    2025

  6. [14]

    Spatial relationship preserving character motion adaptation,

    E. S. L. Ho, T. Komura, and C.-L. Tai, “Spatial relationship preserving character motion adaptation,”ACM Transactions on Graphics, 2010

    2010

  7. [15]

    Physics-driven data generation for contact- rich manipulation via trajectory optimization,

    L. Yang, H. Suh, T. Zhao, B. P. Graesdal, T. Kelestemur, J. Wang, T. Pang, and R. Tedrake, “Physics-driven data generation for contact- rich manipulation via trajectory optimization,”RSS, 2025

    2025

  8. [16]

    Sparse motion semantics for contact-aware retargeting,

    T. Cheynel, T. Rossi, B. Bellot-Gurlet, D. Rohmer, and M.-P. Cani, “Sparse motion semantics for contact-aware retargeting,” inACM SIGGRAPH Conference on Motion, Interaction and Games, 2023

    2023

  9. [17]

    Retargeting human-object interaction to virtual avatars,

    Y . Kim, H. Park, S. Bang, and S.-H. Lee, “Retargeting human-object interaction to virtual avatars,”IEEE transactions on visualization and computer graphics, vol. 22, no. 11, pp. 2405–2412, 2016

    2016

  10. [18]

    Retargetting motion to new characters,

    M. Gleicher, “Retargetting motion to new characters,” inProceedings of the 25th annual conference on Computer graphics and interactive techniques, 1998, pp. 33–42

    1998

  11. [19]

    Skeleton-aware networks for deep motion retargeting,

    K. Aberman, P. Li, D. Lischinski, O. Sorkine-Hornung, D. Cohen-Or, and B. Chen, “Skeleton-aware networks for deep motion retargeting,” ACM Transactions on Graphics (TOG), vol. 39, no. 4, pp. 62–1, 2020

    2020

  12. [20]

    Neural kinematic networks for unsupervised motion retargetting,

    R. Villegas, J. Yang, D. Ceylan, and H. Lee, “Neural kinematic networks for unsupervised motion retargetting,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018

    2018

  13. [21]

    Simulation and retargeting of complex multi-character interactions,

    Y . Zhang, D. Gopinath, Y . Ye, J. Hodgins, G. Turk, and J. Won, “Simulation and retargeting of complex multi-character interactions,” inACM SIGGRAPH 2023 Conference Proceedings, 2023

    2023

  14. [22]

    Interaction mesh based motion adaptation for biped humanoid robots,

    S. Nakaoka and T. Komura, “Interaction mesh based motion adaptation for biped humanoid robots,” inHumanoids, 2012

    2012

  15. [23]

    Sim-to-real learning for humanoid box loco-manipulation,

    J. Dao, H. Duan, and A. Fern, “Sim-to-real learning for humanoid box loco-manipulation,” inICRA. IEEE, 2024

    2024

  16. [24]

    Learning humanoid locomotion with perceptive internal model,

    J. Long, J. Ren, M. Shi, Z. Wang, T. Huang, P. Luo, and J. Pang, “Learning humanoid locomotion with perceptive internal model,” arXiv, 2024

    2024

  17. [25]

    Attention-based map encoding for learning generalized legged loco- motion,

    J. He, C. Zhang, F. Jenelten, R. Grandia, M. B ¨acher, and M. Hutter, “Attention-based map encoding for learning generalized legged loco- motion,”Science Robotics, vol. 10, no. 105, p. eadv3604, 2025

    2025

  18. [26]

    Learning getting-up policies for real-world humanoid robots,

    X. He, R. Dong, Z. Chen, and S. Gupta, “Learning getting-up policies for real-world humanoid robots,”RSS, 2025

    2025

  19. [27]

    Skillblender: Towards versatile humanoid whole-body loco-manipulation via skill blending,

    Y . Kuang, H. Geng, A. Elhafsi, T.-D. Do, P. Abbeel, J. Malik, M. Pavone, and Y . Wang, “Skillblender: Towards versatile humanoid whole-body loco-manipulation via skill blending,”arXiv, 2025

    2025

  20. [28]

    Unleashing humanoid reaching potential via real-world-ready skill space,

    Z. Zhang, C. Chen, H. Xue, J. Wang, S. Liang, Y . Liu, Z. Zhang, H. Wang, and L. Yi, “Unleashing humanoid reaching potential via real-world-ready skill space,”arXiv preprint arXiv:2505.10918, 2025

    2025

  21. [29]

    A unified and general humanoid whole-body controller for versatile locomotion,

    Y . Xue, W. Dong, M. Liu, W. Zhang, and J. Pang, “A unified and general humanoid whole-body controller for versatile locomotion,” RSS, 2025

    2025

  22. [30]

    Wococo: Learning whole-body humanoid control with sequential contacts, 2024,

    C. Zhang, W. Xiao, T. He, and G. Shi, “Wococo: Learning whole-body humanoid control with sequential contacts, 2024,”arXiv, 2024

    2024

  23. [31]

    Falcon: Learning force-adaptive humanoid loco-manipulation,

    Y . Zhang, Y . Yuan, P. Gurunath, T. He, S. Omidshafiei, A.-a. Agha- mohammadi, M. Vazquez-Chanlatte, L. Pedersen, and G. Shi, “Falcon: Learning force-adaptive humanoid loco-manipulation,”arXiv, 2025

    2025

  24. [32]

    Reinforcement learning for versatile, dynamic, and robust bipedal locomotion control,

    Z. Li, X. B. Peng, P. Abbeel, S. Levine, G. Berseth, and K. Sreenath, “Reinforcement learning for versatile, dynamic, and robust bipedal locomotion control,”IJRR, 2025

    2025

  25. [33]

    Beyondmimic: From motion tracking to versatile humanoid control via guided diffusion,

    Q. Liao, T. E. Truong, X. Huang, G. Tevet, K. Sreenath, and C. K. Liu, “Beyondmimic: From motion tracking to versatile humanoid control via guided diffusion,”arXiv e-prints, pp. arXiv–2508, 2025

    2025

  26. [34]

    Unitree lafan1 retargeting dataset,

    U. Robotics and Contributors, “Unitree lafan1 retargeting dataset,” https://huggingface.co/datasets/lvhaidong/LAFAN1 Retargeting Dataset, 2025

    2025

  27. [35]

    Deep imitation learning for humanoid loco-manipulation through human teleoperation,

    M. Seo, S. Han, K. Sim, S. H. Bang, C. Gonzalez, L. Sentis, and Y . Zhu, “Deep imitation learning for humanoid loco-manipulation through human teleoperation,” inHumanoids, 2023

    2023

  28. [36]

    Homie: Humanoid loco-manipulation with isomorphic exoskeleton cockpit,

    Q. Ben, F. Jia, J. Zeng, J. Dong, D. Lin, and J. Pang, “Homie: Humanoid loco-manipulation with isomorphic exoskeleton cockpit,” RSS, 2025

    2025

  29. [37]

    Diffusion meets dagger: Supercharging eye-in-hand imitation learning,

    X. Zhang, M. Chang, P. Kumar, and S. Gupta, “Diffusion meets dagger: Supercharging eye-in-hand imitation learning,” inRSS, 2024

    2024

  30. [38]

    View-invariant policy learning via zero-shot novel view synthesis,

    S. Tian, B. Wulfe, K. Sargent, K. Liu, S. Zakharov, V . C. Guizilini, and J. Wu, “View-invariant policy learning via zero-shot novel view synthesis,” inCoRL, 2025

    2025

  31. [39]

    Rovi-aug: Robot and viewpoint augmentation for cross-embodiment robot learning,

    L. Y . Chen, C. Xu, K. Dharmarajan, Z. Irshad, R. Cheng, K. Keutzer, M. Tomizuka, Q. Vuong, and K. Goldberg, “Rovi-aug: Robot and viewpoint augmentation for cross-embodiment robot learning,” in Conference on Robot Learning (CoRL), 2024

    2024

  32. [40]

    Cacti: A framework for scalable multi-task multi-scene visual imitation learning,

    Z. Mandi, H. Bharadhwaj, V . Moens, S. Song, A. Rajeswaran, and V . Kumar, “Cacti: A framework for scalable multi-task multi-scene visual imitation learning,”arXiv preprint arXiv:2212.05711, 2022

    2022 arXiv

  33. [41]

    Genaug: Retargeting behaviors to unseen situations via generative augmentation,

    Z. Chen, S. Kiami, A. Gupta, and V . Kumar, “Genaug: Retargeting behaviors to unseen situations via generative augmentation,”RSS, 2023

    2023

  34. [42]

    Scaling robot learning with semantically imagined experience,

    T. Yu, T. Xiao, A. Stone, J. Tompson, A. Brohan, S. Wang, J. Singh, C. Tan, J. Peralta, B. Ichteret al., “Scaling robot learning with semantically imagined experience,”RSS, 2023

    2023

  35. [43]

    Mimicgen: A data generation system for scalable robot learning using human demonstrations,

    A. Mandlekar, S. Nasiriany, B. Wen, I. Akinola, Y . Narang, L. Fan, Y . Zhu, and D. Fox, “Mimicgen: A data generation system for scalable robot learning using human demonstrations,” inCoRL, 2023

    2023

  36. [44]

    Dexmimicgen: Automated data generation for bimanual dexterous manipulation via imitation learning,

    Z. Jiang, Y . Xie, K. Lin, Z. Xu, W. Wan, A. Mandlekar, L. Fan, and Y . Zhu, “Dexmimicgen: Automated data generation for bimanual dexterous manipulation via imitation learning,”ICRA, 2025

    2025

  37. [45]

    Skillmimicgen: Automated demonstration generation for efficient skill learning and deployment,

    C. Garrett, A. Mandlekar, B. Wen, and D. Fox, “Skillmimicgen: Automated demonstration generation for efficient skill learning and deployment,” inConference on Robot Learning (CoRL), 2024

    2024

  38. [46]

    Neural state machine for character-scene interactions,

    S. Starke, H. Zhang, T. Komura, and J. Saito, “Neural state machine for character-scene interactions,”ACM Transactions on Graphics, vol. 38, no. 6, p. 178, 2019

    2019

  39. [47]

    Mink: Python inverse kinematics based on MuJoCo,

    K. Zakka, “Mink: Python inverse kinematics based on MuJoCo,” May 2025. [Online]. Available: https://github.com/kevinzakka/mink

    2025

  40. [48]

    Meshing piecewise linear complexes by constrained delaunay tetrahedralizations,

    H. Si and K. G ¨artner, “Meshing piecewise linear complexes by constrained delaunay tetrahedralizations,” inProceedings of the 14th international meshing roundtable. Springer, 2005, pp. 147–163

    2005

  41. [49]

    Differential coordinates for local mesh morphing and deformation,

    M. Alexa, “Differential coordinates for local mesh morphing and deformation,”The Visual Computer, vol. 19, no. 2, pp. 105–114, 2003

    2003

  42. [50]

    Large mesh deformation using the volumetric graph laplacian,

    K. Zhou, J. Huang, J. Snyder, X. Liu, H. Bao, B. Guo, and H.-Y . Shum, “Large mesh deformation using the volumetric graph laplacian,” in ACM SIGGRAPH 2005 Papers. ACM, 2005, pp. 496–503

    2005

  43. [51]

    Drake: Model-based design and verification for robotics,

    R. Tedrake and the Drake Development Team, “Drake: Model-based design and verification for robotics,” 2019

    2019

  44. [52]

    Planning with attitude,

    B. E. Jackson, K. Tracy, and Z. Manchester, “Planning with attitude,” IEEE Robotics and Automation Letters, 2021

    2021

  45. [53]

    Atlas Gets a Grip,

    Boston Dynamics, “Atlas Gets a Grip,” YouTube, available: https:// www.youtube.com/watch?v=-e1 QhJ1EhQ

  46. [54]

    SMPL: A skinned multi-person linear model,

    M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black, “SMPL: A skinned multi-person linear model,”ACM Trans. Graphics (Proc. SIGGRAPH Asia), vol. 34, no. 6, pp. 248:1–248:16, Oct. 2015. APPENDIX A. Different Sources of Human Motion Data Human motion datasets contain ...

    2015

  47. [55]

    a) Model Fitting (PHC, VideoMimic):This strategy fits a scaled SMPL model to the robot’s morphology

    SMPL Data:To handle data from parametric models like SMPL, methods typically follow one of two strategies: fitting the model to the robot’s morphology or directly scaling the human’s keypoints. a) Model Fitting (PHC, VideoMimic):This strategy fits a scaled SMPL model to the ro...

  48. [56]

    Skeleton Hierarchy Data:For formats like BVH, keypoint positions are derived from the skeleton’s forward kinematicsf skeleton. This data is then typically scaled to the robot’s size using the height ratio: psource t,i = hrobot hdemo ·f skeleton i (qdemo t ).(8) A key distincti...

  49. [57]

    It applies gradient descent to minimize the error between the source keypoint positions and the robot’s keypoint positions over the entire trajectory, as shown in Alg

    PHC:PHC formulates retargeting as a large-scale trajectory-wise optimization problem. It applies gradient descent to minimize the error between the source keypoint positions and the robot’s keypoint positions over the entire trajectory, as shown in Alg. 2

  50. [58]

    GMR:GMR performs retargeting by solving an inverse kinematics (IK) problem at each frame (3). At each timestep, GMR finds the robot configurationq t that matches the Algorithm 2Retarget Robot Motion (PHC) Require:Robot urdf, source keypoint positions{p source 0:T,i } Ensure:q ...

  51. [59]

    VideoMimic:VideoMimic jointly optimizes for the robot motionq 0:T and SMPL per-link scaling factorβover the entire trajectory. The primary objective is to preserve the scaled pairwise distance and orientation between each keypoint pair(i,j): Lpairwise = ∑ t,i∈N(j) ∥βi j·(p dem...

  52. [60]

    Then, it solves a separate IK problem to recover joint angles that best match the intermediate keypoints: q⋆ t =argmin qt ∑ i ∥f i(qt )−p ⋆ t,i ∥2

    IMMA Multi-stage Optimization:IMMA relies on a complex, multi-stage pipeline: first, it optimizes the interme- diate robot keypoint positions to warp the interaction mesh Method Optimization Type Primary Objective Preprocessing Data Formats PHC Trajectory-wise Optimization Key...

  53. [61]

    This sequential and fragmented approach produces dynamically consistent motions but fails to consider crucial kinematic constraints like joint and velocity limits

    (13) In later stages, additional hard constraints on the feet and waist are imposed to prevent foot slipping and ensure dy- namic balancing. This sequential and fragmented approach produces dynamically consistent motions but fails to consider crucial kinematic constraints like...

  54. [62]

    Let the original trajectory be de- noted by(p ob j(t),θ ob j(t))

    Augmented Object Trajectory:To generate a perturbed object trajectory, we introduce a transient offset that decays exponentially over time. Let the original trajectory be de- noted by(p ob j(t),θ ob j(t)). We define an initial positional offset∆p ob j and rotational offset∆θ o...

  55. [63]

    This ensures that the Laplacian coordinates, which encode relative spatial relationships, are invariant to the object’s global rotation and translation

    Interaction Mesh Construction in Object Frame:For robot-object interactions, it is crucial to construct the interac- tion mesh in the object’s local coordinate frame. This ensures that the Laplacian coordinates, which encode relative spatial relationships, are invariant to the...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.