Pith. sign in

REVIEW 5 major objections 5 minor 28 cited by

TWIST: Teleoperated Whole-Body Imitation System

T0 review · 5 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single neural network controller can teleoperate a humanoid robot through whole-body skills by tracking live human motion.

desk verdict A credible integrated teleoperation system with impressive demos, but the real-time claim rests on ablations that never inject the measured 0.9s latency. read the letter →

arxiv 2505.02833 v1 pith:NKZNQTGU submitted 2025-05-05 cs.RO cs.CVcs.LG

Yanjie Ze , Zixuan Chen , João Pedro Araújo , Zi-ang Cao , Xue Bin Peng , Jiajun Wu , C. Karen Liu This is my paper · ORCID
classification cs.ROcs.CVcs.LG
keywords humanoidteleoperationwhole-bodycontrolmotionretargetingreinforcementlearningbehaviorcloningteacher-studentdistillationcapturesim-to-real
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that teleoperating a humanoid robot in a coordinated whole-body fashion can be reduced to a real-time motion retargeting and tracking problem, solved by one neural network controller trained in simulation. It argues that a two-stage teacher–student pipeline—where a teacher sees future motion frames and a student sees only the current frame—combined with reinforcement learning plus behavior cloning, produces a policy that tracks diverse live human motions without re-tuning. Adding a small amount of in-house, noisy motion-capture data and training with end-effector perturbations closes the gap between clean offline data and real teleoperation. If true, the system gives robots a practical way to imitate human whole-body behavior, spanning manipulation, kicking, locomotion, and expressive movement.

What carries the argument

The load-bearing mechanism is a teacher–student distillation pair. The teacher policy is trained with proximal policy optimization (PPO) on a tracking reward that includes future reference motion frames spanning two seconds, which lets it plan smooth motions; the deployable student policy observes only the current retargeted frame plus proprioception and is optimized with a hybrid loss $L(\pi_{\mathrm{stu}}) = L_{\mathrm{RL}}(\pi_{\mathrm{stu}}) + \lambda D_{\mathrm{KL}}(\pi_{\mathrm{stu}} \| \pi_{\mathrm{tea}})$, with $\lambda$ annealed during training. Around this sits an online retargeter that converts 120 Hz motion-capture data into 50 Hz robot joint-position and root-velocity targets, optimized to jointly fit 3D positions and orientations, and a dataset mixture of offline retargeted public motion clips and small in-house clips captured under realistic teleoperation noise.

What would settle it

Add an extra 0.5 seconds of latency to the retargeted reference stream during a live teleoperation session and measure the robot's tracking error; if the robot cannot keep balance or the error grows well beyond the tracking reward's acceptable margin, the zero-shot robustness claim fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that a single unified whole-body controller can drive a 29-degree-of-freedom humanoid robot to track retargeted human motion in real time, producing coordinated whole-body skills that earlier teleoperation systems split into separate locomotion and manipulation controllers. The core result is the two-stage training recipe: a privileged reinforcement-learning teacher trained with access to future reference frames is distilled into a deployable student via an RL plus behavior-cloning objective, and the student, using only proprioception and one current reference frame, tracks unseen motions well enough for real-world use. The paper further reports that mixing roughly 150 in-house motion-capture clips recorded under online retargeting conditions into a large offline dataset markedly improves tracking, and that forcing contact during training via end-effector perturbations is needed for tasks that involve lifting or pushing objects.

Load-bearing premise

The whole system rests on the assumption that the online retargeter can convert noisy 120 Hz motion-capture frames into smooth, physically feasible 50 Hz robot joint targets, and that the single-frame student policy can track those live targets without any additional real-world training despite the measured end-to-end delay of about 0.9 seconds.

Editorial extensions

If this is right

  • A single controller, rather than task-specific modules, can serve as the low-level tracking layer for many humanoid skills; task variation enters through the human's motion rather than through controller switching.
  • Whole-body teleoperation data collected this way is a candidate training source for higher-level visuomotor or language-conditioned policies.
  • Adding a small amount of noisy, domain-matched motion data yields large tracking gains, suggesting data-quality matching can matter more than dataset size.
  • The same training pipeline transfers to a second humanoid platform in simulation, indicating the method is not tied to one robot model.
  • Contact-rich tasks require explicit training-time perturbations; without them the policy drifts during stationary poses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the student tracks only a single current reference frame, the same controller could likely be driven by a predictive model of future human motion, which would attack the measured 0.9 second delay without changing the policy.
  • The retargeting step is the dominant latency component; a faster or learned retargeter could cut end-to-end lag substantially and is a concrete, testable improvement.
  • Replacing the motion-capture suit with monocular RGB pose estimation would directly test whether the controller is robust to added pose noise, while addressing the paper's stated portability limitation.
  • The reported failure mode of motor overheating during crouching suggests that thermal limits, not control policy limits, may be the near-term bottleneck for whole-body data collection.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper presents TWIST, a teleoperation system that streams human motion-capture data at 120 Hz, retargets it to humanoid joint targets at 50 Hz, and controls a 29-DoF Unitree G1 humanoid with a single neural network policy trained in simulation using a two-stage teacher-student RL+BC pipeline. The system is claimed to achieve whole-body manipulation, legged manipulation, locomotion, and expressive motions in real time. The paper also reports ablations on the RL+BC objective, in-house MoCap data, end-effector perturbations, and online retargeting, together with a sim-to-sim transfer to the Booster T1 robot.

Significance. If the real-world capabilities hold, TWIST would be a notable system contribution: a single unified controller covering diverse whole-body skills, with a plausible and well-structured training recipe. The paper's strengths include the systematic ablation organization, the sim-to-sim generalization check on a second robot, and the concrete design techniques (future-frame teacher, RL+BC distillation, in-house online MoCap data, end-effector perturbations). However, the central real-time teleoperation claim currently rests on qualitative video demonstrations and on quantitative evaluations that omit the measured 0.9 s system latency. The scientific value is therefore conditional on additional closed-loop and statistical evidence.

major comments (5)
  1. [Section 3.2, Section 4.3] The student policy is trained to track the current (undelayed) retargeted reference frame, with the KL term in Eq. (1) pushing it toward a teacher that sees future frames, while live teleoperation is reported to have an end-to-end delay of roughly 0.9 s (0.7 s retargeting + 0.2 s inference). The quantitative evaluations in Figures 6-8 use recorded references without injecting this delay, so the paper never verifies that the single-frame student policy remains stable and accurate when its input is a 0.7-0.9 s stale pose. Since the videos cannot separate the controller's tracking ability from the human operator's compensatory behavior, the central 'real-time teleoperation' claim is not yet supported. Please add a latency-injection evaluation, for example shifting the reference stream by the measured delays during simulated rollouts or replaying delayed references, and report closed-loop teleoperation metrics.
  2. [Section 4.2, Figures 6 and 8] The ablation plots report single trajectories or single summary values without error bars, standard deviations, or the number of seeds and rollouts. Statements such as 'significantly outperforms' (Key Finding 1) and 'substantially reduces tracking errors' (Key Finding 2) therefore lack statistical support. Report mean +/- standard deviation over at least three training seeds and specify the number of test clips and rollouts per condition.
  3. [Section 4.2, Key Finding 2] The test set used to demonstrate the benefit of in-house MoCap data is collected with the same online MoCap and retargeting setup that motivated adding these data in the first place. This confound makes it unclear whether the improvement reflects better generalization or simply a better match to the test-time data distribution. Please add evaluation on independently collected held-out sessions, for example different operators or different days, or explicitly quantify the distribution shift.
  4. [Section 4.1 and Figure 2] The main real-world results are presented only as qualitative video demonstrations. For a system whose headline contribution is real-world teleoperation, quantitative evaluation is needed: per-task success rates with task definitions and number of trials, measured tracking error during closed-loop teleoperation, and ideally a comparison against a latency-compensated baseline. Without these, the claim that the controller, rather than the human operator's real-time compensation, is responsible for the demonstrated skills is not quantitatively assessed.
  5. [Reference [11]] The online retargeter improvement (Section 3.1 and Figure 8, left) is attributed to [11], cited as 'A. Authors ... Anonymous submission, under review'. This citation is not verifiable by readers, and the manuscript does not describe the optimization objective in enough detail to reproduce the improvement. Please either include the full retargeting formulation in the paper or replace the anonymous citation with a public, citable version.
minor comments (5)
  1. [Figure 6 caption] The caption contains the typo 'unsessen motions'; it should read 'unseen motions'.
  2. [Figure 7 (left)] The axes of the real-world rollout curves are not labeled; please add axis labels, units, and a legend so the curves can be interpreted.
  3. [Table 1] The 'Feet Air Time 5.0' entry is listed under 'Penalty Terms' but has a positive weight and appears to be a reward term; please relocate or relabel it.
  4. [Section 4.2, Key Finding 1] The heading 'RL+BC >> RL >> BC (DAgger)' conflates behavior cloning with DAgger, which is an online interactive imitation algorithm; please clarify the exact baselines compared.
  5. [Section 3.3] The phrase 'on an Nvidia RTX 4090 GPU' should be 'on an Nvidia RTX 4090 GPU'; also note that policy inference is reported at 50 Hz, so stating the GPU used is helpful only if the measurement method is given.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TWIST's claims rest on held-out ablations and real-robot deployment; the latency mismatch is a robustness risk, not circularity.

full rationale

TWIST's derivation chain is empirical: stage 1 curates humanoid motion data by retargeting, stage 2 trains a teacher-student whole-body controller with RL+BC, and stage 3 deploys the student policy zero-shot on a real humanoid. Each claimed improvement—privileged future frames, in-house MoCap data, end-effector perturbations, and the improved online retargeter—is supported by quantitative ablations on a held-out 50-clip MoCap test set not used in training, with additional sim-to-sim transfer to MuJoCo and real-robot demonstrations. None of these evaluations reduces to its input by construction: the student policy's tracking error is measured against retargeted references that are not identical to the training targets, and the in-house-data ablation compares controllers trained with and without that data on the same held-out test set. The measured 0.9 s teleoperation delay is a genuine correctness/robustness risk, because training uses synchronous reference frames while live deployment observes delayed frames, but that is an unverified transfer assumption rather than a circular derivation. Reference [11] is an anonymous under-review citation used for the online retargeter baseline; even if it overlaps with the authors, it is not the load-bearing step of the central argument, so it does not raise the circularity score.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central result is an empirical system built from hand-chosen design choices and domain assumptions. Reward weights, domain randomization ranges, the teacher horizon, the KL schedule, and the perturbation range are all selected by the authors, and zero-shot sim-to-real transfer relies on the simulator being representative. No new physical entities are introduced.

free parameters (5)
  • Reward weights (Table 1) = hand-chosen
    Tracking and penalty reward weights are set by the authors in Table 1; the learned behavior quality depends on these choices and no sensitivity study is given.
  • Domain randomization ranges (Table 2) = hand-chosen ranges
    Mass, friction, motor strength, gravity, and push ranges in Table 2 are selected manually; zero-shot sim-to-real transfer relies on these values being representative.
  • Teacher future horizon = 2 seconds
    The privileged teacher policy observes future reference motions spanning 2 seconds; this window size is hand-picked and affects smoothness.
  • KL weight lambda schedule = gradually reduced
    Equation (1) uses a KL divergence weight that is gradually reduced during training; the schedule is not specified quantitatively.
  • End-effector perturbation range = [0, 20] N
    Training applies pushes to end-effectors up to 20 N in simulation, claimed to improve contact-rich tasks; this range is hand-chosen.
assumptions (4)
  • domain assumption Human motion capture data can be retargeted to feasible humanoid joint trajectories by the offline and online retargeters.
    Section 3.1 says retargeting is needed because of the embodiment gap; if retargeted motions are infeasible, the tracking objective cannot be satisfied.
  • domain assumption A policy trained in simulation with the domain randomization ranges in Table 2 transfers zero-shot to the real Unitree G1.
    Section 3.3 states the student policy can be zero-shot deployed because of careful domain randomization; this assumes the simulator dynamics are a faithful enough model.
  • domain assumption The tracking reward in Table 1 is a sufficient objective for producing contact-rich whole-body behaviors without explicitly modeling contact forces.
    Section 3.2 and Key Finding 3 acknowledge force tasks are out-of-distribution and are addressed via end-effector perturbations, not by a force-aware reward.
  • domain assumption Distilling the privileged teacher into a single-frame student policy via PPO plus KL divergence (Equation 1) preserves enough information for zero-shot deployment.
    Section 3.2 describes the distillation; this relies on standard teacher-student assumptions about the student's capacity and observational overlap.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TWIST: Teleoperated Whole-Body Imitation System." pith.science (2026). https://pith.science/paper/NKZNQTGU

@misc{pith2026250502833,
  author       = {Pith},
  title        = {Pith review of: TWIST: Teleoperated Whole-Body Imitation System},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NKZNQTGU}},
  note         = {Machine review of arXiv:2505.02833}
}
read the original abstract

Teleoperating humanoid robots in a whole-body manner marks a fundamental step toward developing general-purpose robotic intelligence, with human motion providing an ideal interface for controlling all degrees of freedom. Yet, most current humanoid teleoperation systems fall short of enabling coordinated whole-body behavior, typically limiting themselves to isolated locomotion or manipulation tasks. We present the Teleoperated Whole-Body Imitation System (TWIST), a system for humanoid teleoperation through whole-body motion imitation. We first generate reference motion clips by retargeting human motion capture data to the humanoid robot. We then develop a robust, adaptive, and responsive whole-body controller using a combination of reinforcement learning and behavior cloning (RL+BC). Through systematic analysis, we demonstrate how incorporating privileged future motion frames and real-world motion capture (MoCap) data improves tracking accuracy. TWIST enables real-world humanoid robots to achieve unprecedented, versatile, and coordinated whole-body motor skills--spanning whole-body manipulation, legged manipulation, locomotion, and expressive movement--using a single unified neural network controller. Our project website: https://humanoid-teleop.github.io

Figures

Figures reproduced from arXiv: 2505.02833 by the authors.

Figure 1
Figure 1. The Teleoperated Whole-Body Imitation System (TWIST) is a system that teleoperates humanoid robots with real-time whole-body human data and a single neural network controller. TWIST achieves versatile, coordinated, whole-body skills that are not present in previous works. Abstract: Teleoperating humanoid robots in a whole-body manner marks a fun￾damental step toward developing general-purpose robotic intelligence, w… view at source ↗
Figure 2
Figure 2. The Teleoperated Whole-Body Imitation System (TWIST) presents versatile, coordinated, and human-like whole-body skills on real-world humanoid robots. Our robot can perform whole-body manip￾ulation (e.g., lifting boxes from the ground), legged manipulation (e.g., kicking the football), locomotion (e.g., walking sideways), and expressive motions (e.g., Waltz dance). More videos: humanoid-teleop.github.io 2 [PITH_FULL… view at source ↗
Figure 3
Figure 3. The Teleoperated Whole-Body Imitation System (TWIST) consists of 3 stages: 1) curating a humanoid motion dataset by retargeting Internet human data and our MoCap data, 2) training a single whole￾body controller in simulation, 3) teleoperating real-world humanoid robots with MoCap devices. 2 Related Works Learning-Based Whole-Body Control for Humanoid Robots. In recent years, learning-based methods, particularly sim-… view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: Teleoperation delay is roughly measured by the video, around 0.9 seconds. upcoming tracking goals, further leading to smooth locomotion gaits. Besides, we choose to track joint positions and root velocities expressed in the robot’s local frame rather than the world fra…
Figure 6
Figure 6. Figure 6: (left) Tracking errors of different controllers, measured on our MoCap test data. (right) Controller behaviors. Our RL+BC controller produces smooth and robust behaviors [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: (left) Rollout curves in the real world when the robot holds a box. (right) Rollout curves in MuJuCo from different controllers when tracking MoCap data. 4.1 Main Results on Whole-Body Humanoid Teleoperation We showcase the teleoperation capabilities of TWIST on Unitre…
Figure 8
Figure 8. Figure 8: (left) Sum of tracking error metrics on different online retargeters. (right) Tracking errors across different body parts. The feet exhibit the largest errors, highlighting the greater difficulty in accurately track￾ing lower-body movements. achieves superior tracking …
Figure 9
Figure 9. Figure 9: (a) Extreme reachability by TWIST. (b) Failures caused by motor overheating. 5 Conclusion In this work, we introduce the Teleoperated Whole-Body Imitation System (TWIST), a system that teleoperates real-world humanoid robots using whole-body human motion data captured …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 28 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. RoboReact: Agentic Skill Distillation from Generated Egocentric Videos for Generalizable Whole-Body Manipulation

    cs.RO 2026-08 conditional novelty 7.0 of 10

    From a single egocentric RGB-D frame and a language instruction, RoboReact generates a human manipulation video, distills it into object-relative keyframe skills, refines them through a vision-language-model trial loo...

  2. RynnWorld-Teleop: An Action-Conditioned World Model for Digital Teleoperation

    cs.RO 2026-07 conditional novelty 6.5 of 10

    A real-time robot-centric video world model driven by depth-aware hand skeletons generates imitation-learning trajectories that support zero-shot real-robot transfer and improve policies when mixed with real data.

  3. HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark

    cs.RO 2026-08 conditional novelty 6.0 of 10

    HumanTracker introduces a 153-hour categorized humanoid tracking benchmark and a preference-trained metric, HumanScore, that agrees with human judgments better than kinematic error metrics.

  4. Learning Panorama-Aware VLA for Mobile Manipulation with Whole-Body Teleoperation

    cs.RO 2026-08 conditional novelty 6.0 of 10

    Adding a panoramic camera feed to a vision-language-action policy raises end-to-end success on four real-world mobile two-arm tasks from 30% to 73%.

  5. Teleopit: A Full-Embodiment Humanoid Teleoperation System

    cs.RO 2026-08 conditional novelty 6.0 of 10

    Teleopit combines VR body, hand, and head tracking with a learned whole-body tracker and a cross-hand retargeter, and teleop-collected demos train ACT and GR00T policies to around 90 to 95 percent success on a humanoi...

  6. A Change of Frame Makes Balance Observable: Distillation-Free Humanoid Single-Leg Stance

    cs.RO 2026-08 conditional novelty 6.0 of 10

    A support-relative dynamic capture-point observation, reconstructible without base linear velocity, lets a humanoid policy hold clean single-leg balance at 86/90 in simulation and deploy on a Unitree G1 without distillation.

  7. What Matters in Humanoid General Motion Tracking? An Empirical Study

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A controlled ablation of humanoid motion-tracking pipelines shows that explicit reference joint velocities and a short observation history improve tracking, while residual actions and teacher-student training yield on...

  8. Scaling Behavior Foundation Model for Humanoid Robots

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A scaling recipe for humanoid behavior foundation models—global-frame motion tracking, on-policy data quantity plus reference diversity, and a transformer with hyperspherical latents—cuts global tracking error by roug...

  9. EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal

    cs.RO 2026-07 conditional novelty 6.0 of 10

    EgoHTR is a 55-sequence, 150k-frame egocentric 4D human-terrain dataset with a reconstruction pipeline, MoCap-validated benchmark, and perceptive locomotion policies deployed on a Unitree G1.

  10. ContactMimic: Humanoid Object Interaction via Contact Control

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A humanoid tracking policy is trained with contact-following rewards and trajectory augmentation to decouple physical contact from keypoint geometry, enabling runtime contact control.

  11. ThorArena: Benchmarking Humanoid Physical Interaction with Human Motion-Force Demonstrations

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A force-aware humanoid benchmark pairs synchronized human motion-force data with simulation-based force replay to evaluate whole-body control policies under realistic physical disturbances.

  12. Kinodynamic Motion Retargeting for Humanoid Locomotion via Multi-Contact Whole-Body Trajectory Optimization

    cs.RO 2026-03 conditional novelty 6.0 of 10

    A physics-aware motion-retargeting pipeline that uses ground-reaction-force-derived heel-toe contacts produces dynamically feasible humanoid references and improves downstream imitation learning.

  13. Distinguishing Imitation Error from Intrinsic Motion Learning Difficulty

    cs.GR 2025-12 conditional novelty 6.0 of 10

    A physics-based score (MDS) predicts how hard a motion is for a humanoid to imitate by measuring how much joint torques must change under small pose perturbations.

  14. Thor: Towards Human-Level Whole-Body Reactions for Intense Contact-Rich Environments

    cs.RO 2025-10 conditional novelty 6.0 of 10

    A decoupled whole-body RL policy with a force-based lean reward enables a Unitree G1 humanoid to pull with up to 167.7 N, beating prior controllers by 69–75%.

  15. PHUMA: Physically Reliable Humanoid Locomotion Dataset

    cs.RO 2025-10 conditional novelty 6.0 of 10

    PHUMA is a curated 73-hour humanoid locomotion corpus whose physical-reliability metrics are partly defined by the same losses used to optimize it, and whose imitation success claims are confounded by in-distribution ...

  16. A Scalable Whole-body Motion Transfer via Implicit Kinodynamic Motion Retargeting

    cs.RO 2025-09 conditional novelty 6.0 of 10

    A neural retargeting pipeline maps human motion to humanoid robot motion at 5000+ frames per second using a shared latent space and physics-based fine-tuning, filtering noise and producing physically feasible trajectories.

  17. HITTER: A HumanoId Table TEnnis Robot via Hierarchical Planning and Learning

    cs.RO 2025-08 conditional novelty 6.0 of 10

    A humanoid robot with a model-based planner and a reinforcement-learning controller returns table tennis balls and sustained a 106-shot rally against a human.

  18. KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skills

    cs.RO 2025-06 conditional novelty 6.0 of 10

    A robot control method that adaptively tightens motion-tracking reward tolerances achieves lower tracking errors on dynamic skills and transfers zero-shot to a real humanoid.

  19. SkillBlender: Towards Versatile Humanoid Whole-Body Loco-Manipulation via Skill Blending

    cs.RO 2025-06 conditional novelty 6.0 of 10

    SkillBlender pretrains reusable goal-conditioned skills and blends them with softmax per-joint weights to solve simulated humanoid loco-manipulation tasks with one or two reward terms.

  20. Event-Based Upper-Body Humanoid Teleoperation Under Challenging Illumination

    cs.RO 2026-07 conditional novelty 5.0 of 10

    An event-camera-based pipeline for upper-body human-to-humanoid teleoperation achieves 23–34 ms end-to-end latency and more robust tracking than RGB under low light, backlight, and fast motion.

  21. Semantic Audio-driven Understanding for Dynamic Humanoid Whole Body Control

    cs.RO 2026-07 conditional novelty 5.0 of 10

    A multi-modal audio router maps streaming music and speech to imitation-learned whole-body policies for a Unitree G1 humanoid, achieving 84.8% chunk-level retrieval accuracy in simulation.

  22. The Latent Color Subspace: Emergent Order in High-Dimensional Chaos

    cs.LG 2026-03 unverdicted novelty 5.0 of 10

    FLUX.1’s VAE latent space contains an interpretable Hue–Saturation–Lightness structure that enables training-free color prediction and control via closed-form latent edits.

  23. RobotDancing: Residual-Action Reinforcement Learning Enables Robust Long-Horizon Humanoid Motion Tracking

    cs.RO 2025-09 conditional novelty 5.0 of 10

    Residual-action reinforcement learning, with selective corrections on hip and knee pitch joints, enables zero-shot long-horizon dance tracking on real humanoid robots.

  24. HERMES: Human-to-Robot Embodied Learning from Multi-Source Motion Data for Mobile Dexterous Manipulation

    cs.RO 2025-08 conditional novelty 5.0 of 10

    HERMES converts a single human motion demonstration into a deployable mobile bimanual dexterous manipulation policy, using RL, depth-image distillation, and closed-loop PnP pose refinement.

  25. Switch4EAI: Leveraging Console Game Platform for Benchmarking Robotic Athletics

    cs.RO 2025-08 conditional novelty 5.0 of 10

    Switch4EAI uses Just Dance on a Nintendo Switch as a benchmark: a Unitree G1 robot scored 5,707 versus a human's 9,361 on three routines.

  26. GBC: Generalized Behavior-Cloning Framework for Whole-Body Humanoid Imitation

    cs.RO 2025-08 conditional novelty 5.0 of 10

    GBC unifies MoCap retargeting and imitation learning into one framework that trains whole-body humanoid policies across multiple robot morphologies in simulation.

  27. Beyond Topology: A Morphological Symmetry Graph Representation for Locomotion Policy Learning

    cs.RO 2025-11 conditional novelty 4.0 of 10

    Encoding the robot's left-right symmetry and joint connectivity directly into a policy network enforces mirror symmetry by construction, improving out-of-distribution command tracking on quadruped robots.

  28. Feature-Based vs. GAN-Based Learning from Demonstrations: When and Why

    cs.LG 2025-07 conditional novelty 3.0 of 10

    Feature-based and GAN-based imitation learning should be selected by task priorities (fidelity, diversity, interpretability, adaptability), not by paradigm loyalty.

Reference graph

Works this paper leans on

49 extracted references · 18 canonical work pages · cited by 28 Pith papers

  1. [11]

    A. Authors. Mind the mapping: How motion retargeting choices impact sim2real success. Anonymous submission, 2025. Under review

    work page 2025

  2. [1]

    Penco, N

    L. Penco, N. Scianca, V . Modugno, L. Lanari, G. Oriolo, and S. Ivaldi. A multimode teleop- eration framework for humanoid loco-manipulation: An application for the icub robot. IEEE Robotics & Automation Magazine, 26(4):73–82, 2019. 9

    work page 2019

  3. [2]

    Dafarra, U

    S. Dafarra, U. Pattacini, G. Romualdi, L. Rapetti, R. Grieco, K. Darvish, G. Milani, E. Valli, I. Sorrentino, P. M. Viceconte, et al. icub3 avatar system: Enabling remote fully immersive embodiment of humanoid robots. Science Robotics, 9(86):eadh3834, 2024

    2024

  4. [3]

    Penco, B

    L. Penco, B. Cl ´ement, V . Modugno, E. M. Hoffman, G. Nava, D. Pucci, N. G. Tsagarakis, J.-B. Mouret, and S. Ivaldi. Robust real-time whole-body motion retargeting from human to humanoid. In 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Hu- manoids), pages 425–432. IEEE, 2018

    work page 2018

  5. [4]

    Darvish, L

    K. Darvish, L. Penco, J. Ramos, R. Cisneros, J. Pratt, E. Yoshida, S. Ivaldi, and D. Pucci. Teleoperation of humanoid robots: A survey. IEEE Transactions on Robotics , 39(3):1706– 1727, 2023

    2023

  6. [5]

    Darvish, Y

    K. Darvish, Y . Tirupachuri, G. Romualdi, L. Rapetti, D. Ferigo, F. J. A. Chavez, and D. Pucci. Whole-body geometric retargeting for humanoid robots. In2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids), pages 679–686. IEEE, 2019

    work page 2019

  7. [6]

    Z. He, K. Lei, Y . Ze, K. Sreenath, Z. Li, and H. Xu. Learning visual quadrupedal loco- manipulation from demonstrations. arXiv preprint arXiv:2403.20328, 2024

    arXiv 2024

  8. [7]

    Z. Fu, Q. Zhao, Q. Wu, G. Wetzstein, and C. Finn. Humanplus: Humanoid shadowing and imitation from humans. In Conference on Robot Learning (CoRL), 2024

    2024

Show all 49 references
  1. [8]

    T. He, Z. Luo, X. He, W. Xiao, C. Zhang, W. Zhang, K. Kitani, C. Liu, and G. Shi. Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning. arXiv preprint arXiv:2406.08858, 2024

    2024 arXiv

  2. [9]

    T. He, W. Xiao, T. Lin, Z. Luo, Z. Xu, Z. Jiang, J. Kautz, C. Liu, G. Shi, X. Wang, et al. Hover: Versatile neural whole-body controller for humanoid robots.arXiv preprint arXiv:2410.21229, 2024

    2024 arXiv

  3. [10]

    Koenemann, F

    J. Koenemann, F. Burget, and M. Bennewitz. Real-time imitation of human whole-body mo- tions by humanoids. In 2014 IEEE International Conference on Robotics and Automation (ICRA), pages 2806–2812. IEEE, 2014

    2014

  4. [12]

    Z. Chen, X. He, Y .-J. Wang, Q. Liao, Y . Ze, Z. Li, S. S. Sastry, J. Wu, K. Sreenath, S. Gupta, et al. Learning smooth humanoid locomotion through lipschitz-constrained policies. arXiv preprint arXiv:2410.11825, 2024

    2024 arXiv

  5. [13]

    Gu, Y .-J

    X. Gu, Y .-J. Wang, X. Zhu, C. Shi, Y . Guo, Y . Liu, and J. Chen. Advancing humanoid loco- motion: Mastering challenging terrains with denoising world model learning. arXiv preprint arXiv:2408.14472, 2024

    2024 arXiv

  6. [14]

    Radosavovic, S

    I. Radosavovic, S. Kamat, T. Darrell, and J. Malik. Learning humanoid locomotion over chal- lenging terrain. arXiv preprint arXiv:2410.03654, 2024

    2024 arXiv

  7. [15]

    Radosavovic, T

    I. Radosavovic, T. Xiao, B. Zhang, T. Darrell, J. Malik, and K. Sreenath. Real-world humanoid locomotion with reinforcement learning. Science Robotics, 9(89):eadi9579, 2024

    2024

  8. [16]

    J. Long, W. Yu, Q. Li, Z. Wang, D. Lin, and J. Pang. Learning h-infinity locomotion control. In ArXiv, 2024

    2024

  9. [17]

    Zhang, W

    C. Zhang, W. Xiao, T. He, and G. Shi. Wococo: Learning whole-body humanoid control with sequential contacts, 2024

    2024

  10. [18]

    Zhuang, S

    Z. Zhuang, S. Yao, and H. Zhao. Humanoid parkour learning. arXiv preprint arXiv:2406.10759, 2024. 10

    2024 arXiv

  11. [19]

    Cheng, Y

    X. Cheng, Y . Ji, J. Chen, R. Yang, G. Yang, and X. Wang. Expressive whole-body control for humanoid robots. arXiv preprint arXiv:2402.16796, 2024

    2024 arXiv

  12. [20]

    M. Ji, X. Peng, F. Liu, J. Li, G. Yang, X. Cheng, and X. Wang. Exbody2: Advanced expressive humanoid whole-body control. arXiv preprint arXiv:2412.13196, 2024

    2024 arXiv

  13. [21]

    Y . Xue, W. Dong, M. Liu, W. Zhang, and J. Pang. A unified and general humanoid whole-body controller for fine-grained locomotion. arXiv preprint arXiv:2502.03206, 2025

    2025 arXiv

  14. [22]

    Huang, J

    T. Huang, J. Ren, H. Wang, Z. Wang, Q. Ben, M. Wen, X. Chen, J. Li, and J. Pang. Learning humanoid standing-up control across diverse postures.arXiv preprint arXiv:2502.08378, 2025

    2025 arXiv

  15. [23]

    Zhuang and H

    Z. Zhuang and H. Zhao. Embrace collisions: Humanoid shadowing for deployable contact- agnostics motions. arXiv preprint arXiv:2502.01465, 2025

    2025 arXiv

  16. [24]

    X. He, R. Dong, Z. Chen, and S. Gupta. Learning getting-up policies for real-world humanoid robots. arXiv preprint arXiv:2502.12152, 2025

    2025 arXiv

  17. [25]

    Y . Ze, Z. Chen, W. Wang, T. Chen, X. He, Y . Yuan, X. B. Peng, and J. Wu. Generalizable humanoid manipulation with 3d diffusion policies. arXiv preprint arXiv:2410.10803, 2024

    2024 arXiv

  18. [26]

    Cheng, J

    X. Cheng, J. Li, S. Yang, G. Yang, and X. Wang. Open-television: Teleoperation with immer- sive active visual feedback. arXiv preprint arXiv:2407.01512, 2024

    2024 arXiv

  19. [27]

    C. Lu, X. Cheng, J. Li, S. Yang, M. Ji, C. Yuan, G. Yang, S. Yi, and X. Wang. Mobile- television: Predictive motion priors for humanoid whole-body control. arXiv preprint arXiv:2412.07773, 2024

    2024 arXiv

  20. [28]

    Q. Ben, F. Jia, J. Zeng, J. Dong, D. Lin, and J. Pang. Homie: Humanoid loco-manipulation with isomorphic exoskeleton cockpit. arXiv preprint arXiv:2502.13013, 2025

    2025 arXiv

  21. [29]

    S. Yang, M. Liu, Y . Qin, D. Runyu, L. Jialong, X. Cheng, R. Yang, S. Yi, and X. Wang. Ace: A cross-platfrom visual-exoskeletons for low-cost dexterous teleoperation.arXiv preprint arXiv:240, 2024

    2024

  22. [30]

    Jiang, R

    Y . Jiang, R. Zhang, J. Wong, C. Wang, Y . Ze, H. Yin, C. Gokmen, S. Song, J. Wu, and L. Fei- Fei. Behavior robot suite: Streamlining real-world whole-body manipulation for everyday household activities. arXiv preprint arXiv:2503.05652, 2025

    2025 arXiv

  23. [31]

    K. Shaw, Y . Li, J. Yang, M. K. Srirama, R. Liu, H. Xiong, R. Mendonca, and D. Pathak. Bimanual dexterity for complex tasks. In 8th Annual Conference on Robot Learning, 2024

    2024

  24. [32]

    A. D. Dragan, K. C. Lee, and S. S. Srinivasa. Legibility and predictability of robot motion. In 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI) , pages 301–308. IEEE, 2013

    2013

  25. [33]

    T. He, Z. Luo, W. Xiao, C. Zhang, K. Kitani, C. Liu, and G. Shi. Learning human-to-humanoid real-time whole-body teleoperation. arXiv preprint arXiv:2403.04436, 2024

    2024 arXiv

  26. [34]

    S. Shin, J. Kim, E. Halilaj, and M. J. Black. Wham: Reconstructing world-grounded humans with accurate 3d motion. In Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  27. [35]

    X. B. Peng, P. Abbeel, S. Levine, and M. Van de Panne. Deepmimic: Example-guided deep re- inforcement learning of physics-based character skills.ACM Transactions On Graphics (TOG), 37(4):1–14, 2018

    2018

  28. [36]

    Z. Luo, J. Cao, K. Kitani, W. Xu, et al. Perpetual humanoid control for real-time simulated avatars. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10895–10904, 2023. 11

    2023

  29. [37]

    Mahmood, N

    N. Mahmood, N. Ghorbani, N. F. Troje, G. Pons-Moll, and M. J. Black. Amass: Archive of motion capture as surface shapes. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5442–5451, 2019

    2019

  30. [38]

    J. Li, J. Wu, and C. K. Liu. Object motion guided human motion synthesis. ACM Transactions on Graphics (TOG), 42(6):1–11, 2023

    2023

  31. [39]

    K. Zakka. Mink: Python inverse kinematics based on MuJoCo, July 2024. URL https: //github.com/kevinzakka/mink

    2024

  32. [40]

    Makoviychuk, L

    V . Makoviychuk, L. Wawrzyniak, Y . Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470, 2021

    2021 arXiv

  33. [41]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  34. [42]

    Rudin, D

    N. Rudin, D. Hoeller, P. Reist, and M. Hutter. Learning to walk in minutes using massively parallel deep reinforcement learning. InConference on Robot Learning, pages 91–100. PMLR, 2022

    2022

  35. [43]

    S. Ross, G. Gordon, and D. Bagnell. A reduction of imitation learning and structured predic- tion to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 627–635. JMLR Workshop and Conference Pro-...

    2011

  36. [44]

    Optitrack - motion capture systems, 2025

    NaturalPoint, Inc. Optitrack - motion capture systems, 2025. URL https://www. optitrack.com/. Accessed: 2025-04-22

    2025

  37. [45]

    Todorov, T

    E. Todorov, T. Erez, and Y . Tassa. Mujoco: A physics engine for model-based control. In2012 IEEE/RSJ international conference on intelligent robots and systems, pages 5026–5033. IEEE, 2012

    2012

  38. [46]

    Unitree G1 Humanoid Robot

    Unitree Robotics. Unitree G1 Humanoid Robot. https://www.unitree.com/cn/g1/,

  39. [47]

    Booster t1 humanoid robot

    Booster Robotics. Booster t1 humanoid robot. https://www.boosterobotics.com/, 2025

    2025

  40. [48]

    T. Chu, Y . Zhai, J. Yang, S. Tong, S. Xie, D. Schuurmans, Q. V . Le, S. Levine, and Y . Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501.17161, 2025. 12

    2025 arXiv

  41. [2024]

    Accessed: 2025-04-26

    2025

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.