pith. sign in

arxiv: 2606.29940 · v1 · pith:FKPAU627new · submitted 2026-06-29 · 💻 cs.RO

WARP: Whole-Body Retargeting for Learning from Offline Human Demonstrations

Zhenyang Chen , Chuizheng Kong , Chuye Zhang , Yuanshao Yang , Lawrence Y. Zhu , Shreyas Kousik , Danfei Xu This is my paper

Pith reviewed 2026-06-30 05:56 UTC · model grok-4.3

classification 💻 cs.RO
keywords whole-body retargetinghuman demonstrationsmobile manipulationembodiment gapgeometric solverimitation learningoffline data
0
0 comments X

The pith

WARP retargets offline human demonstrations into precise whole-body robot actions for zero-shot mobile manipulation.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper seeks to demonstrate that human demonstration data can be directly retargeted to robot actions for complex whole-body mobile manipulation tasks. It addresses embodiment gaps that cause inconsistent actions in prior methods by introducing a geometric solver to produce unique, accurate trajectories. A sympathetic reader would care because this removes the need for expensive human-in-the-loop teleoperation, allowing robot learning to scale with easier-to-collect human pose data.

Core claim

WARP is the first framework to achieve zero-shot whole-body mobile manipulation directly from offline human demonstrations by using a closed-form Shoulder-Elbow-Wrist geometric solver for exact end-effector tracking that preserves structural intent, combined with lazy mobile-base control to extract consistent robot trajectories without action multi-modality.

What carries the argument

Closed-form Shoulder-Elbow-Wrist (SEW) geometric solver that computes precise end-effector tracking while preserving whole-body structural intent across embodiment gaps.

Load-bearing premise

The SEW geometric solver extracts precise and unique actions from human poses while preserving intent without creating multi-modal action distributions.

What would settle it

A policy trained via supervised learning on WARP data either converges to consistent behaviors or fails due to inconsistent trajectories in physical robot tests.

Figures

Figures reproduced from arXiv: 2606.29940 by Chuizheng Kong, Chuye Zhang, Danfei Xu, Lawrence Y. Zhu, Shreyas Kousik, Yuanshao Yang, Zhenyang Chen.

Figure 1
Figure 1. Figure 1: Whole-body-Aware Retargeting from human Pose (WARP). (a) We collect human manipula￾tion data offline using VR devices, and WARP retargets this motion into whole-body robot actions, producing human-like trajectories directly usable for policy training. (b) The central difficulty of the offline setting is the absence of online human correction to close the embodiment gap. With no human in the loop to absorb … view at source ↗
Figure 2
Figure 2. Figure 2: Offline retargeting with WARP. (a) Finding the optimal robot torso placement using Adaptive Offset (b) After aligning robot palm to human, robot wrist position can be solved (c) Prioritizing EEF alignment leaves the elbow configuration underconstrained. (d) Given fixed robot wrist and shoulder, we identify the elbow nullspace. Using stereo-sew [24], we find a unique plane intersecting the nullspace circle.… view at source ↗
Figure 3
Figure 3. Figure 3: Our whole￾body mobile manipu￾lation platform. Data Collection and Robot System We collect demonstrations using a sin￾gle Meta Quest headset, without external motion-capture rigs or robots in the loop. Unlike traditional teleoperation and UMI-style interfaces, which com￾press a demonstration down to the end-effector’s spatial pose, we capture the operator’s whole-body motion. The system logs the operator’s … view at source ↗
Figure 4
Figure 4. Figure 4: WARP is precise, consistent, and smooth. (a) WARP matches both the task constraint (end￾effector pose) and the human body pose; baselines satisfy only one—SEW-M loses the end-effector, MINK-EF self-collides and twists the torso. (b) Under a perturbed initial guess with the end-effector target fixed, MINK’s optimization is inconsistent, while WARP’s closed-form solution is identical every time. (c) A respon… view at source ↗
Figure 5
Figure 5. Figure 5: Simulation retargeting results. Left: radar visualization of retargeting feasibility and motion-quality diagnostics. Right: quantitative results for the highlighted variants. All metrics are lower-is-better. Best results are shown in bold; second-best results are underlined. solutions are deterministic and initial-condition-independent, so the policy learns from consistent, non-conflicting targets rather t… view at source ↗
Figure 6
Figure 6. Figure 6: WARP retargets one robot motion to different robot embodiments. can sort pouring coffee average Method replay policy replay policy replay policy replay policy MINK 99.5% 94% 88.5% 74% 50.5% 8% 79.5% 59% WARP 98.5% 100% 90.5% 78% 51.0% 34% 80.0% 71% [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Real-world evaluation of retargeted data re￾play and policy rollout (10 trials each). We collect 50 human demonstrations for each task and train a policy with it Sec. B.7. Pick￾up-laundry. The operator lifts a laundry basket by both handles, moves it to an adjacent table, and sets it down. The task stresses bimanual wrist control: each handle must be gripped at a precise orientation and that orientation re… view at source ↗
Figure 8
Figure 8. Figure 8: Human demonstrations and robot executions across four real-world tasks. a) Rotate Box b) Push Cart c) Pick up Laundry d) Fridge Door Closing [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Comparison with WARP and MINK on our tasks. WARP resolves constraints across the whole kinematic chain, preserving palm pose and human-like posture—a motion-quality advantage driving policy success. MINK drives the torso and base into extreme or colliding configurations: a) over-rotating, b) hitting the cart, and c) accumulating wrist-orientation errors that break grasps. d) shows that WARP can also treat … view at source ↗
Figure 10
Figure 10. Figure 10: Posture-cost sweeps: MINK (blue) vs. the tuning-free WARP reference (orange dashed). Panels (a)–(c) sweep elbow angle cost; (d)–(f) sweep torso orientation cost. No single weight matches SEW on palm accuracy, posture error, and action consistency at once; see Sec. A.2. facing direction. The vertical axis uz = ux × uy completes a right-handed orthonormal basis R = [ux, uy, uz]. The construction is delibera… view at source ↗
read the original abstract

Direct transfer from human demonstration to learnable robot action is a crucial step towards scalable whole-body mobile manipulation. While human data scales better than mobile teleoperation, it requires overcoming significant embodiment gaps. Existing retargeting methods yield imprecise or inconsistent solutions, causing action multi-modality that prevents supervised policies from reliably converging. We present Whole-body-Aware Retargeting from human Pose (WARP), an offline pipeline that explicitly models embodiment differences to extract precise, unique whole-body actions. WARP leverages a closed-form Shoulder-Elbow-Wrist (SEW) geometric solver for exact end-effector tracking while preserving whole-body structural intent. Paired with lazy mobile-base control, it extracts accurate, consistent robot trajectories. Evaluations show WARP provides highly reliable data for open-loop real-world replay. To our knowledge, WARP is the first framework to achieve zero-shot whole-body mobile manipulation directly from offline human demonstrations, eliminating the need for human-in-the-loop teleoperation action data. More details on https://warp-retarget.github.io/

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper introduces WARP, an offline retargeting pipeline for whole-body mobile manipulation robots. It claims to extract precise and unique robot actions directly from human pose demonstrations by combining a closed-form Shoulder-Elbow-Wrist (SEW) geometric solver for end-effector tracking with lazy mobile-base control, thereby eliminating action multi-modality and enabling zero-shot supervised learning without any human-in-the-loop teleoperation data. The abstract asserts this is the first such framework and reports reliable open-loop real-world replay performance.

Significance. If the uniqueness and consistency claims hold with supporting analysis, the result would be significant for scalable robot learning: it would allow direct use of abundant offline human demonstration data for complex whole-body tasks, removing a major bottleneck of teleoperation collection. The closed-form solver and lazy-base approach, if shown to be parameter-free and multi-modality-free, would constitute a concrete technical contribution.

major comments (2)
  1. [Abstract] Abstract: the central claim that the closed-form SEW solver plus lazy base control 'extracts accurate, consistent robot trajectories' and removes multi-modality rests on an unproven assumption. Geometric IK solvers for SEW chains are known to admit multiple solutions (elbow flip, base offset) for the same wrist target; the manuscript supplies neither a selection rule, uniqueness proof, nor empirical distribution of output variance across embodiment gaps.
  2. [Abstract] Abstract: no quantitative results, error metrics, or ablation on embodiment mismatch are provided to support the assertions of 'highly reliable data' or 'zero-shot' performance; the soundness assessment is therefore limited to the abstract description alone.
minor comments (1)
  1. [Abstract] The supplementary website link is given but the abstract does not indicate what additional material (videos, code, datasets) is hosted there.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. We address each major comment below and will revise the paper to strengthen the presentation of the SEW solver's properties and the supporting evidence.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that the closed-form SEW solver plus lazy base control 'extracts accurate, consistent robot trajectories' and removes multi-modality rests on an unproven assumption. Geometric IK solvers for SEW chains are known to admit multiple solutions (elbow flip, base offset) for the same wrist target; the manuscript supplies neither a selection rule, uniqueness proof, nor empirical distribution of output variance across embodiment gaps.

    Authors: The SEW solver is formulated as a closed-form geometric procedure that directly uses the demonstrated shoulder-elbow-wrist positions to compute a unique arm configuration by preserving the human's relative joint structure and elbow position with respect to the shoulder-wrist vector; this choice rule eliminates elbow-flip ambiguity by construction. Lazy base control further removes base-offset degrees of freedom by holding the mobile base fixed during arm motion. We will add an explicit subsection in the method describing this selection mechanism, a short uniqueness argument based on the closed-form equations, and an empirical plot of output variance across embodiment gaps in the revised manuscript. revision: yes

  2. Referee: [Abstract] Abstract: no quantitative results, error metrics, or ablation on embodiment mismatch are provided to support the assertions of 'highly reliable data' or 'zero-shot' performance; the soundness assessment is therefore limited to the abstract description alone.

    Authors: The abstract is intentionally concise; the full manuscript contains quantitative tracking error metrics, real-world open-loop replay success rates, and embodiment-mismatch ablations in the Experiments section. To address the concern we will insert a brief sentence with key numerical results into the abstract and ensure the zero-shot claim is tied to the reported metrics in the revision. revision: yes

Circularity Check

0 steps flagged

No circularity; derivation self-contained via geometric construction

full rationale

The abstract and described pipeline present WARP as a direct geometric retargeting method using a closed-form SEW solver plus lazy base control to produce consistent trajectories. No equations, fitted parameters renamed as predictions, or self-citation chains are shown that reduce the uniqueness or consistency claims to inputs by construction. The method is positioned as addressing embodiment gaps through explicit modeling, with external evaluation claims, satisfying the criteria for a non-circular, self-contained derivation.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Based solely on abstract; no free parameters, invented entities, or additional axioms are explicitly stated beyond the core modeling assumption.

axioms (1)
  • domain assumption Embodiment differences can be explicitly modeled via a closed-form SEW geometric solver to yield precise and unique robot actions.
    Invoked as the foundation of the WARP pipeline in the abstract.

pith-pipeline@v0.9.1-grok · 5734 in / 1176 out tokens · 34288 ms · 2026-06-30T05:56:35.068909+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 14 canonical work pages · 1 internal anchor

  1. [1]

    Z. Fu, T. Z. Zhao, and C. Finn. Mobile ALOHA: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. InProceedings of The 8th Conference on Robot Learning, 2024

    2024

  2. [2]

    Open-television: Teleoperation with immersive active visual feedback,

    X. Cheng, J. Li, S. Yang, G. Yang, and X. Wang. Open-television: Teleoperation with immer- sive active visual feedback, 2024. URLhttps://arxiv.org/abs/2407.01512

    work page arXiv 2024

  3. [3]

    X. Xu, J. Park, H. Zhang, E. Cousineau, A. Bhat, J. Barreiros, D. Wang, and S. Song. Hommi: Learning whole-body mobile manipulation from human demonstrations.arXiv preprint arXiv:2603.03243, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  4. [4]

    J. P. Araujo, Y . Ze, P. Xu, J. Wu, and C. K. Liu. Retargeting matters: General motion retargeting for humanoid motion tracking, 2025. URLhttps://arxiv.org/abs/2510.02252

    work page arXiv 2025

  5. [5]

    Sundaresan, R

    P. Sundaresan, R. Malhotra, P. Miao, J. Yang, J. Wu, H. Hu, R. Antonova, F. Engelmann, D. Sadigh, and J. Bohg. Homer: Learning in-the-wild mobile manipulation via hybrid imitation and whole-body control, 2025. URLhttps://arxiv.org/abs/2506.01185

    work page arXiv 2025

  6. [6]

    V . Liu, A. Adeniji, H. Zhan, R. Bhirangi, P. Abbeel, and L. Pinto. Egozero: Robot learning from smart glasses, 2025. URLhttps://arxiv.org/abs/2505.20290

    work page arXiv 2025

  7. [7]

    Guzey, H

    I. Guzey, H. Qi, J. Urain, C. Wang, J. Yin, K. Bodduluri, M. M. Lambeta, L. Pinto, A. Rai, J. Malik, T. Wu, A. Sharma, and H. Bharadhwaj. Dexterity from smart lenses: Multi-fingered robot manipulation with in-the-wild human demonstrations, 2025. URLhttps://arxiv. org/abs/2511.16661

    work page arXiv 2025

  8. [8]

    C. Kong, Y . Cho, W. Jung, I. Wibowo, P. Shinde, S. Vinodh-Sangeetha, L. K. Chung, Z. Chen, A. Mattei, A. Nidumukkala, A. Elias, D. Xu, T. Higgins, and S. Kousik. A closed- form geometric retargeting solver for upper body humanoid robot teleoperation, 2026. URL https://arxiv.org/abs/2602.01632

    work page arXiv 2026

  9. [9]

    K. Zakka. Mink: Python inverse kinematics for robotic systems.https://github.com/ kevinzakka/mink, 2024. Software library

    2024

  10. [10]

    Q. Ben, F. Jia, J. Zeng, J. Dong, D. Lin, and J. Pang. HOMIE: Humanoid loco-manipulation with isomorphic exoskeleton cockpit, 2025. URLhttps://arxiv.org/abs/2502.13013

    work page arXiv 2025

  11. [11]

    P. Wu, Y . Shentu, Z. Yi, X. Lin, and P. Abbeel. GELLO: A general, low-cost, and intuitive teleoperation framework for robot manipulators. InIEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024

    2024

  12. [12]

    R. Ding, Y . Qin, J. Zhu, C. Jia, S. Yang, R. Yang, X. Qi, and X. Wang. Bunny-VisionPro: Real-time bimanual dexterous teleoperation for imitation learning, 2024

    2024

  13. [13]

    C. Chi, Z. Xu, C. Pan, E. Cousineau, B. Burchfiel, S. Feng, R. Tedrake, and S. Song. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots. InRobotics: Science and Systems (RSS), 2024

    2024

  14. [14]

    C. Wang, H. Shi, W. Wang, R. Zhang, L. Fei-Fei, and C. K. Liu. DexCap: Scalable and portable mocap data collection system for dexterous manipulation. InRobotics: Science and Systems (RSS), 2024

    2024

  15. [15]

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware. InRobotics: Science and Systems (RSS), 2023

    2023

  16. [16]

    C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion. InRobotics: Science and Systems (RSS), 2023. 9

    2023

  17. [17]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Fos- ter, G. Lam, P. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn. OpenVLA: An open-source vision-language-action model. InConfer- ence on Robot Learning (CoRL), 2024

    2024

  18. [18]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Haus- man, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinsky.π 0: A vision-language-action flow model for general robot control, 2024

    2024

  19. [19]

    Xiong, R

    H. Xiong, R. Mendonca, K. Shaw, and D. Pathak. Adaptive mobile manipulation for articulated objects in the open world, 2024. URLhttps://arxiv.org/abs/2401.14403

    work page arXiv 2024

  20. [20]

    Behavior robot suite: Streamlining real-world whole-body manipulation for everyday household activities.arXiv preprint arXiv:2503.05652, 2025

    Y . Jiang, R. Zhang, J. Wong, C. Wang, Y . Ze, H. Yin, C. Gokmen, S. Song, J. Wu, and L. Fei- Fei. BEHA VIOR robot suite: Streamlining real-world whole-body manipulation for everyday household activities, 2025. URLhttps://arxiv.org/abs/2503.05652

    work page arXiv 2025

  21. [21]

    Y . Li, Y . Lin, J. Cui, T. Liu, W. Liang, Y . Zhu, and S. Huang. CLONE: Closed-loop whole- body humanoid teleoperation for long-horizon tasks, 2025. URLhttps://arxiv.org/abs/ 2506.08931

    work page arXiv 2025

  22. [22]

    J. Yang, I. Huang, B. Vu, M. Bajracharya, R. Antonova, and J. Bohg. Mobi-π: Mobilizing your robot learning policy.arXiv preprint arXiv:2505.23692, 2025

    work page arXiv 2025

  23. [23]

    Y . Qin, W. Yang, B. Huang, K. V . Wyk, H. Su, X. Wang, Y .-W. Chao, and D. Fox. Anyteleop: A general vision-based dexterous robot arm-hand teleoperation system, 2023. URLhttps: //arxiv.org/abs/2307.04577

    work page arXiv 2023

  24. [24]

    A. J. Elias and J. T. Wen. Redundancy parameterization and inverse kinematics of 7-dof revo- lute manipulators.Mechanism and Machine Theory, 204:105824, 2024

    2024

  25. [25]

    A. J. Elias and J. T. Wen. Ik-geo: Unified robot inverse kinematics using subproblem decom- position.Mechanism and Machine Theory, 209:105971, 2025

    2025

  26. [26]

    Aria Gen 2: An advanced research device for egocentric ai research

    Project Aria Team. Aria Gen 2: An advanced research device for egocentric ai research. https://www.projectaria.com/ariagen2devicepaper, 2025. Accessed: 2026-05-29

    2025

  27. [27]

    BONES-SEED: Skeletal everyday embodiment dataset.https:// huggingface.co/datasets/bones-studio/seed, 2026

    Bones Studio. BONES-SEED: Skeletal everyday embodiment dataset.https:// huggingface.co/datasets/bones-studio/seed, 2026. Hugging Face dataset. Accessed: 2026-05-10

    2026

  28. [28]

    Jiang, Y

    Z. Jiang, Y . Xie, K. Lin, Z. Xu, W. Wan, A. Mandlekar, L. Fan, and Y . Zhu. Dexmimicgen: Automated data generation for bimanual dexterous manipulation via imitation learning. In 2025 IEEE International Conference on Robotics and Automation (ICRA), 2025. 10 Supplementary Material Supplementary Material 11 A Why Existing Retargeting Fails? 11 A.1 MINK Base...

    work page arXiv 2025