Pith. sign in

REVIEW 4 major objections 5 minor 79 references

EgoHTR claims that foothold-precise humanoid locomotion on rough terrain strictly requires centimeter-accurate, scene-aligned human motion references, and provides a dataset and pipeline to produce them.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-02 05:04 UTC pith:XQ7GBQ2J

load-bearing objection Genuinely new rough-terrain 4D human-scene dataset with a credible capture pipeline and a real robot demo, but the outdoor centimeter-precision claim leans on an unvalidated Aria SLAM backbone and should be tempered in revision. the 4 major comments →

arxiv 2607.13472 v1 pith:XQ7GBQ2J submitted 2026-07-15 cs.RO cs.CV

EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal

Alex Brandes , Haig Conti Georges Sajelian , Manthan Patel , Dominik Hollidt , Chenhao Li , Matthias Heyrman , Oliver Hausdoerfer , Manuel Kaufmann
show 6 more authors
Xi Wang Jonas Frey Angela P. Schoellig Christian Holz Marc Pollefeys Marco Hutter
This is my paper
classification cs.RO cs.CV
keywords EgoHTRegocentric 4D reconstructionhuman-scene datasetrough terrainperceptive locomotionhumanoidSLAMterrain traversal
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that prior human-motion datasets and monocular reconstruction methods fail to support training of humanoid robots on unstructured terrain because they lack scene-aligned reference motion with enough precision. It introduces EgoHTR, a dataset of 55 in-the-wild sequences (over 150k frames) in seven scenes, reconstructed with a pipeline that fuses an egocentric SLAM headset, an IMU-based motion capture suit, and a portable 3D scanner. The central quantitative claim is that reference foot placement tolerates noise up to about 5 centimeters but collapses above 10 centimeters, and that current monocular methods exhibit global drift exceeding this window. The paper demonstrates that perceptive locomotion policies trained on this data can run on real hardware, establishing both a benchmark and a training resource for context-aware rough-terrain locomotion.

Core claim

The paper's central discovery is that the bottleneck for perceptive humanoid locomotion on rough terrain is not policy design but reference-motion precision: foot placement must be specified within roughly five centimeters of the scanned terrain, and no existing dataset or monocular reconstruction pipeline meets that threshold in unstructured environments. By anchoring the human body model to the closed-loop trajectory of egocentric Aria SLAM and refining alignment with a dense 3D scan via ICP, the proposed pipeline produces scene-aligned 4D sequences with reported local MPJPE of 73.2 mm and PA-MPJPE of 54.3 mm against motion-capture ground truth, and global RTE of 0.09%. The noise-injection

What carries the argument

The load-bearing mechanism is the two-stage spatial alignment pipeline: Eq. 1 anchors the MoCap-derived SMPL-X body model to the Aria SLAM world frame via a static head-to-camera transform, eliminating global IMU drift; Eq. 2 refines the human-to-scene transform by initializing with a learned visual geometry transformer and then running ICP between the semi-dense Aria point cloud and the dense scanner mesh. The noise-injection experiment is the key evidence mechanism, mapping Gaussian root-position noise to policy success rates and yielding the tolerance threshold that separates 'necessary' from 'merely convenient' precision.

Load-bearing premise

The pipeline treats the Aria SLAM closed-loop trajectory as effectively ground-truth global reference, and if that trajectory drifts or fails (as the paper concedes in featureless or high-acceleration conditions), every global metric, the centimeter-precision claim, and the noise-threshold experiment inherit that error.

What would settle it

Set up a survey-grade total station or RTK-GPS marker rigidly mounted on the Aria glasses, record a 90-second high-acceleration parkour sequence in a large, feature-poor outdoor area, and compare the SLAM trajectory to the reference in world coordinates; a deviation persistently exceeding 5 cm would invalidate the global-precision claim for that scenario. Alternatively, find a monocular 4D reconstruction method whose world-aligned errors on rough terrain stay below 5 cm and show it can train the same policies to comparable success, which would undercut the necessity argument.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Perceptive locomotion policies trained on EgoHTR's reference motions transfer to a physical Unitree G1 humanoid on beam and box-up tasks, showing the data is sufficient for real-world deployment.
  • Existing monocular 4D human-scene reconstruction methods (e.g., those reliant on exocentric video) produce global errors exceeding the 0.1 m collapse threshold, so they cannot serve as drop-in training references for foothold-critical locomotion.
  • The open-source pipeline provides a practical recipe for other groups to collect similar rough-terrain datasets, potentially scaling beyond the initial seven scenes and eight subjects.
  • The dataset's ground-truth and benchmark suite exposes concrete failure modes of exocentric, egocentric, and IMU-based mesh recovery methods under occlusion, motion blur, and confined spaces.
  • Adding a temporal foot-contact reward to a mimic-based policy accelerates convergence and improves success on sparse terrain such as stepping stones, indicating the reference contact labels carry useful training signal.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The 5 cm tolerance window may reflect a general property of contact-rich imitation learning, not just this robot or terrain; if so, any capture system—optical, inertial, or vision-based—should be audited for global drift at that scale before being used to generate training references.
  • One could directly test the necessity claim by attempting to train the same policies on a monocular reconstruction whose global errors are artificially corrected to below 5 cm; if success matches EgoHTR, the bottleneck is precision, not data modality.
  • The Aria SLAM-as-ground-truth assumption could be independently validated by mounting a survey-grade total-station or RTK-GPS target on the glasses during a high-acceleration, feature-poor sequence; if the discrepancy exceeds 5 cm, the pipeline would need loop closure or an external anchor to preserve its claimed precision.
  • The dataset's dense meshes paired with foot-contact labels invite training contact-aware human-scene reconstruction or motion synthesis models that take continuous local geometry as input, a direction the paper only hints at.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces EgoHTR, a dataset of 55 egocentric, scene-aligned 4D human motion sequences (150k+ frames) captured in rough, unstructured terrain, together with an open-source reconstruction pipeline. The pipeline combines Aria glasses SLAM, an IMU-based MoCap suit, and a handheld 3D scanner to produce SMPL-X body sequences registered to dense scene meshes. The authors evaluate local pose accuracy against optical MoCap ground truth, benchmark state-of-the-art HPS methods on the new data, and train perceptive locomotion policies on a Unitree G1, including a noise-injection experiment intended to show that centimeter-level reference accuracy is necessary for foothold-critical tasks.

Significance. If the central claims hold, EgoHTR would be a valuable resource: it targets a genuine data gap for 4D human-scene reconstruction in rough terrain, ships a reusable capture pipeline, and provides an external optical MoCap check for local pose. The noise-threshold experiment (§5.1) is a useful, falsifiable statement about required reference precision. However, the load-bearing global-precision claim — that the released in-the-wild reference motions are centimeter-precise — rests on Aria SLAM as an effectively ground-truth global reference, and the current validation does not independently confirm that assumption for the outdoor sequences. The benchmark comparison in Table 2 is also not head-to-head. These issues are fixable but require additional validation and careful re-framing.

major comments (4)
  1. [§4.2, Table 2, App. D.1] The state-of-the-art comparison is not head-to-head. The baseline numbers come from different datasets and different protocols: PROX is evaluated on its own MoCap test set, RICH uses only non-contact frames, and the SLOPER4D entry is the mean of best cross-evaluation results. The sentence claiming 'EgoHTR achieves a 6.2% lower MPJPE than SLOPER4D' is therefore not a controlled comparison. Please either run the baselines on the EgoHTR test set or present Table 2 as contextual reference and remove the state-of-the-art claim. This is important because 'rigorous benchmark' is one of the paper's stated contributions.
  2. [§B.5, Eq. (B.2)] The MoCap ground-truth global trajectory is registered to the Aria SLAM trajectory via ICP over a fixed time window. Any slow drift in the Aria SLAM trajectory is therefore absorbed into the 'ground truth' and cannot be detected by the reported RTE of 0.09% or W-MPJPE of 151.3 mm. The global accuracy evaluation is thus not independent of the system it is meant to validate. Please provide an external global reference — e.g., surveyed markers, a second independent tracking system, or loop-closure consistency checks over full sequences — for at least a subset of the outdoor rough-terrain sequences.
  3. [§3.2, Eqs. (1)–(2), §6] The global frame is anchored entirely to the Aria SLAM trajectory, and Section 6 concedes 'potential failures in featureless environments or during high-acceleration maneuvers that challenge sensor localization capabilities.' The debris-field and parkour sequences are precisely such cases, yet no per-sequence SLAM-quality metric, failure detection, or drift estimate is reported for the production dataset. Without this, the claim that the released in-the-wild reference motions are centimeter-precise is unverified for the sequences the dataset is designed to serve. Please add per-sequence localization quality control and report how many sequences meet the precision threshold.
  4. [§5.1, Table 2] The link between the noise-ablation experiment and the dataset precision claim is not directly supported by the reported numbers. The ablation shows training tolerates σ=0.05 m root noise and collapses above 0.1 m, but the paper reports only W-MPJPE (151.3 mm), W A-MPJPE (66.7 mm), and a normalized RTE (0.09%). None of these is explicitly shown to be the relevant root/contact error below 5 cm. Please report absolute root-translation error and foot-contact error distributions (e.g., mean/percentiles) on the test set, and state exactly which quantity satisfies the centimeter-precision threshold. As written, the central conclusion that 'foothold-precise locomotion strictly requires human-terrain coupling' is underspecified.
minor comments (5)
  1. [Table 2] The 'Ours (local)' row contains a stray '72.6' before '73.2'; the formatting appears corrupted. Please check the table and the caption.
  2. [App. D.1] The definition of W A-MPJPE says alignment uses 'either the first two frames or all frames of 100-frame segments.' Please specify which protocol is used in Table 2 and whether the same protocol is applied consistently to all compared baselines.
  3. [§B.5, Fig. D.1] Figure D.1 labels the test set as 'EgoHTG vs Pseudo GT'; this appears to be a typo for EgoHTR. Also, the term 'MoCap GT' and 'pseudo GT' are used interchangeably; please standardize the terminology.
  4. [§B.1, §B.3] Aria SLAM is recorded at 1000 fps and the MoCap suit at 30 fps, with temporal alignment performed via a hand clap and bounded to less than 60 ms. The residual temporal uncertainty should be stated as an error contribution in the metric evaluation, since it directly affects joint position errors during fast motions.
  5. [§4.1, Table C.1] The 'Test Subset' contains only Robotics Hall sequences (X2, X3). The paper should state explicitly that the quantitative evaluation does not cover the Gym Hall and Debris Field outdoor sequences, and that the qualitative statements about these scenes are not backed by the reported numbers.

Circularity Check

2 steps flagged

Accuracy validation is partly self-referential: the MoCap 'ground truth' is retargeted with the same IK and registered to the same Aria SLAM reference it is meant to validate.

specific steps
  1. self definitional [Appendix B.5, Eq. B.2; Section 3.2, Eqs. 1-2]
    "We spatial align the marker-based and the egocentric sensor coordinates by tracking a dedicated marker mounted above the Aria glasses’ camera. Over a fixed time window, we register this marker trajectory to the closed-loop Aria SLAM trajectory. ... We refine the coarse registration with Iterative Closest Point (ICP) f_ICP on the temporally aligned trajectories from both systems, T M and T A."

    The production global reference is the Aria SLAM trajectory: Eq. 1 sets the body sequence in the Aria world frame and the text says 'we eliminate global IMU drift by relying entirely on Aria SLAM'. Eq. B.2 then defines the MoCap 'ground truth' frame by ICP-registering the Qualisys marker trajectory T_M to that same Aria trajectory T_A. So the GT into which global errors are measured is itself fitted to the measured reference; the component of Aria drift that is constant over the fitted window is absorbed into the transform and cannot appear in W-MPJPE/RTE, and any residual is measured against a non-independent reference. The reported RTE 0.09% therefore does not independently validate the Aria trajectory that Eq. 1 promotes to ground truth, and the extension to outdoor rough-terrain precis

  2. self definitional [Appendix B.5 'Motion Capture Ground Truth Acquisition'; Section 3.2 'Body Model' / Eq. B.1]
    "Finally, we retarget this skeleton to the SMPL-X body model and refine the skeleton mapping via a subsequent inverse kinematics (IK) optimization, following the same methodology detailed in Section 3."

    The production SMPL-X pose is obtained by the same two-stage conversion: rotational retargeting after Ze et al. plus an IK refinement (Eq. B.1) that minimizes limb-joint displacements. Because the MoCap 'ground truth' SMPL-X is generated with the same retargeting+IK procedure, any systematic bias in that procedure (fixed hand/face parameters, per-joint offsets, IK loss over the limb subset L) is common to both the test and the reference. The local MPJPE/PA-MPJPE therefore measures only the residual between the IMU-suit and optical-marker input streams after identical conversion, not the absolute accuracy of the SMPL-X pose produced by the pipeline.

full rationale

The dataset and most downstream content are not circular: EgoHTR provides raw multi-modal sensor streams, dense scene meshes, an external Qualisys marker system for reference, and the Section 5.1 noise-threshold experiment is an independent simulation with controlled injected Gaussian noise. The central claimed contribution, however, includes 'centimeter-precise scene-aligned reference motions' and an evaluation 'against motion-capture ground truth'. That validation is partly self-definitional. Appendix B.5 constructs the MoCap GT by (i) retargeting the marker skeleton to SMPL-X with 'the same methodology' (rotational retargeting + IK, Eq. B.1) used in the production body-model stage, and (ii) globally registering the marker trajectory via ICP to the same Aria closed-loop SLAM trajectory that Eq. 1 makes the production global reference. Systematic retargeting bias and slow Aria drift are therefore common to both the test outputs and the GT, so the reported MPJPE, W-MPJPE, and RTE values cannot detect them. The RTE/centimeter-precision results are computed on the indoor Robotics Hall subset, and the claim that in-the-wild rough-terrain sequences also meet the 5-cm training tolerance is an unvalidated extrapolation, not a measured benchmark. This is partial circularity of the accuracy validation, not a reduction of the dataset content or of the locomotion-policy conclusions, hence a moderate score of 5.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The central accuracy claims rest on four calibration/estimation tasks whose values are not reported in the preprint, plus an external-SLAM assumption and shared retargeting between production and ground truth. No new physical entities are introduced.

free parameters (4)
  • Aria-to-head static offset r_CAH = empirically calibrated per subject/sequence; value not reported
    Eq. 1 models the glasses-to-head relationship as a static translation and the text states it 'necessitates an empirical calibration of the offset.' Global anchoring of the body model depends on this number.
  • Per-joint retargeting offset C_j = per-joint SO(3) offsets; values not reported
    Appendix B.2 introduces C_j to compensate systematic skeletal axis misalignment between the Rokoko suit and SMPL-X; these offsets are fit to the source skeleton and directly affect pose accuracy.
  • Body shape beta = 10-dim shape vector per subject; values not reported
    Shape parameters are estimated with external toolkits [33,51] and assumed constant per sequence; they determine the body proportions in the reconstructed SMPL-X model.
  • MoCap-to-Aria alignment offset T_offset = constant transform; values not reported
    Appendix B.5 fits T_offset to account for displacement between the tracked marker and the body-model root when registering the MoCap ground truth to the Aria world frame.
axioms (5)
  • domain assumption Aria SLAM closed-loop trajectory is an accurate, drift-free global reference.
    Section 3.2, Eq. 1 replaces the MoCap suit's global drift with Aria SLAM, citing [53] rather than verifying drift on every EgoHTR scene. Section 6 acknowledges failure risk in featureless or high-acceleration conditions.
  • domain assumption The glasses do not move relative to the head, so a static transform suffices.
    Eq. 1 models C_A-to-H as a static translation; if the glasses slip or deform during dynamic motions, the body anchoring is biased.
  • domain assumption The scene is rigid and static during capture.
    Section 6 states the pipeline is restricted to static environments devoid of articulated objects; moving scene elements would break the ICP/VGGT alignment in Eq. 2.
  • domain assumption Clock drift is negligible for sequences up to 5 minutes.
    Section 3.2 Temporal Alignment bounds cross-sensor alignment error using a hand-clap event and relies on short sequences to keep hardware clock drift negligible, without continuous synchronization.
  • domain assumption MoCap ground truth retargeted to SMPL-X via the same pipeline is a valid external reference.
    Appendix B.5 uses the same retargeting and IK methodology as the production pipeline, so shared calibration errors between the 'ground truth' and the reconstruction are not exposed by the reported error metrics.

reviewed 2026-08-02 · how reviews work

0 comments
Cite this review

Pith. "Pith review of EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal." pith.science (2026). https://pith.science/paper/XQ7GBQ2J

@misc{pith2026260713472,
  author       = {Pith},
  title        = {Pith review of: EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XQ7GBQ2J}},
  note         = {Machine review of arXiv:2607.13472}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Deploying humanoid robots in unstructured terrain remains an open problem. While classic reinforcement learning struggles with the sheer complexity of real-world interactions, more promising methods leveraging human priors remain limited to models lacking contextual awareness. The restricted motion synthesis is a direct consequence of existing dataset pipelines failing to capture human-scene sequences in challenging environments. To bridge this gap between humanoid learning and scene reconstruction, we introduce the Egocentric Human-Terrain Reconstruction (EgoHTR) dataset. We develop and open-source a reconstruction pipeline capturing 55 scene-aligned 4D human motion sequences in diverse, complex environments using a multi-sensor setup of egocentric wearables and a portable 3D scanner. The resulting dataset comprises over 150k frames, which we evaluate against motion-capture ground truth, demonstrating state-of-the-art accuracy and establishing a rigorous benchmark for human motion analysis and synthesis. Further, we leverage this data to train perceptive locomotion policies, demonstrating hardware deployment on a Unitree G1 for reconstructed reference motions. Our pipeline enables community-driven dataset extensions and factors the problem to help researchers build foundational, context-aware robots that reliably traverse uneven terrain.

Figures

Figures reproduced from arXiv: 2607.13472 by Alex Brandes, Angela P. Schoellig, Chenhao Li, Christian Holz, Dominik Hollidt, Haig Conti Georges Sajelian, Jonas Frey, Manthan Patel, Manuel Kaufmann, Marco Hutter, Marc Pollefeys, Matthias Heyrman, Oliver Hausdoerfer, Xi Wang.

Figure 1
Figure 1. Figure 1: EgoHTR dataset preview, a set of in-the-wild 4D human-scene demonstrations focused on rough terrain. Left: Example outdoor (debris field) and indoor (gym hall) scene, highlighting the reconstructed 4D human-scene sequences with its provided modalities (light blue; ego- and exocentric Aria video and SLAM streams, 3D mesh and point cloud, parametrized model, raw IMU) and the diversity of human motions (dark … view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the implemented dataset generation pipeline. We categorize the methodology into two stages: data acquisition and human-terrain reconstruction – enabling human2robot retargeting (Appendix B.4). A capture system with 3 core sensors (Aria glasses, MoCap suit, 3D Scanner) provide data for a three stage human-scene reconstruction: (I) body model parametrization, (II) temporal and (III) spatial align… view at source ↗
Figure 3
Figure 3. Figure 3: The image streams show the aligned 3D environment (background) with the estimated 3D human mesh (red) overlaid with the projected observer image (blue border) of versatile motions: walking through a debris field (A), balancing over a narrow beam (B), and crawling into a pipe (C). 5 Applications 5.1 Perceptive Locomotion from Demonstrations Training whole-body mimic policies on sparse, complex terrain requi… view at source ↗
Figure 5
Figure 5. Figure 5: Reference motion precision ablation. Dif￾ferent levels of Gaussian noise applied to root po￾sition of reference motion. Training performance is highly reduced above σ = 0.05 m and impossible at errors above 0.1 m (evaluated over 5 seeds each). 0 200 400 600 800 0.0 0.2 0.4 0.6 0.8 S. Stones [m] 0.0 0.01 0.02 0.05 0.1 0 200 400 600 800 1000 Beam Iteration Success Rate 5.2 Human Mesh Recovery The multi-modal… view at source ↗
Figure 6
Figure 6. Figure 6: Failure cases of SOTA human mesh recovery methods under challenging environmental conditions present in our dataset. Left: Observer Aria image frames (top). Our EgoHTR reconstructions (middle). Failure cases of SOTA methods induced by complex scene interactions (bottom). Right: Visualizing the global motion estimates, together with color-encoded chamfer distance of the reconstructed scene point cloud. 6 Li… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

79 extracted references · 21 linked inside Pith

  1. [1]

    Zhang, Y

    Y . Zhang, Y . Seo, J. Chen, Y . Yuan, K. Sreenath, P. Abbeel, C. Sferrazza, K. Liu, R. Duan, and G. Shi. Rpl: Learning robust humanoid perceptive locomotion on challenging terrains.arXiv preprint arXiv:2602.03002, 2026

    arXiv 2026

  2. [2]

    Rudin, J

    N. Rudin, J. He, J. Aurand, and M. Hutter. Parkour in the wild: Learning a general and extensible agile locomotion policy using multi-expert distillation and rl fine-tuning.arXiv preprint arXiv:2505.11164, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    J. Sun, G. Han, P. Sun, W. Zhao, J. Cao, J. Wang, Y . Guo, and Q. Zhang. Dpl: Depth- only perceptive humanoid locomotion via realistic depth synthesis and cross-attention terrain reconstruction.arXiv preprint arXiv:2510.07152, 2025

    arXiv 2025

  4. [4]

    J. Long, J. Ren, M. Shi, Z. Wang, T. Huang, P. Luo, and J. Pang. Learning humanoid lo- comotion with perceptive internal model.2025 IEEE International Conference on Robotics and Automation (ICRA), pages 9997–10003, 2024. URLhttps://api.semanticscholar. org/CorpusID:274165896

    2025

  5. [5]

    Z. Fu, Q. Zhao, Q. Wu, G. Wetzstein, and C. Finn. Humanplus: Humanoid shadowing and imitation from humans. InConference on Robot Learning (CoRL), 2024

    2024

  6. [6]

    K. Yin, W. Zeng, K. Fan, M. Dai, Z. Wang, Q. Zhang, Z. Tian, J. Wang, J. Pang, and W. Zhang. Unitracker: Learning universal whole-body motion tracker for humanoid robots, 2025. URL https://arxiv.org/abs/2507.07356

    arXiv 2025

  7. [7]

    Q. Liao, T. E. Truong, X. Huang, Y . Gao, G. Tevet, K. Sreenath, and C. K. Liu. Beyondmimic: From motion tracking to versatile humanoid control via guided diffusion, 2025. URLhttps: //arxiv.org/abs/2508.08241

    Pith/arXiv arXiv 2025

  8. [8]

    Z. Chen, M. Ji, X. Cheng, X. Peng, X. B. Peng, and X. Wang. Gmt: General motion tracking for humanoid whole-body control.arXiv:2506.14770, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    S. Zhao, Y . Ze, Y . Wang, C. K. Liu, P. Abbeel, G. Shi, and R. Duan. Resmimic: From general motion tracking to humanoid whole-body loco-manipulation via residual learning, 2025. URL https://arxiv.org/abs/2510.05070

    arXiv 2025

  10. [10]

    Zhang, J

    Z. Zhang, J. Guo, C. Chen, J. Wang, C. Lin, Y . Lian, H. Xue, Z. Wang, M. Liu, J. Lyu, H. Liu, H. Wang, and L. Yi. Track any motions under any disturbances, 2025. URLhttps: //arxiv.org/abs/2509.13833

    arXiv 2025

  11. [11]

    Z. Luo, Y . Yuan, T. Wang, C. Li, S. Chen, F. Casta ˜neda, Z.-A. Cao, J. Li, D. Minor, Q. Ben, X. Da, R. Ding, C. Hogg, L. Song, E. Lim, E. Jeong, T. He, H. Xue, W. Xiao, Z. Wang, S. Yuen, J. Kautz, Y . Chang, U. Iqbal, L. J. Fan, and Y . Zhu. Sonic: Supersizing motion tracking for natural humanoid whole-body control, 2025. URLhttps://arxiv.org/abs/2511.07820

    Pith/arXiv arXiv 2025

  12. [12]

    Allshire, H

    A. Allshire, H. Choi, J. Zhang, D. McAllister, A. Zhang, C. M. Kim, T. Darrell, P. Abbeel, J. Malik, and A. Kanazawa. Visual imitation enables contextual humanoid control. InPro- ceedings of the Conference on Robot Learning (CoRL), 2025

    2025

  13. [13]

    Zhang, J

    Q. Zhang, J. Ma, P. Liu, S. Shi, Z. Su, Z. Wang, J. Sun, W. Cui, J. Yu, G. Han, W. Zhao, P. Sun, K. Yin, J. Wang, J. Cao, L. Zhang, H. Cheng, X. Hao, Y . Ji, J. Liang, J. Tang, R. Xu, and Y . Guo. Meshmimic: Geometry-aware humanoid motion learning through 3d scene recon- struction, 2026. URLhttps://arxiv.org/abs/2602.15733

    arXiv 2026

  14. [14]

    W. Wang, L. Pan, H. Pi, Y . Lou, X. Ren, Y . Wu, Z. Liao, L. Yang, R. Dabral, C. Theobalt, and T. Komura. Embodmocap: In-the-wild 4d human-scene reconstruction for embodied agents,

  15. [15]

    Mahmood, N

    N. Mahmood, N. Ghorbani, N. F. Troje, G. Pons-Moll, and M. J. Black. AMASS: Archive of motion capture as surface shapes. InInternational Conference on Computer Vision, pages 5442–5451, Oct. 2019

    2019

  16. [16]

    F. G. Harvey, M. Yurick, D. Nowrouzezahrai, and C. Pal. Robust motion in-betweening.ACM Transactions on Graphics (TOG), 39(4):60–1, 2020

    2020

  17. [17]

    C. Guo, S. Zou, X. Zuo, S. Wang, W. Ji, X. Li, and L. Cheng. Generating diverse and natural 3d human motions from text. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5152–5161, June 2022

    2022

  18. [18]

    Kaufmann, J

    M. Kaufmann, J. Song, C. Guo, K. Shen, T. Jiang, C. Tang, J. J. Z ´arate, and O. Hilliges. EMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wild. In International Conference on Computer Vision (ICCV), 2023

    2023

  19. [19]

    Grauman, A

    K. Grauman, A. Westbury, L. Torresani, K. Kitani, J. Malik, T. Afouras, K. Ashutosh, V . Baiyya, S. Bansal, B. Boote, E. Byrne, Z. Chavis, J. Chen, F. Cheng, F.-J. Chu, S. Crane, A. Dasgupta, J. Dong, M. Escobar, C. Forigua, A. Gebreselasie, S. Haresh, J. Huang, M. M. Islam, S. Jain, R. Khirodkar, D. Kukreja, K. J. Liang, J.-W. Liu, S. Majumder, Y . Mao, ...

    Pith/arXiv arXiv 2024

  20. [20]

    Zhang, J

    Y . Zhang, J. Lin, A. Zeng, G. Wu, S. Lu, Y . Fu, Y . Cai, R. Zhang, H. Wang, and L. Zhang. Motion-x++: A large-scale multimodal 3d whole-body human motion dataset.arXiv preprint arXiv:2501.05098, 2025

    Pith/arXiv arXiv 2025

  21. [21]

    L. Ma, Y . Ye, F. Hong, V . Guzov, Y . Jiang, R. Postyeni, L. Pesqueira, A. Gamino, V . Baiyya, H. J. Kim, K. Bailey, D. S. Fosas, C. K. Liu, Z. Liu, J. Engel, R. D. Nardi, and R. Newcombe. Nymeria: A massive collection of multimodal egocentric daily motion in the wild, 2024. URL https://arxiv.org/abs/2406.09905

    Pith/arXiv arXiv 2024

  22. [22]

    Hassan, V

    M. Hassan, V . Choutas, D. Tzionas, and M. J. Black. Resolving 3D human pose ambiguities with 3D scene constraints. InInternational Conference on Computer Vision, pages 2282–2292, Oct. 2019. URLhttps://prox.is.tue.mpg.de

    2019

  23. [23]

    C.-H. P. Huang, H. Yi, M. H ¨oschle, M. Safroshkin, T. Alexiadis, S. Polikovsky, D. Scharstein, and M. J. Black. Capturing and inferring dense full-body human-scene contact. InProceedings IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pages 13274–13285, June 2022

    2022

  24. [24]

    Zhang, Q

    S. Zhang, Q. Ma, Y . Zhang, Z. Qian, T. Kwon, M. Pollefeys, F. Bogo, and S. Tang. Egobody: Human body shape and motion of interacting people from head-mounted devices. InEuropean Conference on Computer Vision, pages 180–200. Springer, 2022

    2022

  25. [25]

    Zheng, Y

    Y . Zheng, Y . Yang, K. Mo, J. Li, T. Yu, Y . Liu, K. Liu, and L. Guibas. Gimo: Gaze-informed human motion prediction in context. InECCV, 2022

    2022

  26. [26]

    J. Li, J. Wu, and C. K. Liu. Object motion guided human motion synthesis.ACM Trans. Graph., 42(6), 2023. 10

    2023

  27. [27]

    J. Kim, J. Kim, J. Na, and H. Joo. Parahome: Parameterizing everyday home activities towards 3d generative modeling of human-object interactions, 2024

    2024

  28. [28]

    DeTone, F

    D. DeTone, F. Bogo, E.-T. Le, D. Frost, J. Straub, Y . Siddiqui, Y . Ye, J. Engel, R. Newcombe, and L. Ma. Nymeriaplus: Enriching nymeria dataset with additional annotations and data,

  29. [29]

    Guzov, A

    V . Guzov, A. Mir, T. Sattler, and G. Pons-Moll. Human poseitioning system (hps): 3d human pose estimation and self-localization in large scenes from body-mounted sensors. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, jun 2021

    2021

  30. [30]

    URLhttps://arxiv.org/abs/2603.18496

    arXiv

  31. [31]

    Z. Liu, J. Lin, W. Wu, and B. Zhou. Joint optimization for 4d human-scene reconstruction in the wild.The F ourteenth International Conference on Learning Representations, 2026

    2026

  32. [32]

    Y . Dai, Y . Lin, X. Lin, C. Wen, L. Xu, H. Yi, S. Shen, Y . Ma, and C. Wang. Sloper4d: A scene- aware dataset for global 4d human pose estimation in urban environments. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 682– 692, June 2023

    2023

  33. [33]

    M. J. Black, P. Patel, J. Tesch, and J. Yang. BEDLAM: A synthetic dataset of bodies exhibiting detailed lifelike animated motion. InProceedings IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pages 8726–8737, June 2023

    2023

  34. [34]

    Y . Chen, X. Chen, Y . Xue, A. Chen, Y . Xiu, and P.-M. Gerard. Human3r: Everyone everywhere all at once.arXiv preprint arXiv:2510.06219, 2025

    arXiv 2025

  35. [35]

    Zhang, B

    T. Zhang, B. Zheng, R. Nai, Y . Hu, Y .-J. Wang, G. Chen, F. Lin, J. Li, C. Hong, K. Sreenath, et al. Hub: Learning extreme humanoid balance.arXiv preprint arXiv:2505.07294, 2025

    Pith/arXiv arXiv 2025

  36. [36]

    AI Datasets for Machine Learning and Motion Capture.https://bones

    Bones Studio. AI Datasets for Machine Learning and Motion Capture.https://bones. studio/ai-datasets/, 2026

    2026

  37. [37]

    Y . Ze, Z. Chen, J. P. Ara´ujo, Z. ang Cao, X. B. Peng, J. Wu, and C. K. Liu. Twist: Teleoperated whole-body imitation system.arXiv preprint arXiv:2505.02833, 2025

    Pith/arXiv arXiv 2025

  38. [38]

    Y . Wang, M. Yang, Z. Ding, Y . Zhang, W. Zeng, X. Xu, H. Jiang, and Z. Lu. From ex- perts to a generalist: Toward general whole-body control for humanoid robots.arXiv preprint arXiv:2506.12779, 2025

    Pith/arXiv arXiv 2025

  39. [39]

    Y . Ze, J. P. Ara ´ujo, J. Wu, and C. K. Liu. Gmr: General motion retargeting, 2025. URL https://github.com/YanjieZe/GMR. GitHub repository

    2025

  40. [40]

    L. Yang, X. Huang, Z. Wu, A. Kanazawa, P. Abbeel, C. Sferrazza, C. K. Liu, R. Duan, and G. Shi. Omniretarget: Interaction-preserving data generation for humanoid whole-body loco- manipulation and scene interaction, 2025. URLhttps://arxiv.org/abs/2509.26633

    Pith/arXiv arXiv 2025

  41. [41]

    Zhuang, S

    Z. Zhuang, S. Zhu, M. Zhao, and H. Zhao. Deep whole-body parkour, 2026. URLhttps: //arxiv.org/abs/2601.07701

    arXiv 2026

  42. [42]

    Z. Wu, X. Huang, L. Yang, Y . Zhang, K. Sreenath, X. Chen, P. Abbeel, R. Duan, A. Kanazawa, C. Sferrazza, G. Shi, and C. K. Liu. Perceptive humanoid parkour: Chaining dynamic human skills via motion matching, 2026. URLhttps://arxiv.org/abs/2602.15827

    Pith/arXiv arXiv 2026

  43. [43]

    S ´ar´andi and G

    I. S ´ar´andi and G. Pons-Moll. Neural localizer fields for continuous 3d human pose and shape estimation.Advances in Neural Information Processing Systems, 37:140032–140065, 2024. 11

    2024

  44. [44]

    von Marcard, R

    T. von Marcard, R. Henschel, M. Black, B. Rosenhahn, and G. Pons-Moll. Recovering accurate 3d human pose in the wild using imus and a moving camera. InEuropean Conference on Computer Vision (ECCV), sep 2018

    2018

  45. [45]

    S. Shin, J. Kim, E. Halilaj, and M. J. Black. WHAM: Reconstructing world-grounded humans with accurate 3D motion. InIEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), June 2024

    2024

  46. [46]

    Baradel*, M

    F. Baradel*, M. Armando, S. Galaaoui, R. Br ´egier, P. Weinzaepfel, G. Rogez, and T. Lucas*. Multi-hmr: Multi-person whole-body human mesh recovery in a single shot. InECCV, 2024

    2024

  47. [47]

    X. Yi, Y . Zhou, M. Habermann, S. Shimada, V . Golyanik, C. Theobalt, and F. Xu. Physical in- ertial poser (pip): Physics-aware real-time human motion tracking from sparse inertial sensors. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022

    2022

  48. [48]

    J. Li, K. Liu, and J. Wu. Ego-body pose estimation via ego-head pose estimation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17142–17151, 2023

    2023

  49. [49]

    Jiang, Y

    Y . Jiang, Y . Ye, D. Gopinath, J. Won, A. W. Winkler, and C. K. Liu. Transformer inertial poser: Real-time human motion reconstruction from sparse imus with simultaneous terrain generation. InSIGGRAPH Asia 2022 Conference Papers, pages 1–9, 2022

    2022

  50. [50]

    T. Sur, S. Tripathi, N. Athanasiou, H. L. Nguyen, K. Xu, M. J. Black, and A. Yao. Unicon3r: Contact-aware 3d human-scene reconstruction from monocular video, 2026. URLhttps: //arxiv.org/abs/2604.19923

    Pith/arXiv arXiv 2026

  51. [51]

    Pavlakos, V

    G. Pavlakos, V . Choutas, N. Ghorbani, T. Bolkart, A. A. A. Osman, D. Tzionas, and M. J. Black. Expressive body capture: 3D hands, face, and body from a single image. InProceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 10975–10985, 2019

    2019

  52. [52]

    Engel, K

    J. Engel, K. Somasundaram, M. Goesele, A. Sun, A. Gamino, A. Turner, A. Talattof, A. Yuan, B. Souti, B. Meredith, et al. Project aria: A new tool for egocentric multi-modal ai research. arXiv preprint arXiv:2308.13561, 2023

    Pith/arXiv arXiv 2023

  53. [53]

    Krishnan, S

    A. Krishnan, S. Liu, P.-E. Sarlin, O. Gentilhomme, D. Caruso, M. Monge, R. Newcombe, J. Engel, and M. Pollefeys. Benchmarking egocentric visual-inertial slam at city scale. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025

    2025

  54. [54]

    Loper, N

    M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black. SMPL: A skinned multi- person linear model.ACM Trans. Graphics (Proc. SIGGRAPH Asia), 34(6):248:1–248:16, Oct. 2015

    2015

  55. [55]

    Y . Wang, Y . Sun, P. Patel, K. Daniilidis, M. J. Black, and M. Kocabas. Prompthmr: Promptable human mesh recovery, 2025. URLhttps://arxiv.org/abs/2504.06397

    Pith/arXiv arXiv 2025

  56. [56]

    J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny. Vggt: Visual geometry grounded transformer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025

    2025

  57. [57]

    B. Yi, V . Ye, M. Zheng, Y . Li, L. M ¨uller, G. Pavlakos, Y . Ma, J. Malik, and A. Kanazawa. Estimating body and hand motion in an ego-sensed world. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 7072–7084, 2025

    2025

  58. [58]

    Y . Wang, Z. Wang, L. Liu, and K. Daniilidis. Tram: Global trajectory and motion of 3d humans from in-the-wild videos.arXiv preprint arXiv:2403.17346, 2024

    Pith/arXiv arXiv 2024

  59. [59]

    J. Lin, A. Zeng, S. Lu, Y . Cai, R. Zhang, H. Wang, and L. Zhang. Motion-x: A large-scale 3d expressive whole-body human motion dataset.Advances in Neural Information Processing Systems, 2023. 12

    2023

  60. [60]

    H.-Y . Hsu, T. Cheng, J. Wen, A. G. Schwing, and S. Wang. Seeing without eyes: 4d human- scene understanding from wearable imus.arXiv preprint, 2026

    2026

  61. [61]

    L. Xu, X. Lv, Y . Yan, X. Jin, S. Wu, C. Xu, Y . Liu, Y . Zhou, F. Rao, X. Sheng, Y . Liu, W. Zeng, and X. Yang. Inter-x: Towards versatile human-human interaction analysis.arXiv preprint arXiv:2312.16051, 2023

    Pith/arXiv arXiv 2023

  62. [62]

    Y . Zhu, N. Samet, and D. Picard. H3wb: Human3.6m 3d wholebody dataset and benchmark. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 20166–20177, October 2023

    2023

  63. [63]

    D. Yang, J. Kang, L. Ma, J. Greer, Y . Ye, and S.-H. Lee. Divatrack: Diverse bodies and motions from acceleration-enhanced three-point trackers, 2024. URLhttps://arxiv.org/ abs/2402.09211

    Pith/arXiv arXiv 2024

  64. [64]

    Khirodkar, A

    R. Khirodkar, A. Bansal, L. Ma, R. Newcombe, M. V o, and K. Kitani. Egohumans: An egocentric 3d multi-human benchmark.arXiv preprint arXiv:2305.16487, 2023

    Pith/arXiv arXiv 2023

  65. [65]

    Patel and M

    P. Patel and M. J. Black. Camerahmr: Aligning people with perspective. In2025 International Conference on 3D Vision (3DV), pages 1562–1571. IEEE, 2025

    2025

  66. [66]

    P. Cong, Z. Wang, Z. Dou, Y . Ren, W. Yin, K. Cheng, Y . Sun, X. Long, X. Zhu, and Y . Ma. Laserhuman: language-guided scene-aware human motion generation in free environment. arXiv preprint arXiv:2403.13307, 2024

    Pith/arXiv arXiv 2024

  67. [67]

    J. Li, J. Cao, H. Zhang, D. Rempe, J. Kautz, U. Iqbal, and Y . Yuan. Genmo: A generalist model for human motion. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 11766–11776, 2025

    2025

  68. [68]

    Z. Shen, H. Pi, Y . Xia, Z. Cen, S. Peng, Z. Hu, H. Bao, R. Hu, and X. Zhou. World-grounded human motion recovery via gravity-view coordinates. InSIGGRAPH Asia Conference Pro- ceedings, 2024

    2024

  69. [69]

    X. Yi, S. Pan, and F. Xu. Improving global motion estimation in sparse imu-based motion capture with physics.ACM Transactions on Graphics (TOG), 44(4):1–16, 2025

    2025

  70. [70]

    X. Yang, D. Kukreja, D. Pinkus, A. Sagar, T. Fan, J. Park, S. Shin, J. Cao, J. Liu, N. Ugrinovic, et al. Sam 3d body: Robust full-body human mesh recovery.arXiv preprint arXiv:2602.15989, 2026

    arXiv 2026

  71. [71]

    Y . Feng, V . Choutas, T. Bolkart, D. Tzionas, and M. Black. Collaborative regression of ex- pressive bodies using moderation. InInternational Conference on 3D Vision (3DV), pages 792–804, Dec. 2021. doi:10.1109/3DV53792.2021.00088

    arXiv 2021

  72. [72]

    X. Wei, M. Liu, Z. Ling, and H. Su. Approximate convex decomposition for 3d meshes with collision-aware concavity and tree search.ACM Transactions on Graphics (TOG), 41(4):1–18, 2022

    2022

  73. [73]

    X. B. Peng, P. Abbeel, S. Levine, and M. van de Panne. Deepmimic: example-guided deep reinforcement learning of physics-based character skills.ACM Trans. Graph., 37(4), July

  74. [74]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms.CoRR, abs/1707.06347, 2017. URLhttp://arxiv.org/abs/1707.06347

    Pith/arXiv arXiv 2017

  75. [75]

    Huang, T

    X. Huang, T. Truong, Y . Zhang, F. Yu, J. P. Sleiman, J. Hodgins, K. Sreenath, and F. Farshid- ian. Diffuse-cloc: Guided diffusion for physics-based character look-ahead control.ACM Transactions on Graphics (TOG), 44(4):1–12, 2025. 13

    2025

  76. [76]

    Rempe, M

    D. Rempe, M. Petrovich, Y . Yuan, H. Zhang, X. B. Peng, Y . Jiang, T. Wang, U. Iqbal, D. Minor, M. de Ruyter, et al. Kimodo: Scaling controllable human motion generation.arXiv preprint arXiv:2603.15546, 2026. 14 EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal Supplementary Material This supplementary document provides additional technical ...

    arXiv 2026

  77. [77]

    Karunratanakul, K

    K. Karunratanakul, K. Preechakul, S. Suwajanakorn, and S. Tang. Guided motion diffusion for controllable human motion synthesis. InProceedings of the IEEE/CVF international con- ference on computer vision, pages 2151–2162, 2023

    2023

  78. [2018]

    doi:10.1145/3197517.3201311

    ISSN 0730-0301. doi:10.1145/3197517.3201311. URLhttps://doi.org/10.1145/ 3197517.3201311

    arXiv

  79. [2026]

    URLhttps://arxiv.org/abs/2602.23205. 9

    arXiv

This paper was first reviewed by deepseek-v4-flash on August 2, 2026.