REVIEW 3 major objections 4 minor 114 references
Distributed HMD fusion — each wearer's ego-tracking plus another wearer's camera view — reconstructs full-body motion in the wild more accurately than either signal alone.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 22:05 UTC pith:Q2H4IDAQ
load-bearing objection Credible ego-exo fusion system; the known-shape assumption means the 'two glasses' pitch is ahead of what's evaluated, but the core result holds up. the 3 major comments →
EgoExoMoCap: Distributed Ego-Exo Human Motion Capture
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper claims that a distributed setup — two or more people wearing head-mounted devices — can capture full-body motion in the wild more accurately than ego-only or exo-only methods. On Nymeria, the fused model reports 5.72 cm MPJPE versus 6.47 cm for the strongest ego-exo baseline (3-point), and 8.28 versus 9.03 cm (1-point); a similar gap holds on EgoHumans, the exception being jitter, where a rolling-prediction baseline is smoother. The mechanism: a coarse ego-only pose localizes the wearer in the observer's image; 2D keypoints become 3D rays, scaled by inter-person distance and rotated into the wearer's head frame; a DINOv3-context gate scales each ray by learned visibility; spatial a
What carries the argument
The load-bearing machinery has three parts. Ray-based pose canonicalization unprojects each 2D keypoint from the observer's image into a 3D ray, scales it by observer–wearer head distance, and rotates it into the wearer's head-local frame, making the exocentric signal invariant to observer head motion and wearer global position. Learned visibility gating maps a DINOv3 CLS token of the cropped region through a small MLP to per-joint confidence scores that scale each ray, suppressing unreliable detections under occlusion or out-of-view. Ego-exo token fusion passes an Ego Token (egocentric tracking plus coarse EgoNet pose) and an Exo Token (gated rays) through a spatial transformer biased towar
Load-bearing premise
The reported absolute accuracies hold only if each subject's SMPL body shape is known in advance; the network predicts joint rotations, not shape, so with an unknown or wrong shape the joint-position errors would be larger than the numbers reported.
What would settle it
Recompute the reported MPJPEs with the ground-truth subject shape replaced by the SMPL mean body (or by a shape estimated from a single image). If the fused model's error rises toward the ego-only baseline, the claimed fusion advantage depends on knowing body shape in advance rather than on the fusion itself. Separately, in a long sequence with the wearer fully outside the observer's field of view, the DINO gate should reduce exo weights to near zero and output should match the ego-only fallback; any large divergence would falsify the gating claim.
If this is right
- Full-body motion capture in the wild works with two people wearing smart glasses; no camera rigs, mocap suits, or fixed infrastructure.
- Lower-body accuracy, the classic weakness of ego-only tracking, improves markedly: 10.05 cm lower-body MPJPE versus 13.96 cm for the best ego-only baseline (3-point, Nymeria).
- Adding more observers improves accuracy with no architectural change: on EgoHumans, multi-observer fusion reaches 7.11 cm MPJPE versus 8.80 cm for the best single-observer variant.
- The method tolerates missing wrist signals (1-point tracking) and degrades gracefully when the coarse egocentric prior is noisy: 10 cm of injected noise raises final MPJPE by only 0.54 cm.
- Because wrist signals are optional, the same pipeline applies to glasses-only wearers and to VR users with hand controllers.
Where Pith is reading between the lines
- Beyond the reported results, the shape assumption means a fully self-contained pipeline would need image- or sensor-based SMPL shape estimation; the reported MPJPEs are an upper bound until then.
- Beyond the paper, the DINO-context gating is a general mechanism for per-signal reliability, testable in any multi-view fusion setup that currently trusts 2D detector confidence.
- Beyond the paper, since global translation is recovered from HMD SLAM, absolute trajectory accuracy is bounded by localization drift; improving head tracking may matter more than further pose refinement.
- Beyond the paper, the framework yields pairwise relative motion in a group of wearers, which could enable social-interaction capture without external cameras.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. EgoExoMoCap proposes a distributed HMD-based human motion capture system. A target subject ('wearer') wears smart glasses providing head (and optionally wrist) tracking via SLAM, while one or more nearby observers wearing the same glasses provide RGB views of the wearer. An ego-only EgoNet produces coarse SMPL poses, which are used to crop the observer images. ViTPose 2D keypoints in those crops are unprojected to 3D rays, scaled by observer–wearer distance, canonicalized into the wearer's head frame (Section 3.4), and gated by DINOv3-derived confidence scores (Section 3.5). Ego and gated exo tokens are fused through spatial and temporal transformers to regress SMPL joint rotations; root translation is recovered analytically from the HMD head position. Training is two-stage with L1 losses on root orientation, joint rotations, and joint positions. Experiments on Nymeria and cross-dataset evaluation on EgoHumans compare against AvatarPoser, EgoPoser, EgoAllo, RPM, PromptHMR, and a naive ego-exo fusion baseline, in both 1-point and 3-point tracking settings, with ablations. The paper reports consistently lower MPJPE for the proposed method and claims that the approach is 'as simple as two (or more) people, each wearing a pair of smart glasses.'
Significance. The central idea is timely and the system design is well motivated. The paper provides a useful demonstration that combining egocentric device tracking with intermittent exocentric HMD views improves full-body pose accuracy without multi-camera rigs or mocap suits. The ray-based canonicalization is a clean solution to the observer ego-motion problem, and the DINOv3-based gating is a sensible, well-ablated component. The cross-dataset evaluation on EgoHumans and the robustness-to-EgoNet perturbation test are also valuable. However, two issues condition the headline claims: the absolute MPJPE numbers are computed with ground-truth subject-specific SMPL shape, and the EgoHumans 3-point evaluation uses wrist signals synthesized from ground-truth body parameters. If these are addressed, the contribution would be solid; as written, the practical 'two people wearing glasses' claim is stronger than the evidence supports.
major comments (3)
- [§3.1, §4.1, §4.4, Tables 1–2] The model does not predict SMPL shape parameters (Section 3.1), and Section 4.1 states that evaluation uses 'SMPL identity ground-truth parameters.' Section 4.4 acknowledges: 'we assume subject shape is provided when calculating the joint positions.' Consequently, the reported MPJPE values (e.g., 5.72 cm three-point and 8.28 cm one-point on Nymeria) are achievable only when the true body shape is known at inference. This is a load-bearing caveat for the abstract's claim that the system is 'as simple as two people wearing glasses': a shape-calibration or shape-estimation step is missing from the stated pipeline. Please report numbers with the SMPL mean shape or with shape estimated from the available image/sensor signals, and re-scope the accessibility claim accordingly. The relative fusion advantage may survive, but the absolute errors and the practical deployment claim are currently ove
- [§4.1, Table 2] For EgoHumans, the 3-point tracking results use 'synthesized 6DoF tracking signals from ground-truth body parameters' because the dataset does not provide wrist tracking. This bypasses the noise, dropout, and drift characteristics of real wrist trackers, and it can inflate the 3-point numbers and the apparent cross-dataset generality. Please evaluate on real or realistically perturbed wrist signals, or present EgoHumans results primarily in the 1-point setting. At minimum, add a noise/dropout analysis to quantify how sensitive the reported 3-point gains are to the synthetic-signal assumption.
- [§4.2, Tables 1–3] The central claim that 'across all metrics and both tracking setups, our method outperforms the baselines' rests on a single random split with no error bars or significance testing. The margins over the best ego-exo baseline are relatively small (e.g., Nymeria 5.72 vs 6.47 cm three-point, 8.28 vs 9.03 cm one-point). Report mean and standard deviation over subject-disjoint splits or bootstraps, and assess whether the reported ordering is statistically reliable. The ablation table has the same issue; without variance, the component-level conclusions are difficult to evaluate quantitatively.
minor comments (4)
- [§4.2, Tables 1–2] The sentence 'Across all metrics and both tracking setups, our method outperforms the baselines. The only exception is Jitter, where RPM achieves the best performance' is internally inconsistent: Tables 1 and 2 show RPM has lower Jitter in all configurations. Please qualify the 'all metrics' statement in the text.
- [Figure 6] The numeric labels inside the skeleton diagrams and the color scale for the DINO scores are not explained. Please add a legend or caption describing what the numbers represent and what the color mapping means.
- [Table 2] The relationship between the full-set 'EgoExoMoCap (Ours)' row and the subset rows marked with '*' (EgoExo-single-observer, EgoExo-triangulation, EgoExo-multi-observer) is unclear. Clarify whether the subset is the same across rows and whether the comparison is fair.
- [§4.1] The paper states that the baselines AvatarPoser and EgoPoser were 'improved' by predicting a sequence instead of just the last frame. This changes the original methods; please describe the modification in more detail and indicate whether the retrained baselines and data splits will be released to support reproducibility. The paper currently does not mention any code release.
Circularity Check
No significant circularity: the fusion pipeline is empirically trained with held-out and cross-dataset evaluation; the known-shape and analytic-root caveats are limitations, not circular inputs.
full rationale
The paper's central claim is an empirical one — that ego-exo fusion outperforms ego-only, exo-only, and naive fusion baselines on Nymeria and EgoHumans. The derivation chain is a learned pipeline (EgoNet coarse pose, ray lifting/canonicalization, DINOv3 gating, spatial/temporal transformers) whose outputs (joint rotations) are supervised by L_orient, L_rot, and L_pos (Eq. 7) on training subjects and evaluated on disjoint test subjects, with EgoHumans used as a zero-shot cross-dataset benchmark. No predicted quantity is defined in terms of the evaluation target by construction. The manually flagged limitations are not circular: Section 3.1 states 'the model does not predict SMPL shape parameters' and Section 4.4 says 'we mainly focus on motion reconstruction and assume subject shape is provided when calculating the joint positions' — shape is an input/identity condition, not a fitted parameter, and the paper explicitly acknowledges it could be estimated later. Similarly, Section 3.6 recovers root translation analytically from the known HMD head position (p_w,root_t = p_w,head_t - FK_head(theta_hat_t)); the paper does not claim to predict head translation, and the same convention is standard in the ego-pose baselines. Self-citations to AvatarPoser, EgoPoser, LAMP, and NymeriaPlus occur as baselines, datasets, or geometric design inspiration, not as load-bearing uniqueness arguments, so they do not make the central comparison circular. The only notable textual issue is Section 4.2's 'Across all metrics and both tracking setups, our method outperforms the baselines' immediately qualified by 'The only exception is Jitter' — an internal inconsistency in the claim wording, not a circularity in the evaluation. Overall, the paper is self-contained against external benchmarks for its central fusion claim, and the caveats affect absolute MPJPE interpretability, not the validity of the relative comparison.
Axiom & Free-Parameter Ledger
free parameters (4)
- EgoNet, Spatial/Temporal Transformer, and ScoreNet network weights =
not reported (millions of learned weights)
- Loss weights lambda_orient=0.02, lambda_rot=1.0, lambda_pos=1.0 =
0.02 / 1.0 / 1.0
- Temporal window T=96 =
96 frames
- Bounding-box expansion margin =
fixed margin, value not stated
axioms (5)
- standard math SMPL forward kinematics maps predicted joint rotations to 3D joint positions, and the body model is an adequate representation of real human bodies.
- domain assumption Aria-class HMDs provide accurate global head/wrist trajectories from visual-inertial SLAM, factory-calibrated cameras, and time-synchronized streams.
- domain assumption Subject-specific SMPL body shape is known at inference and evaluation.
- domain assumption ViTPose 2D keypoint detections and DINOv3 CLS features carry enough visual information about the wearer in the observer's image, even under occlusion.
- domain assumption Nymeria training data generalizes to EgoHumans for a zero-shot cross-dataset evaluation.
Cite this review
Pith. "Pith review of EgoExoMoCap: Distributed Ego-Exo Human Motion Capture." pith.science (2026). https://pith.science/paper/Q2H4IDAQ
@misc{pith2026260715868,
author = {Pith},
title = {Pith review of: EgoExoMoCap: Distributed Ego-Exo Human Motion Capture},
year = {2026},
howpublished = {\url{https://pith.science/paper/Q2H4IDAQ}},
note = {Machine review of arXiv:2607.15868}
}
read the original abstract
Human motion capture from head-mounted devices (HMDs) offers a scalable way to acquire real-world human motion and interaction data, which is crucial for applications in embodied AI and VR/AR. Existing approaches focus on either egocentric body tracking, estimating the motion of the subject wearing the device, or exocentric tracking, capturing the movements of people in the wearer's surroundings. So far, these two paradigms have largely been explored in isolation. In this paper, we propose a novel distributed framework that jointly leverages ego- and exocentric multi-modal signals for human motion estimation from HMDs. Unlike traditional motion capture systems requiring bulky multi-camera setups or obtrusive mocap suits, our approach, EgoExoMoCap, is as simple as two (or more) people, each wearing a pair of smart glasses. The method leverages head (plus potentially wrist) tracking signals for accurate estimation of global motion in the 3D world and combines context-aware image features based on DINOv3 to achieve robustness in the presence of noise and occlusions. Extensive experiments on two in-the-wild datasets show that our approach can robustly reconstruct motion even in challenging scenarios.
Figures
Reference graph
Works this paper leans on
-
[1]
Accessed: 2026-02-28
Aria gen2 glasses.https://ai.meta.com/blog/ aria-gen-2-research-glasses-under-the- hood-reality-labs/. Accessed: 2026-02-28. 2, 10
2026
-
[2]
Movella xsens.https : / / www . movella . com / motion - capture / xsens - link - specifications. Accessed: 2026-02-28. 7
2026
-
[3]
com/hololens, 2019
Microsoft HoloLens 2.https://www.microsoft. com/hololens, 2019. Accessed: 2026-02-28. 3
2019
-
[4]
Accessed: 2026-02-28
Meta Quest 3.https://www.meta.com/quest/ quest-3/, 2023. Accessed: 2026-02-28. 3
2023
-
[5]
Apple Vision Pro.https : / / www . apple . com / apple- vision- pro/, 2024. Accessed: 2026-02-28. 3
2024
-
[6]
Flag: Flow- based 3d avatar generation from sparse observations
Sadegh Aliakbarian, Pashmina Cameron, Federica Bogo, Andrew Fitzgibbon, and Thomas J Cashman. Flag: Flow- based 3d avatar generation from sparse observations. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 13253–13262, 2022. 3
2022
-
[7]
HMD-Nemo: Online 3d avatar motion generation from sparse observation
Sadegh Aliakbarian, Fatemeh Saleh, David Collier, Pash- mina Cameron, and Darren Cosker. HMD-Nemo: Online 3d avatar motion generation from sparse observation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 9622–9631, 2023. 3
2023
-
[8]
Circle: Capture in rich contextual environments
Joao Pedro Ara ´ujo, Jiaman Li, Karthik Vetrivel, Rishi Agarwal, Jiajun Wu, Deepak Gopinath, Alexander William Clegg, and Karen Liu. Circle: Capture in rich contextual environments. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 21211–21221, 2023. 2
2023
-
[9]
Accurately tracking rel- ative positions on moving trackers based on uwb ranging and inertial sensing without anchors
Rayan Armani and Christian Holz. Accurately tracking rel- ative positions on moving trackers based on uwb ranging and inertial sensing without anchors. In2024 IEEE/RSJ In- ternational Conference on Intelligent Robots and Systems (IROS). 10
-
[10]
Ultra Inertial Poser: Scalable motion capture and tracking from sparse inertial sensors and ultra-wideband ranging
Rayan Armani, Changlin Qian, Jiaxi Jiang, and Christian Holz. Ultra Inertial Poser: Scalable motion capture and tracking from sparse inertial sensors and ultra-wideband ranging. InACM SIGGRAPH 2024 Conference Papers,
2024
-
[11]
Multi-HMR: Multi-person whole-body hu- man mesh recovery in a single shot.European Conference on Computer Vision, 2024
Fabien Baradel, Matthieu Armando, Salma Galaaoui, Ro- main Br ´egier, Philippe Weinzaepfel, Gr ´egory Rogez, and Thomas Lucas. Multi-HMR: Multi-person whole-body hu- man mesh recovery in a single shot.European Conference on Computer Vision, 2024. 2
2024
-
[12]
From sparse signal to smooth motion: Real-time motion generation with rolling prediction models
German Barquero, Nadine Bertsch, Manojkumar Marram- reddy, Carlos Chac ´on, Filippo Arcadu, Ferran Rigual, Nicky Sijia He, Cristina Palmero, Sergio Escalera, Yuting Ye, et al. From sparse signal to smooth motion: Real-time motion generation with rolling prediction models. InPro- ceedings of the Computer Vision and Pattern Recognition Conference, pages 185...
2025
-
[13]
Bodiffusion: Diffusing sparse observations for full-body human motion synthesis
Angela Castillo, Maria Escobar, Guillaume Jeanneret, Al- bert Pumarola, Pablo Arbel ´aez, Ali Thabet, and Artsiom Sanakoyeu. Bodiffusion: Diffusing sparse observations for full-body human motion synthesis. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4221–4231, 2023. 3
2023
-
[14]
Motion capture from inertial and vision sensors.arXiv preprint arXiv:2407.16341, 2024
Xiaodong Chen, Wu Liu, Qian Bao, Xinchen Liu, Quanwei Yang, Ruoli Dai, and Tao Mei. Motion capture from inertial and vision sensors.arXiv preprint arXiv:2407.16341, 2024. 3
Pith/arXiv arXiv 2024
-
[15]
M2d2m: Multi-motion generation from text with discrete diffusion models
Seunggeun Chi, Hyung-gun Chi, Hengbo Ma, Nakul Agar- wal, Faizan Siddiqui, Karthik Ramani, and Kwonjoon Lee. M2d2m: Multi-motion generation from text with discrete diffusion models. InEuropean conference on computer vi- sion, pages 18–36. Springer, 2024. 2
2024
-
[16]
Estimating ego- body pose from doubly sparse egocentric video data.Ad- vances in neural information processing systems, 2024
Seunggeun Chi, Pin-Hao Huang, Enna Sachdeva, Hengbo Ma, Karthik Ramani, and Kwonjoon Lee. Estimating ego- body pose from doubly sparse egocentric video data.Ad- vances in neural information processing systems, 2024. 3
2024
-
[17]
Camera distortion-aware 3d human pose estimation in video with optimization-based meta-learning
Hanbyel Cho, Yooshin Cho, Jaemyung Yu, and Junmo Kim. Camera distortion-aware 3d human pose estimation in video with optimization-based meta-learning. InProceed- ings of the IEEE/CVF international conference on com- puter vision, pages 11169–11178, 2021. 5
2021
-
[18]
Pose2Mesh: Graph convolutional network for 3D human pose and mesh recovery from a 2D human pose
Hongsuk Choi, Gyeongsik Moon, and Kyoung Mu Lee. Pose2Mesh: Graph convolutional network for 3D human pose and mesh recovery from a 2D human pose. InEu- ropean Conference on Computer Vision, pages 769–787. Springer, 2020. 3
2020
-
[19]
Beyond static features for temporally con- sistent 3D human pose and shape from a video
Hongsuk Choi, Gyeongsik Moon, Ju Yong Chang, and Ky- oung Mu Lee. Beyond static features for temporally con- sistent 3D human pose and shape from a video. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1964–1973, 2021. 3
1964
-
[20]
Hmd-poser: On-device real-time human motion tracking from scalable sparse observations
Peng Dai, Yang Zhang, Tao Liu, Zhen Fan, Tianyuan Du, Zhuo Su, Xiaozheng Zheng, and Zeming Li. Hmd-poser: On-device real-time human motion tracking from scalable sparse observations. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, 2024. 3
2024
-
[21]
Daniel DeTone, Federica Bogo, Eric-Tuan Le, Duncan Frost, Julian Straub, Yawar Siddiqui, Yuting Ye, Jakob En- gel, Richard Newcombe, and Lingni Ma. Nymeriaplus: Enriching nymeria dataset with additional annotations and data.arXiv preprint arXiv:2603.18496, 2026. 7
arXiv 2026
-
[22]
Enhanc- ing egocentric 3d pose estimation with third person views
Ameya Dhamanaskar, Mariella Dimiccoli, Enric Corona, Albert Pumarola, and Francesc Moreno-Noguer. Enhanc- ing egocentric 3d pose estimation with third person views. Pattern Recognition, 138:109358, 2023. 2
2023
-
[23]
Full- body motion from a single head-mounted device: Generat- ing smpl poses from partial observations
Andrea Dittadi, Sebastian Dziadzio, Darren Cosker, Ben Lundell, Thomas J Cashman, and Jamie Shotton. Full- body motion from a single head-mounted device: Generat- ing smpl poses from partial observations. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 11687–11697, 2021. 3
2021
-
[24]
Realistic full-body motion generation from sparse tracking with state space model
Kun Dong, Jian Xue, Zehai Niu, Xing Lan, Ke Lu, Qingyuan Liu, and Xiaoyu Qin. Realistic full-body motion generation from sparse tracking with state space model. In Proceedings of the 32nd ACM International Conference on Multimedia, page 4024–4033. Association for Computing Machinery, 2024. 3 11
2024
-
[25]
Avatars grow legs: Generating smooth human motion from sparse track- ing inputs with diffusion model
Yuming Du, Robin Kips, Albert Pumarola, Sebastian Starke, Ali Thabet, and Artsiom Sanakoyeu. Avatars grow legs: Generating smooth human motion from sparse track- ing inputs with diffusion model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 3, 8
2023
-
[26]
Project aria: A new tool for egocentric multi-modal ai research
Jakob Engel, Kiran Somasundaram, Michael Goesele, Al- bert Sun, Alexander Gamino, Andrew Turner, Arjang Talat- tof, Arnie Yuan, Bilal Souti, Brighid Meredith, et al. Project aria: A new tool for egocentric multi-modal ai research. arXiv preprint arXiv:2308.13561, 2023. 2, 3, 7, 10
Pith/arXiv arXiv 2023
-
[27]
Stratified avatar generation from sparse observations
Han Feng, Wenchao Ma, Quankai Gao, Xianwei Zheng, Nan Xue, and Huijuan Xu. Stratified avatar generation from sparse observations. InProceedings of the IEEE con- ference on computer vision and pattern recognition, pages 153–163, 2024. 3
2024
-
[28]
Reconstructing and tracking humans with transformers.Proceedings of the IEEE/CVF International Conference on Computer Vision,
Shubham Goel, Georgios Pavlakos, Jathushan Rajasegaran, Angjoo Kanazawa, and Jitendra Malik. Reconstructing and tracking humans with transformers.Proceedings of the IEEE/CVF International Conference on Computer Vision,
-
[29]
Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives
Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Triantafyllos Afouras, Kumar Ashutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote, et al. Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 193...
2024
-
[30]
HMD 2: Environment-aware motion gen- eration from single egocentric head-mounted device
Vladimir Guzov, Yifeng Jiang, Fangzhou Hong, Gerard Pons-Moll, Richard Newcombe, C Karen Liu, Yuting Ye, and Lingni Ma. HMD 2: Environment-aware motion gen- eration from single egocentric head-mounted device. InIn- ternational Conference on 3D Vision (3DV), 2025. 2, 3
2025
-
[31]
BodySLAM: joint camera localisation, mapping, and human motion tracking
Dorian F Henning, Tristan Laidlow, and Stefan Leuteneg- ger. BodySLAM: joint camera localisation, mapping, and human motion tracking. InEuropean Conference on Com- puter Vision, pages 656–673, 2022. 3
2022
-
[32]
BodySLAM++: Fast and tightly- coupled visual-inertial camera and human motion track- ing
Dorian F Henning, Christopher Choi, Simon Schaefer, and Stefan Leutenegger. BodySLAM++: Fast and tightly- coupled visual-inertial camera and human motion track- ing. InIEEE/RSJ International Conference on Intelligent Robots and Systems, pages 3781–3788. IEEE, 2023. 3
2023
-
[33]
EgoSim: an egocentric multi-view simulator and real dataset for body-worn cameras during motion and activ- ity
Dominik Hollidt, Paul Streli, Jiaxi Jiang, Yasaman Haghighi, Changlin Qian, Xintong Liu, and Christian Holz. EgoSim: an egocentric multi-view simulator and real dataset for body-worn cameras during motion and activ- ity. InAdvances in Neural Information Processing Systems,
-
[34]
EgoLM: Multi-Modal Language Model of Egocentric Motions
Fangzhou Hong, Vladimir Guzov, Hyo Jin Kim, Yuting Ye, Richard Newcombe, Ziwei Liu, and Lingni Ma. EgoLM: Multi-Modal Language Model of Egocentric Motions. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, 2025. 3
2025
-
[35]
Ryan Hoque, Peide Huang, David J Yoon, Mouli Sivapu- rapu, and Jian Zhang. Egodex: Learning dexterous manip- ulation from large-scale egocentric video.arXiv preprint arXiv:2505.11709, 2025. 1
Pith/arXiv arXiv 2025
-
[36]
Deep inertial poser: Learning to reconstruct human pose from sparse inertial measurements in real time.ACM Trans- actions on Graphics (TOG), 37(6):1–15, 2018
Yinghao Huang, Manuel Kaufmann, Emre Aksan, Michael J Black, Otmar Hilliges, and Gerard Pons-Moll. Deep inertial poser: Learning to reconstruct human pose from sparse inertial measurements in real time.ACM Trans- actions on Graphics (TOG), 37(6):1–15, 2018. 3
2018
-
[37]
Andela Ilic, Jiaxi Jiang, Paul Streli, Xintong Liu, and Chris- tian Holz. Human motion capture from loose and sparse in- ertial sensors with garment-aware diffusion models.arXiv preprint arXiv:2506.15290, 2025. 3
Pith/arXiv arXiv 2025
-
[38]
Avatar- poser: Articulated full-body pose tracking from sparse mo- tion sensing
Jiaxi Jiang, Paul Streli, Huajian Qiu, Andreas Fender, Larissa Laich, Patrick Snape, and Christian Holz. Avatar- poser: Articulated full-body pose tracking from sparse mo- tion sensing. InProceedings of the European Conference on Computer Vision (ECCV), 2022. 2, 4, 6, 8, 9
2022
-
[39]
Manikin: biomechanically accurate neural inverse kinematics for human motion estimation
Jiaxi Jiang, Paul Streli, Xuejing Luo, Christoph Gebhardt, and Christian Holz. Manikin: biomechanically accurate neural inverse kinematics for human motion estimation. In European Conference on Computer Vision, pages 128–146. Springer, 2024. 2
2024
-
[40]
Egoposer: Robust real-time egocentric pose estimation from sparse and intermittent observations everywhere
Jiaxi Jiang, Paul Streli, Manuel Meier, and Christian Holz. Egoposer: Robust real-time egocentric pose estimation from sparse and intermittent observations everywhere. In European Conference on Computer Vision, 2024. 2, 3, 6, 8, 9
2024
-
[41]
End-to-end recovery of human shape and pose
Angjoo Kanazawa, Michael J Black, David W Jacobs, and Jitendra Malik. End-to-end recovery of human shape and pose. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 7122–7131,
-
[42]
Learning 3D human dynamics from video
Angjoo Kanazawa, Jason Y Zhang, Panna Felsen, and Ji- tendra Malik. Learning 3D human dynamics from video. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5614–5623, 2019. 3
2019
-
[43]
Egomimic: Scaling imitation learning via ego- centric video
Simar Kareer, Dhruv Patel, Ryan Punamiya, Pranay Mathur, Shuo Cheng, Chen Wang, Judy Hoffman, and Dan- fei Xu. Egomimic: Scaling imitation learning via ego- centric video. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 13226–13233. IEEE, 2025. 1
2025
-
[44]
EgoHumans: An Egocentric 3D Multi-Human Benchmark
Rawal Khirodkar, Aayush Bansal, Lingni Ma, Richard Newcombe, Minh V o, and Kris Kitani. EgoHumans: An Egocentric 3D Multi-Human Benchmark. In2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023. 2, 3, 8
2023
-
[45]
Harmony4D: a video dataset for in- the-wild close human interactions
Rawal Khirodkar, Jyun-Ting Song, Jinkun Cao, Zhengyi Luo, and Kris Kitani. Harmony4D: a video dataset for in- the-wild close human interactions. InProceedings of the 38th International Conference on Neural Information Pro- cessing Systems, 2024. 3
2024
-
[46]
VIBE: Video inference for human body pose and shape estimation
Muhammed Kocabas, Nikos Athanasiou, and Michael J Black. VIBE: Video inference for human body pose and shape estimation. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 5253–5263, 2020. 3
2020
-
[47]
PARE: Part attention regressor for 3D human body estimation
Muhammed Kocabas, Chun-Hao P Huang, Otmar Hilliges, and Michael J Black. PARE: Part attention regressor for 3D human body estimation. InProceedings of the 12 IEEE/CVF International Conference on Computer Vision, pages 11127–11137, 2021. 3
2021
-
[48]
Huang, Joachim Tesch, Lea M ¨uller, Otmar Hilliges, and Michael J
Muhammed Kocabas, Chun-Hao P. Huang, Joachim Tesch, Lea M ¨uller, Otmar Hilliges, and Michael J. Black. SPEC: Seeing people in the wild with an estimated camera. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11035–11045, 2021. 3
2021
-
[49]
PACE: Human and camera motion estimation from in-the-wild videos
Muhammed Kocabas, Ye Yuan, Pavlo Molchanov, Yun- rong Guo, Michael J Black, Otmar Hilliges, Jan Kautz, and Umar Iqbal. PACE: Human and camera motion estimation from in-the-wild videos. InInternational Conference on 3D Vision, pages 397–408, 2024. 3
2024
-
[50]
Learning to reconstruct 3D human pose and shape via model-fitting in the loop
Nikos Kolotouros, Georgios Pavlakos, Michael J Black, and Kostas Daniilidis. Learning to reconstruct 3D human pose and shape via model-fitting in the loop. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 2252–2261, 2019. 3
2019
-
[51]
Mocap everyone everywhere: Lightweight motion capture with smartwatches and a head- mounted camera
Jiye Lee and Hanbyul Joo. Mocap everyone everywhere: Lightweight motion capture with smartwatches and a head- mounted camera. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), pages 1091–1100, 2024. 3
2024
-
[52]
QuestEnvSim: Environment- Aware Simulated Motion Tracking from Sparse Sensors
Sunmin Lee, Sebastian Starke, Yuting Ye, Jungdam Won, and Alexander Winkler. QuestEnvSim: Environment- Aware Simulated Motion Tracking from Sparse Sensors. In ACM SIGGRAPH 2023 Conference Proceedings, 2023. 2
2023
-
[53]
Ego-body pose es- timation via ego-head pose estimation
Jiaman Li, Karen Liu, and Jiajun Wu. Ego-body pose es- timation via ego-head pose estimation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17142–17151, 2023. 3
2023
-
[54]
Object motion guided human motion synthesis.ACM Transactions on Graphics (TOG), 42(6):1–11, 2023
Jiaman Li, Jiajun Wu, and C Karen Liu. Object motion guided human motion synthesis.ACM Transactions on Graphics (TOG), 42(6):1–11, 2023. 2
2023
-
[55]
GENMO: Generative Models for Human Motion Synthesis
Jiefeng Li, Jinkun Cao, Haotian Zhang, Davis Rempe, Jan Kautz, Umar Iqbal, and Ye Yuan. GENMO: Generative Models for Human Motion Synthesis. InProceedings of the IEEE/CVF International Conference on Computer Vision,
-
[56]
CLIFF: Carrying location information in full frames into human pose and shape estimation
Zhihao Li, Jianzhuang Liu, Zhensong Zhang, Songcen Xu, and Youliang Yan. CLIFF: Carrying location information in full frames into human pose and shape estimation. In European Conference on Computer Vision, pages 590–606,
-
[57]
Mesh graphormer
Kevin Lin, Lijuan Wang, and Zicheng Liu. Mesh graphormer. InProceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 12939– 12948, 2021. 3
2021
-
[58]
EgoHDM: An Online Egocentric-Inertial Human Motion Capture, Local- ization, and Dense Mapping System.ACM Trans
Bonan Liu, Handi Yin, Manuel Kaufmann, Jinhao He, Sammy Christen, Jie Song, and Pan Hui. EgoHDM: An Online Egocentric-Inertial Human Motion Capture, Local- ization, and Dense Mapping System.ACM Trans. Graph., 43(6), 2024. 3
2024
-
[59]
Umotion: Uncertainty-driven human motion es- timation from inertial and ultra-wideband units
Huakun Liu, Hiroki Ota, Xin Wei, Yutaro Hirao, Mon- ica Perusquia-Hernandez, Hideaki Uchiyama, and Kiyoshi Kiyokawa. Umotion: Uncertainty-driven human motion es- timation from inertial and ultra-wideband units. InPro- ceedings of the Computer Vision and Pattern Recognition Conference, pages 7085–7094, 2025. 3
2025
-
[60]
4d human body capture from egocentric video via 3d scene grounding
Miao Liu, Dexin Yang, Yan Zhang, Zhaopeng Cui, James M Rehg, and Siyu Tang. 4d human body capture from egocentric video via 3d scene grounding. In2021 in- ternational conference on 3D vision (3DV), pages 930–939. IEEE, 2021. 2
2021
-
[61]
Smpl: A skinned multi-person linear model.ACM transactions on graphics (TOG), 34(6):1–16, 2015
Matthew Loper, Naureen Mahmood, Javier Romero, Ger- ard Pons-Moll, and Michael J Black. Smpl: A skinned multi-person linear model.ACM transactions on graphics (TOG), 34(6):1–16, 2015. 4
2015
-
[62]
Alireza Golestaneh, and Kris M
Zhengyi Luo, S. Alireza Golestaneh, and Kris M. Kitani. 3d human motion estimation via motion compression and re- finement. InProceedings of the Asian Conference on Com- puter Vision, 2020. 3
2020
-
[63]
Real-time simu- lated avatar from head-mounted sensors
Zhengyi Luo, Jinkun Cao, Rawal Khirodkar, Alexander Winkler, Kris Kitani, and Weipeng Xu. Real-time simu- lated avatar from head-mounted sensors. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 571–581, 2024. 2
2024
-
[64]
Nymeria: A massive collection of multimodal egocentric daily motion in the wild
Lingni Ma, Yuting Ye, Fangzhou Hong, Vladimir Guzov, Yifeng Jiang, Rowan Postyeni, Luis Pesqueira, Alexander Gamino, Vijay Baiyya, Hyo Jin Kim, et al. Nymeria: A massive collection of multimodal egocentric daily motion in the wild. InEuropean Conference on Computer Vision, pages 445–465, 2024. 2, 3, 6, 7, 10
2024
-
[65]
IMUPoser: Full-Body Pose Esti- mation Using IMUs in Phones, Watches, and Earbuds
Vimal Mollyn, Riku Arakawa, Mayank Goel, Chris Harri- son, and Karan Ahuja. IMUPoser: Full-Body Pose Esti- mation Using IMUs in Phones, Watches, and Earbuds. In Proceedings of the 2023 CHI Conference on Human Fac- tors in Computing Systems, 2023. 3
2023
-
[66]
I2L-MeshNet: Image-to-lixel prediction network for accurate 3d human pose and mesh estimation from a single RGB image
Gyeongsik Moon and Kyoung Mu Lee. I2L-MeshNet: Image-to-lixel prediction network for accurate 3d human pose and mesh estimation from a single RGB image. In European Conference on Computer Vision, pages 752–768,
-
[67]
Fusing monoc- ular images and sparse imu signals for real-time human mo- tion capture
Shaohua Pan, Qi Ma, Xinyu Yi, Weifeng Hu, Xiong Wang, Xingkang Zhou, Jijunnan Li, and Feng Xu. Fusing monoc- ular images and sparse imu signals for real-time human mo- tion capture. InSIGGRAPH Asia 2023 Conference Papers,
2023
-
[68]
Expressive body capture: 3D hands, face, and body from a single image
Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed AA Osman, Dimitrios Tzionas, and Michael J Black. Expressive body capture: 3D hands, face, and body from a single image. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10975–10985, 2019. 3
2019
-
[69]
The virtual caliper: Rapid creation of metrically accurate avatars from 3d measurements.IEEE transactions on visualization and computer graphics, 25 (5):1887–1897, 2019
Sergi Pujades, Betty Mohler, Anne Thaler, Joachim Tesch, Naureen Mahmood, Nikolas Hesse, Heinrich H B ¨ulthoff, and Michael J Black. The virtual caliper: Rapid creation of metrically accurate avatars from 3d measurements.IEEE transactions on visualization and computer graphics, 25 (5):1887–1897, 2019. 10
2019
-
[70]
Ryan Punamiya, Simar Kareer, Zeyi Liu, Josh Citron, Ri- Zhao Qiu, Xiongyi Cai, Alexey Gavryushin, Jiaqi Chen, Davide Liconti, Lawrence Y Zhu, et al. Egoverse: An ego- centric human dataset for robot learning from around the world.arXiv preprint arXiv:2604.07607, 2026. 1
Pith/arXiv arXiv 2026
-
[71]
13 World-grounded human motion recovery via gravity-view coordinates
Zehong Shen, Huaijin Pi, Yan Xia, Zhi Cen, Sida Peng, Zechen Hu, Hujun Bao, Ruizhen Hu, and Xiaowei Zhou. 13 World-grounded human motion recovery via gravity-view coordinates. InSIGGRAPH Asia, 2024. 2, 3
2024
-
[72]
Caring-ai: Towards authoring context-aware augmented re- ality instruction through generative artificial intelligence
Jingyu Shi, Rahul Jain, Seunggeun Chi, Hyungjun Doh, Hyung-gun Chi, Alexander J Quinn, and Karthik Ramani. Caring-ai: Towards authoring context-aware augmented re- ality instruction through generative artificial intelligence. In Proceedings of the 2025 CHI conference on human factors in computing systems, pages 1–23, 2025. 2
2025
-
[73]
Modi Shi, Shijia Peng, Jin Chen, Haoran Jiang, Yinghui Li, Di Huang, Ping Luo, Hongyang Li, and Li Chen. Egohumanoid: Unlocking in-the-wild loco-manipulation with robot-free egocentric demonstration.arXiv preprint arXiv:2602.10106, 2026. 1
Pith/arXiv arXiv 2026
-
[74]
Wham: Reconstructing world-grounded humans with accurate 3d motion
Soyong Shin, Juyong Kim, Eni Halilaj, and Michael J Black. Wham: Reconstructing world-grounded humans with accurate 3d motion. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2070–2080, 2024. 2, 3
2070
-
[75]
Dinov3.arXiv preprint arXiv:2508.10104, 2025
Oriane Sim ´eoni, Huy V V o, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Micha ¨el Ramamonjisoa, et al. Dinov3.arXiv preprint arXiv:2508.10104, 2025. 2, 4
Pith/arXiv arXiv 2025
-
[76]
Categorical codebook matching for embod- ied character controllers.ACM Transactions on Graphics (TOG), 43(4):1–14, 2024
Sebastian Starke, Paul Starke, Nicky He, Taku Komura, and Yuting Ye. Categorical codebook matching for embod- ied character controllers.ACM Transactions on Graphics (TOG), 43(4):1–14, 2024. 3
2024
-
[77]
Human mesh recovery from monocular images via a skeleton-disentangled representation
Yu Sun, Yun Ye, Wu Liu, Wenpeng Gao, Yili Fu, and Tao Mei. Human mesh recovery from monocular images via a skeleton-disentangled representation. InProceedings of the IEEE/CVF International Conference on Computer Vision,
-
[78]
DROID-SLAM: Deep vi- sual slam for monocular, stereo, and RGB-D cameras
Zachary Teed and Jia Deng. DROID-SLAM: Deep vi- sual slam for monocular, stereo, and RGB-D cameras. Advances in Neural Information Processing Systems, 34: 16558–16569, 2021. 3
2021
-
[79]
Deep patch vi- sual odometry.Advances in Neural Information Processing Systems, 36, 2024
Zachary Teed, Lahav Lipson, and Jia Deng. Deep patch vi- sual odometry.Advances in Neural Information Processing Systems, 36, 2024. 3
2024
-
[80]
Real-time inverse kinematics techniques for anthropomor- phic limbs.Graphical models, 62(5):353–388, 2000
Deepak Tolani, Ambarish Goswami, and Norman I Badler. Real-time inverse kinematics techniques for anthropomor- phic limbs.Graphical models, 62(5):353–388, 2000. 2
2000
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.