Pith. sign in

REVIEW 3 major objections 6 minor 50 references

Full-body 3D motion can be reconstructed from any mix of everyday wearables—phones, watches, pressure insoles, and a headset—using one generative model.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 15:48 UTC pith:7F6GIUY7

load-bearing objection Solid multi-modal consumer-wearable MoCap dataset + any-subset generative baseline and complementarity map; the real soft spot is idealized per-block mounting recalibration under a real-world claim, not the tables themselves. the 3 major comments →

arxiv 2607.09780 v1 pith:7F6GIUY7 submitted 2026-07-08 cs.CV cs.LG

Towards Real-World Wearable Motion Reconstruction

Andrea Boscolo Camiletto , Rishabh Dabral , Eduardo Alvarado , Thabo Beeler , Marc Habermann , Christian Theobalt This is my paper
classification cs.CV cs.LG
keywords motion capturewearable sensorsmultimodal learninghuman pose estimationflow matchingsensor complementarityconsumer devices
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Most motion-capture research assumes a fixed sensor suit or headset setup. This paper argues the practical problem is different: people already wear shifting combinations of phones, watches, smart insoles, and head-mounted devices, so reconstruction must work from whatever subset is present. The authors release a multi-modal dataset that time-aligns those consumer sensors with high-quality multi-camera 3D motion across 50 everyday, sports, and social activities. They introduce WHIP, a flow-matching generative model with per-modality attention that accepts any sensor subset, skips missing ones, and produces full-body motion. A systematic complementarity study then measures each sensor’s marginal value and pairwise synergy, identifying which device combinations give the best accuracy under hardware limits.

Core claim

A single generative model can map arbitrary subsets of commodity wearable signals—smartphone and smartwatch IMUs, plantar pressure insoles, and headset 6-DoF pose—into physically plausible full-body 3D motion, outperforming matched regression baselines and prior specialized methods when all are trained on the same multi-modal data, while a large synchronized dataset makes the first quantitative cross-modal complementarity analysis possible.

What carries the argument

WHIP: a Diffusion-Transformer-style flow-matching network whose blocks run dedicated cross-attention modules only for the sensors that are present; modality dropout at training time teaches the model to ignore missing inputs, and motion is obtained by integrating a learned velocity field from noise to data.

Load-bearing premise

Phone and watch orientations are re-aligned to ground-truth joint rotations for every 20-second action block—an idealized correction that would not be available in true everyday use without frequent recalibration.

What would settle it

Run continuous multi-minute sessions with only a single initial calibration (no per-block remounting of phone or watch offsets) and check whether N-MPJPE rises enough to erase the reported gains over specialized fixed-sensor methods.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Motion capture can adapt on the fly to whichever phones, watches, insoles, or headsets a person is wearing instead of requiring a fixed suit.
  • Pressure insoles supply lower-body contact information that is largely complementary to IMUs and head tracking.
  • Headset tracking alone recovers metric scale better than any other single device; watches and phones then fill in arms and legs.
  • For each device count there exist Pareto-optimal sensor combinations that balance accuracy against how many gadgets a user must wear.
  • Modeling motion as a conditional generative process yields lower reconstruction error than direct regression under the same sparse, heterogeneous inputs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Without periodic ground-truth remounting, real pocket and strap drift may shrink the reported accuracy gap more than studio numbers suggest.
  • The marginal-score tables could drive adaptive fusion policies that power only the most informative sensors for battery or privacy.
  • The same per-modality attention design could be tested for streaming, on-device latency to see whether generative reconstruction is viable for everyday AR.
  • Insole contact cues might transfer as weak supervision to other wearable datasets that lack pressure sensors.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper argues that full-body motion capture should work from arbitrary subsets of everyday consumer wearables rather than fixed IMU suits or HMD-centric rigs. It contributes (i) a ~7-hour multimodal dataset of 14 subjects performing 50 actions, synchronizing two phones, two watches, pressure insoles, and a Quest 3 HMD with 120-camera markerless MoCap; (ii) WHIP, a DiT-style conditional flow-matching model with per-modality cross-attention that reconstructs 3D joint trajectories from any available sensor subset (with random modality dropout at train time); and (iii) a systematic complementarity study over all 128 sensor combinations, including marginal scores, pairwise interactions, and a Pareto frontier of device count vs. N-MPJPE. Empirically, WHIP beats parameter-matched MLP/Transformer regressors (Table 2) and prior methods retrained on the same data (Table 3), with qualitative in-the-wild examples and a Nymeria transfer check.

Significance. If the results hold under realistic deployment conditions, this is a useful step toward consumer-grade, configuration-agnostic wearable MoCap. The dataset fills a genuine gap: prior resources are siloed by modality, often lack high-quality GT, or use intrusive full IMU suits. The arbitrary-subset formulation and the exhaustive complementarity analysis (marginal scores, pairwise I(a,b), Pareto frontier) are more systematic than typical fixed-config papers. Code and data release, retraining of external baselines on the same data, and a generative formulation that handles missing modalities by design are concrete strengths. The main significance risk is that quantitative claims are measured under an idealized spatial registration regime that everyday phone-in-pocket / watch-on-wrist use does not provide.

major comments (3)
  1. [Sec. 3.2; Tables 2–3; Figs. 6–8] Sec. 3.2 (Phones and Watches) recomputes the mounting offset R^{D,S}_r from MoCap joint orientations for every 20 s action block, and the authors explicitly call this an “idealized setting.” All primary quantitative results—Tables 2–3, the 128-subset complementarity study, Figs. 6–8, and the N-MPJPE gaps marketed as real-world (e.g., 61.7 vs 75.5 IMU-only)—are therefore conditioned on near-perfect spatial registration that removes pocket/wrist drift. Only qualitative Fig. 5 uses coarser manual calibration. This is load-bearing for the paper’s “real-world wearable” framing (title, abstract, intro). Please either (a) report a controlled ablation with fixed session-level (or longer-horizon) mounting offsets and/or injected orientation bias, or (b) substantially soften the real-world claims and restate the contribution as complementarity under clean registration, with residual mounting error
  2. [Sec. 5.4; Fig. 5] In-the-wild evidence is only qualitative (Fig. 5, “manual calibration procedure”). Given that the central pitch is unconstrained everyday capture, at least a small quantitative wild or semi-wild evaluation—e.g., against a portable optical/marker reference, or a controlled drift protocol—would substantially strengthen the claim that the learned sensor-to-motion map survives domain shift. Without it, the real-world generalization argument rests almost entirely on studio metrics under idealized alignment.
  3. [Sec. 5; Eq. (7); Table 2] Sec. 5 (Metrics / Eq. 7): generative WHIP is evaluated via the posterior mean over K=10 samples and compared to deterministic regressors under L2 pose metrics. This is a reasonable Bayes estimator under L2, but the paper should also report single-sample (or best-of-1) performance and, ideally, a diversity/plausibility metric (e.g., contact/footskate, bone-length consistency, or sample variance), so readers can separate “mean is accurate” from “samples are usable.” As written, the generative advantage in Table 2 could partly reflect averaging rather than better single reconstructions.
minor comments (6)
  1. [Table 1; Sec. 3.3] Table 1: “Ours” lists 7 hours and 14 subjects; clarify whether the held-out subject/actions are excluded from the hour count and whether both sessions per subject are always used in train/test splits.
  2. [Sec. 4.2; Eq. (6)] Eq. (6): the 1/√|M| residual normalization is sensible; a short ablation (or citation to multi-source residual practice) would help readers judge sensitivity to this choice.
  3. [Sec. 5.3; Table 4] Nymeria transfer (Sec. 5.3 / Table 4) substitutes pelvis IMU for phones and labeled contacts for pressure; state clearly that this is an input proxy study, not a like-for-like sensor transfer, so numbers are not comparable to published Nymeria baselines.
  4. [Fig. 7; Sec. 5.5] Fig. 7 caption says “Higher values indicate greater synergy,” but the text defines I(a,b) ≤ 0 with more negative = more redundancy. Align caption language with the signed definition (e.g., “less negative = more complementary”).
  5. [Sec. 5.5; Fig. 8] Action labels are included as a modality in the 128-subset study but are not a wearable sensor; consider reporting the Pareto frontier both with and without action labels so hardware-only trade-offs are visible.
  6. [Abstract; Fig. 2] Minor typos / polish: “Themodern-daysurge” spacing in abstract; “W earable” in Fig. 2 caption; inconsistent “smart glasses” in abstract vs. HMD/Quest 3 in the body.

Circularity Check

1 steps flagged

Empirical ML paper with held-out metrics and standard conditioning; no derivation reduces to its inputs by construction.

specific steps
  1. self citation load bearing [Sec. 3.2 Headset calibration, Eq. (1)]
    "Following Camiletto et al. [4], we attach an ArUco board to the HMD to align it with the MoCap. By tracking the board’s pose Nt ∈ SE(3) from the studio cameras and collecting the HMD inside-out tracking Ht ∈ SE(3), we solve T∗c , T∗r , t∗0 = arg min ..."

    The cited calibration procedure is prior work by largely the same author list. It is used only for spatial/temporal registration of one sensor and is not invoked as a uniqueness result that forces the motion-reconstruction claims or the complementarity scores; the main empirical package remains independent of it.

full rationale

WHIP and the complementarity analysis are ordinary supervised generative modeling: a DiT-style flow-matching model is trained with random modality dropout on the authors' multi-sensor sequences, then evaluated by posterior-mean sampling on a held-out subject and held-out actions (Tables 2–3, Figs. 6–8). Reported N-MPJPE/MPJPE numbers are ordinary prediction errors against independent multi-view MoCap labels; they are not algebraically forced by any fitted parameter that is later re-labeled a prediction. The per-block recomputation of mounting offsets R_r (Sec. 3.2) is an idealized calibration step that the authors themselves flag; it affects the strength of the real-world claim but does not make the tabulated errors tautological. The single self-citation used for HMD spatial alignment (Camiletto et al. FRAME) is a practical engineering choice, not a uniqueness theorem or load-bearing premise that forbids alternatives. No self-definitional identities, no fitted-input-as-prediction, and no renaming of known results appear. Score 1 only for the minor, non-load-bearing self-reference.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 2 invented entities

Load-bearing content is empirical: a new dataset, a conditional flow-matching model, and combinatorial evaluation. Free parameters are training/architecture choices and evaluation sample count. Axioms are standard ML and sensing assumptions plus the paper's idealized on-body calibration regime. Invented entities are the dataset resource and the WHIP architecture as named systems, not new physical objects.

free parameters (4)
  • Modality keep probability during training = 0.5
    Each sensor retained with probability 0.5 during training; chosen to balance sparse vs full configurations and directly shapes missing-modality robustness.
  • Posterior-mean sample count K = 10
    Evaluation averages K=10 generative samples as L2 Bayes estimator; reported metrics depend on this choice versus single-sample inference.
  • WHIP architecture and training hyperparameters = as in Sec. 5 Implementation Details
    8 DiT blocks, d=768, 12 heads, per-modality latent 64, T=90 frames, AdamW lr 1e-3, 110k WSD steps—hand-chosen design that determines capacity and reported accuracy.
  • Per-block mounting offset R_r = re-estimated per 20s block
    Mounting rotation recomputed every 20 s action block from GT joint orientations; not a learned model weight but a fitted calibration parameter that removes real placement drift from the main evaluation.
axioms (5)
  • domain assumption Conditional flow matching with linear paths and Euler integration yields a usable conditional motion distribution from sparse wearable signals.
    Sec. 4.1–4.3 adopts Lipman et al. flow matching without proving optimality for this sensor suite; performance is empirical.
  • domain assumption Device-local IMU frames (no shared global magnetometer frame) plus offline spatial/temporal calibration suffice for supervised learning.
    Sec. 3.1–3.2 reject global magnetometer frames as noisy and rely on studio-aligned calibration sequences.
  • ad hoc to paper Recomputing phone/watch mounting offsets every 20 s from MoCap is an acceptable evaluation regime for studying sensor complementarity.
    Sec. 3.2 explicitly calls this idealized; the real-world claim depends on this premise remaining approximately true or being replaceable by online calibrators.
  • ad hoc to paper Averaging multiple generative samples is a fair comparison to deterministic regressors under L2 pose metrics.
    Sec. 5 Evaluation justifies K=10 mean as Bayes estimator; single-sample or latency-constrained use may differ.
  • standard math Standard SO(3)/SE(3) geodesic alignment and multi-view markerless MoCap provide reliable ground truth for training and metrics.
    Calibration objectives (Eqs. 1–2) and Captury GT are conventional in the field.
invented entities (2)
  • WHIP model no independent evidence
    purpose: Named baseline generative architecture with per-modality cross-attention that accepts arbitrary wearable subsets.
    Architectural composition of DiT + flow matching + modality dropout; independent evidence is empirical performance on held-out splits, not a new physical law.
  • Authors' multi-modal wearable MoCap dataset independent evidence
    purpose: Synchronized phones, watches, insoles, HMD, and multi-camera GT for complementarity study and training.
    New resource claimed available at project URL; falsifiable via public release and re-use by others.

pith-pipeline@v1.1.0-grok45 · 19419 in / 3713 out tokens · 37686 ms · 2026-07-14T15:48:39.317152+00:00 · methodology

0 comments
read the original abstract

The modern-day surge in popularity of wearable devices poses a fundamentally unique motion capture problem: reconstructing full-body movement from any set of sensing hardware worn at a given moment. Yet, most research efforts assume fixed sensor configurations (e.g. IMU suits or HMD-centric rigs) and cannot generalize across them. In contrast, we argue that motion capture should prioritize unobtrusive and lightweight devices such as smartphones, smartwatches, smart glasses, and smart insoles, and study the interplay between them. To this end, we make three contributions. First, we present a large-scale multi-modal dataset synchronizing these consumer-grade sensors with ground-truth 3D motion, spanning 50 diverse activities including everyday tasks, sports, and social interactions. Second, we propose WHIP, a baseline generative model that reconstructs motion from arbitrary subsets of available sensors, robustly handling missing modalities and producing physically plausible motions. Third, we conduct a systematic study of sensor complementarity, quantifying how different modalities complement one another. Code and dataset are available at https://vcai.mpi-inf.mpg.de/projects/WHIP/

Figures

Figures reproduced from arXiv: 2607.09780 by Andrea Boscolo Camiletto, Christian Theobalt, Eduardo Alvarado, Marc Habermann, Rishabh Dabral, Thabo Beeler.

Figure 1
Figure 1. Figure 1: We present a multimodal wearable sensor suite for motion capture (left) and show full-body motion reconstructed by our model from sparse sensor readings (right). Abstract. The modern-day surge in popularity of wearable devices poses a fundamentally unique motion capture problem: reconstructing full￾body movement from any set of sensing hardware worn at a given mo￾ment. Yet, most research efforts assume fix… view at source ↗
Figure 2
Figure 2. Figure 2: Wearable capture rig. Two smart in￾soles (pressure + IMU), two smartwatches and two smartphones (IMUs), and an HMD (6-DoF pose), all synchronized with markerless MoCap. This requires properties that ex￾isting datasets lack: sparse, low￾overhead and diverse sensing. To address this gap, we collected a multimodal dataset in which participants wore two smartphones, two smartwatches, a head-mounted device, and… view at source ↗
Figure 3
Figure 3. Figure 3: WHIP model architecture. (a) High-level overview, from noise to the velocity field. (b) DiT transformer block. (c) Sensor-specific cross-attention modules. (d) Con￾ditioning information. consequently governed by the ordinary differential equation \frac {d}{d\tau }\psi _\tau (x) = u_\tau (\psi _\tau (x)), \label {eq:ode} (3) where ψτ denotes the flow map from p0 to pτ . Given sensor observations C and groun… view at source ↗
Figure 4
Figure 4. Figure 4: WHIP on Nymeria. Wrist signals recover arm motion missed by VR-only reconstruction. 5.4 In-the-Wild Qualitative Results To demonstrate applicability beyond controlled settings, we recorded additional sequences in unconstrained environments using a manual calibration procedure [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative results on in-the-wild recordings. Despite the domain shift, WHIP produces plausible motion reconstructions. The figure shows that (i) HMD contributes the most across all contexts, partly because it is the only sensor that recovers metric scale. When evaluated with N-MPJPE, its contribution, although still substantial, is less pronounced. (ii) Smartwatches are the second most important sensor o… view at source ↗
Figure 7
Figure 7. Figure 7: Pairwise sensor interactions on N-MPJPE. Higher values indicate greater synergy. where Xa := X ∪ {a}, Xb := X ∪ {b}, and Xab := X ∪ {a, b}. Since adding a sensor can only reduce (or maintain) the error, I(a, b) ≤ 0: it is zero when the two sensors are perfectly complementary, i.e., their joint gain equals the sum of their individual gains, and increasingly negative as their contributions overlap (redundanc… view at source ↗
Figure 6
Figure 6. Figure 6: Sensor importance via marginal scores across four contexts: full body, legs, arms, and normalized full body. Bars show the error reduction contributed by each modality (larger is better). MPJPE. We observe that, while the insoles contribute modestly when consid￾ered alone, they exhibit the strongest synergy when paired with other modalities, providing information most distinct from other sensors. Best-k-ou… view at source ↗
Figure 8
Figure 8. Figure 8: Pareto frontier of N-MPJPE versus number of devices. Each point represents a sensor combination; the frontier highlights the best subset for each device count. datasets in sensor diversity and scale. Our dataset enables unobtrusive, real￾world configurations and supports a systematic analysis of sensor complemen￾tarity, quantifying how modalities interact and reinforce one another. Together, these contribu… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

50 extracted references · 5 canonical work pages

  1. [1]

    Sensors12(7), 9884–9912 (2012).https://doi

    Abdul Razak, A.H., Zayegh, A., Begg, R.K., Wahab, Y.: Foot plantar pressure measurement system: A review. Sensors12(7), 9884–9912 (2012).https://doi. org/10.3390/s120709884,https://www.mdpi.com/1424-8220/12/7/9884

    work page doi:10.3390/s120709884 2012

  2. [2]

    In: European Conference on Computer Vision (ECCV) (2022)

    Akada, H., Wang, J., Shimada, S., Takahashi, M., Theobalt, C., Golyanik, V.: Unrealego: A new dataset for robust egocentric 3d human motion capture. In: European Conference on Computer Vision (ECCV) (2022)

    2022

  3. [3]

    com / documentation / coremotion / getting - processed - device - motion - data (2025), accessed: 2026-06-29

    Apple Inc.: Getting processed device-motion data.https://developer.apple. com / documentation / coremotion / getting - processed - device - motion - data (2025), accessed: 2026-06-29

    2025

  4. [4]

    CVPR (2025)

    Boscolo Camiletto, A., Wang, J., Alvarado, E., Dabral, R., Beeler, T., Habermann, M., Theobalt, C.: Frame: Floor-aligned representation for avatar motion from ego- centric video. CVPR (2025)

    2025

  5. [5]

    Accessed: 2026-06-29 16 A

    The Captury GmbH, Saarbrücken, Germany: The Captury: Markerless Motion Capture System (2023), available athttps://www.thecaptury.com. Accessed: 2026-06-29 16 A. Boscolo Camiletto et al

    2023

  6. [6]

    In: Proceedings of the IEEE/CVF International Confer- ence on Computer Vision Workshops (ICCVW)

    Castillo, A., Escobar, M., Jeanneret, G., Pumarola, A., Arbeláez, P., Thabet, A., Sanakoyeu, A.: Bodiffusion: Diffusing sparse observations for full-body hu- man motion synthesis. In: Proceedings of the IEEE/CVF International Confer- ence on Computer Vision Workshops (ICCVW). pp. 4223–4233 (2023).https: //doi.org/10.1109/ICCVW60793.2023.00456

    doi:10.1109/iccvw60793.2023.00456 2023

  7. [7]

    In: ProceedingsoftheIEEE/CVFConferenceonComputerVisionandPatternRecog- nition (CVPR)

    Dai, P., Zhang, Y., Liu, T., Fan, Z., Du, T., Su, Z., Zheng, X., Li, Z.: Hmd-poser: On-device real-time human motion tracking from scalable sparse observations. In: ProceedingsoftheIEEE/CVFConferenceonComputerVisionandPatternRecog- nition (CVPR). pp. 874–884 (2024).https://doi.org/10.1109/CVPR52733.2024. 00089

    doi:10.1109/cvpr52733.2024 2024

  8. [8]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Du,Y.,Kips,R.,Pumarola,A.,Starke,S.,Thabet,A.,Sanakoyeu,A.:Avatarsgrow legs: Generating smooth human motion from sparse tracking inputs with diffusion model. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 481–490 (2023).https://doi.org/10.1109/ CVPR52729.2023.00054

    arXiv 2023

  9. [9]

    Sensors 20(21), 6330 (2020).https://doi.org/10.3390/s20216330

    Geissinger, J.H., Asbeck, A.T.: Motion inference using sparse inertial sensors, self-supervised learning, and a new dataset of unscripted human motion. Sensors 20(21), 6330 (2020).https://doi.org/10.3390/s20216330

    work page doi:10.3390/s20216330 2020

  10. [10]

    Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Ham- burger, J., Jiang, H., Liu, M., Liu, X., Martin, M., Nagarajan, T., Radosavovic, I., Ramakrishnan, S.K., Ryan, F., Sharma, J., Wray, M., Xu, M., Xu, E.Z., Zhao, C., Bansal, S., Batra, D., Cartillier, V., Crane, S., Do, T., Doulaty, M., Erapalli, A., Feichtenhofer, C., Fragome...

    Pith/arXiv arXiv 2022

  11. [11]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Grauman, K., Westbury, A., Torresani, L., Kitani, K., Malik, J., Afouras, T., Ashutosh, K., Baiyya, V., Bansal, S., Boote, B., et al.: Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 19383–19400 (2024)

    2024

  12. [12]

    In: SIGGRAPH Asia 2023 Conference Papers

    Han, X., Senderling, B., To, S., Kumar, D., Whiting, E., Saito, J.: Groundlink: A dataset unifying human body movement and ground reaction dynamics. In: SIGGRAPH Asia 2023 Conference Papers. p. 1–10. SA ’23, ACM (Dec 2023). https://doi.org/10.1145/3610548.3618247,http://dx.doi.org/10.1145/ 3610548.3618247

    doi:10.1145/3610548.3618247 2023

  13. [13]

    ACM Transactions on Graphics, (Proc

    Huang, Y., Kaufmann, M., Aksan, E., Black, M.J., Hilliges, O., Pons-Moll, G.: Deep inertial poser learning to reconstruct human pose from sparse inertial mea- surements in real time. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia) 37(6), 185:1–185:15 (Nov 2018)

    2018

  14. [14]

    In: Eu- ropean Conference on Computer Vision (ECCV)

    Jiang, J., Streli, P., Qiu, H., Fender, A., Laich, L., Snape, P., Holz, C.: Avatar- poser: Articulated full-body pose tracking from sparse motion sensing. In: Eu- ropean Conference on Computer Vision (ECCV). pp. 443–460 (2022).https: //doi.org/10.1007/978-3-031-20065-6_26 Towards Real-World Wearable Motion Reconstruction 17

    work page doi:10.1007/978-3-031-20065-6_26 2022

  15. [15]

    In: SIGGRAPH Asia 2022 Conference Pa- pers

    Jiang, Y., Ye, Y., Gopinath, D.E., Won, J., Winkler, A.W., Liu, C.K.: Trans- former inertial poser: Real-time human motion reconstruction from sparse imus with simultaneous terrain generation. In: SIGGRAPH Asia 2022 Conference Pa- pers. pp. 1–9 (2022).https://doi.org/10.1145/3550469.3555428

    doi:10.1145/3550469.3555428 2022

  16. [16]

    In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR) (2024)

    Lee, J., Joo, H.: Mocap everyone everywhere: Lightweight motion capture with smartwatches and a head-mounted camera. In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR) (2024)

    2024

  17. [17]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Li, J., Liu, K., Wu, J.: Ego-body pose estimation via ego-head pose estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 17142–17151 (2023)

    2023

  18. [18]

    arXiv preprint arXiv:2210.02747 (2022)

    Lipman, Y., Chen, R.T., Ben-Hamu, H., Nickel, M., Le, M.: Flow matching for generative modeling. arXiv preprint arXiv:2210.02747 (2022)

    Pith/arXiv arXiv 2022

  19. [19]

    In: International Conference on Learning Representations (ICLR) (2019)

    Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (ICLR) (2019)

    2019

  20. [20]

    Scientific Data9(473) (2022).https://doi.org/10

    Losing, V., Hasenjäger, M.: A multi-modal gait database of natural everyday-walk in an urban environment. Scientific Data9(473) (2022).https://doi.org/10. 1038/s41597-022-01580-3

    2022

  21. [21]

    Ma, L., Ye, Y., Hong, F., Guzov, V., Jiang, Y., Postyeni, R., Pesqueira, L., Gamino, A., Baiyya, V., Kim, H.J., Bailey, K., Fosas, D.S., Liu, C.K., Liu, Z., Engel, J., Nardi, R.D., Newcombe, R.: Nymeria: A massive collection of multimodal egocen- tric daily motion in the wild (2024),https://arxiv.org/abs/2406.09905

    Pith/arXiv arXiv 2024

  22. [22]

    Computer Graphics Forum 36(2), 349–360 (2017).https://doi.org/10.1111/cgf.13131

    von Marcard, T., Rosenhahn, B., Black, M.J., Pons-Moll, G.: Sparse inertial poser: Automatic 3d human pose estimation from sparse imus. Computer Graphics Forum 36(2), 349–360 (2017).https://doi.org/10.1111/cgf.13131

    work page doi:10.1111/cgf.13131 2017

  23. [23]

    Meta Platforms, Inc.: Meta Quest 3 VR Headset.https://www.meta.com/quest/ quest-3/(2025), accessed: 2025-09-01

    2025

  24. [24]

    In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

    Mollyn, V., Arakawa, R., Goel, M., Harrison, C., Ahuja, K.: Imuposer: Full-body pose estimation using imus in phones, watches, and earbuds. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. CHI ’23, Association for Computing Machinery, New York, NY, USA (2023).https://doi. org/10.1145/3544548.3581392,https://doi.org/10.1145...

    doi:10.1145/3544548.3581392 2023

  25. [25]

    Moticon ReGo AG: Moticon OpenGo Sensor Insoles.https://moticon.com/ opengo(2025), accessed: 2025-09-01

    2025

  26. [26]

    Computer Graphics Forum41(8), 195–206 (2022).https://doi.org/10.1111/ cgf.14635

    Mourot, L., Hoyet, L., Clerc, F.L., Hellier, P.: Underpressure: Deep learning for foot contact detection, ground reaction force estimation and footskate cleanup. Computer Graphics Forum41(8), 195–206 (2022).https://doi.org/10.1111/ cgf.14635

    2022

  27. [27]

    de/products/loadsol(2025), accessed: 2025-09-01

    novel GmbH: loadsol: Wireless in-shoe plantar pressure sensors.https://novel. de/products/loadsol(2025), accessed: 2025-09-01

    2025

  28. [28]

    novel GmbH: pedar: In-shoe plantar pressure measurement system.https:// novel.de/products/pedar(2025), accessed: 2025-09-01

    2025

  29. [29]

    prnewswire

    NURVV: NURVV Run Smart Insoles.https :/ / www . prnewswire . com / news - releases/nurvv-debuts-first-ever-tech-to-accurately-measure-running- power-using-smart-insoles-301365702.html(2021), accessed: 2026-06-24

    2021

  30. [30]

    arXiv preprint arXiv:2202.06164 (2022).https://doi.org/10.48550/ arXiv.2202.06164

    Palermo, M., Cerqueira, S., André, J., Pereira, A., Santos, C.P.: Complete inertial pose dataset: from raw measurements to pose with low-cost and high-end marg sensors. arXiv preprint arXiv:2202.06164 (2022).https://doi.org/10.48550/ arXiv.2202.06164

    Pith/arXiv arXiv 2022

  31. [31]

    Peebles,W.,Xie,S.:Scalablediffusionmodelswithtransformers.In:Proceedingsof the IEEE/CVF international conference on computer vision. pp. 4195–4205 (2023) 18 A. Boscolo Camiletto et al

    2023

  32. [32]

    In: Conference on Computer Vision and Pattern Recognition (CVPR)

    Tripathi, S., Müller, L., Huang, C.H.P., Omid, T., Black, M.J., Tzionas, D.: 3D human pose estimation via intuitive physics. In: Conference on Computer Vision and Pattern Recognition (CVPR). pp. 4713–4725 (2023)

    2023

  33. [33]

    In: 2017 British Machine Vision Conference (BMVC) (2017)

    Trumble,M.,Gilbert,A.,Malleson,C.,Hilton,A.,Collomosse,J.:Totalcapture:3d human pose estimation fusing video and inertial sensors. In: 2017 British Machine Vision Conference (BMVC) (2017)

    2017

  34. [34]

    Accessed: 2026-06-29

    Vicon Motion Systems Ltd., Oxford, UK: Vicon Motion Capture System (2023), available athttps://www.vicon.com. Accessed: 2026-06-29

    2023

  35. [35]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Wang, J., Dabral, R., Luvizon, D., Cao, Z., Liu, L., Beeler, T., Theobalt, C.: Ego4o: Egocentric human motion capture and understanding from multi-modal input. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 22668–22679 (June 2025)

    2025

  36. [36]

    arXiv preprint arXiv:2505.00755 (2025)

    Watanabe, A., Aisuwarya, R., Jing, L.: P2p-insole: Human pose estimation using foot pressure distribution and motion sensors. arXiv preprint arXiv:2505.00755 (2025)

    Pith/arXiv arXiv 2025

  37. [37]

    In: European Conference on Computer Vision

    Werling, K., Kaneda, J., Tan, T., Agarwal, R., Skov, S., Van Wouwe, T., Uhlrich, S., Bianco, N., Ong, C., Falisse, A., et al.: Addbiomechanics dataset: Capturing the physics of human motion at scale. In: European Conference on Computer Vision. pp. 490–508. Springer (2024)

    2024

  38. [38]

    In: SIGGRAPH Asia 2022 Conference Papers

    Winkler, A., Won, J., Ye, Y.: Questsim: Human motion tracking from sparse sen- sors with simulated avatars. In: SIGGRAPH Asia 2022 Conference Papers. SA ’22, Association for Computing Machinery, New York, NY, USA (2022).https://doi. org/10.1145/3550469.3555411,https://doi.org/10.1145/3550469.3555411

    doi:10.1145/3550469.3555411 2022

  39. [39]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Wouwe, T.V., Lee, S., Falisse, A., Delp, S.L., Liu, C.K.: Diffusionposer: Real-time human motion reconstruction from arbitrary sparse sensors using autoregressive diffusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 2513–2523 (2024).https://doi.org/10.1109/ CVPR52733.2024.00243

    arXiv 2024

  40. [40]

    In: Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology

    Wu, E., Khirodkar, R., Koike, H., Kitani, K.: Soleposer: Full body pose estimation using a single pair of insole sensor. In: Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology. UIST ’24, Association for Computing Machinery, New York, NY, USA (2024).https://doi.org/10.1145/ 3654777.3676418,https://doi.org/10.1145/3654777.3676418

    doi:10.1145/3654777.3676418 2024

  41. [41]

    In: 2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)

    Wu, E., Peng, Y., Khirodkar, R., Koike, H., Kitani, K.: Dual-modal 3d human pose estimation using insole foot pressure sensors. In: 2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). pp. 131–135 (2024).https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00036

    doi:10.1109/ismar-adjunct64951.2024.00036 2024

  42. [42]

    In: Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology

    Xu, V., Gao, C., Hoffmann, H., Ahuja, K.: Mobileposer: Real-time full-body pose estimation and 3d human translation from imus in mobile consumer devices. In: Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology. pp. 1–11 (2024).https://doi.org/10.1145/3654777.3676461

    doi:10.1145/3654777.3676461 2024

  43. [43]

    In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR)

    Yi, B., Ye, V., Zheng, M., Li, Y., Müller, L., Pavlakos, G., Ma, Y., Malik, J., Kanazawa, A.: Estimating body and hand motion in an ego-sensed world. In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR). pp. 7072–7084 (June 2025)

    2025

  44. [44]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Yi, X., Zhou, Y., Habermann, M., Shimada, S., Golyanik, V., Theobalt, C., Xu, F.: Physical inertial poser (pip): Physics-aware real-time human motion tracking from sparse inertial sensors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 13157–13168 (2022).https://doi. org/10.1109/CVPR52688.2022.01282 Towar...

    doi:10.1109/cvpr52688.2022.01282 2022

  45. [45]

    ACM Trans

    Yi, X., Zhou, Y., Xu, F.: Transpose: real-time 3d human translation and pose esti- mation with six inertial sensors. ACM Trans. Graph.40(4) (Jul 2021).https: //doi.org/10.1145/3450626.3459786,https://doi.org/10.1145/3450626. 3459786

    doi:10.1145/3450626.3459786 2021

  46. [46]

    In: ProceedingsoftheIEEE/CVFConferenceonComputerVisionandPatternRecog- nition (CVPR)

    Zhang, H., Ren, S., Yuan, H., Zhao, J., Li, F., Sun, S., Liang, Z., Yu, T., Shen, Q., Cao, X.: Mmvp: A multimodal mocap dataset with vision and pressure sensors. In: ProceedingsoftheIEEE/CVFConferenceonComputerVisionandPatternRecog- nition (CVPR). pp. 21842–21852 (2024).https://doi.org/10.1109/CVPR52733. 2024.02063

    doi:10.1109/cvpr52733 2024

  47. [47]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Zhang, Y., Xia, S., Chu, L., Yang, J., Wu, Q., Pei, L.: Dynamic inertial poser (dynaip): Part-based motion dynamics learning for enhanced human pose estima- tion with sparse inertial sensors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1889–1899 (2024)

    2024

  48. [48]

    In: 2021 International Conference on 3D Vision (3DV)

    Zhao, D., Wei, Z., Mahmud, J., Frahm, J.M.: Egoglass: Egocentric-view human pose estimation from an eyeglass frame. In: 2021 International Conference on 3D Vision (3DV). pp. 32–41 (2021).https://doi.org/10.1109/3DV53792.2021. 00014

    doi:10.1109/3dv53792.2021 2021

  49. [49]

    IEEE Transactions on Instrumentation and Measurement (2025)

    Zhu, Z., Zhao, Y., Hu, Y., Wang, G., Qiu, H., Zheng, B., Yan, C., Xu, F.: Pro- gressive inertial poser: Progressive real-time kinematic chain estimation for 3d full-body pose from three imu sensors. IEEE Transactions on Instrumentation and Measurement (2025)

    2025

  50. [50]

    ACM Transactions on Graphics44(4), 1–14 (July 2025)

    Zuo, C., Huang, J., Jiang, X., Yao, Y., Shi, X., Cao, R., Yi, X., Xu, F., Guo, S., Qin, Y.: Transformer IMU calibrator: Dynamic on-body IMU calibration for inertial motion capture. ACM Transactions on Graphics44(4), 1–14 (July 2025). https://doi.org/10.1145/3730937

    work page doi:10.1145/3730937 2025