REVIEW 3 major objections 5 minor 21 references
Calibrated Uncertainty for Trustworthy Clinical Gait Analysis Using Probabilistic Multiview Markerless Motion Capture
T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Probabilistic video gait analysis confirms its confidence intervals against clinical references, showing calibrated uncertainty for step, stride, and joint angles.
desk verdict Spatial calibration and uncertainty filtering are solid; the kinematic calibration only holds after per-participant bias removal, so the deployment claim is half-supported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the variational posterior over joint angles, $q_{\phi}(\theta_t)=\mathcal{N}(\theta_t;\,\mu_{\phi}(t),\Sigma_{\phi}(t))$, produced by an implicit neural function that maps time to the mean and a low-rank covariance of 40 kinematic degrees of freedom. The model is fit by optimizing an evidence lower bound augmented with geometric regularizers (site-offset and joint-limit penalties) and an internal Expected Calibration Error term that treats high-confidence detected keypoints as pseudo-ground truth. External calibration is then assessed by pushing each observed error through the predicted CDF (Probability Integral Transform) and measuring the deviation of the resulting values from uniformity via ECE.
What would settle it
Run the probabilistic model on a new participant's video without performing per-participant bias subtraction and compare the joint-angle intervals against simultaneous marker-based kinematics; if the ECE exceeds 0.1 or median errors exceed the reported 1.5 to 3.8 degrees, the calibration claim depends on validation-time bias correction rather than the model alone.
Extended reading notes
Core claim
The paper claims that a variational-inference-based probabilistic MMMC model, trained only on video keypoints, produces posterior confidence intervals that are externally calibrated: ECE is 0.05 for step length and 0.04 for stride length, and for bias-corrected lower-limb joint angles ECE is below 0.1 for every joint except pelvis rotation (0.18). The magnitude of predicted uncertainty closely tracks actual error: filtering to the lowest 50% predicted uncertainty reduces median step length error from 16.2 mm to 12.0 mm, while the noisiest 10% of steps have median errors near 39 mm. The paper interprets this as evidence that the model quantifies epistemic uncertainty, enabling a workflow where the model's own confidence, not a reference instrument, gates data quality.
Load-bearing premise
The kinematic calibration and reported joint-angle errors are computed only after subtracting a constant per-participant, per-joint bias measured on the same validation trials, so the results assume that the markerless-to-marker offset is stable across time and conditions.
Editorial extensions
If this is right
- A clinician could use the model's predicted uncertainty to flag or exclude individual steps that are unreliable, without needing an instrumented walkway or marker system at the point of care.
- Filtering by uncertainty improves data quality: retaining only the most confident half of steps reduces median step length error from roughly 16 mm to 12 mm and narrows the interquartile range.
- Calibration holds across diverse populations including children, prosthetic users, and individuals with neurological gait impairments, suggesting the confidence intervals transfer beyond able-bodied adults.
- Because the posterior is computed per time point and per joint, uncertainty can be propagated into derived metrics like step length, enabling per-step error bars rather than population averages.
- The remaining miscalibration in pelvis rotation (ECE 0.18) indicates that aligning the biomechanical model's rotation convention with clinical models would likely restore calibration at that joint.
Reading between the lines
- If the per-participant, per-joint bias is stable across sessions and camera configurations, a short marker-based calibration recording per patient could make the joint-level confidence intervals valid in routine video-only use; the paper does not test that stability.
- The internal ECE regularizer, which uses high-confidence keypoints as pseudo-ground truth during training, may be the mechanism that transfers calibration to external references; an ablation study removing this term would directly test that.
- The uncertainty-based filtering principle could generalize to other clinical measurement pipelines beyond gait, such as upper-limb range-of-motion or balance assessments, wherever a probabilistic reconstruction is available.
- Combining predicted uncertainty with explicit occlusion or view-count features might further sharpen the trust filter, since the paper reports anecdotal occlusion effects but does not quantify them.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript validates a probabilistic multiview markerless motion capture (MMMC) pipeline, previously introduced by the authors, against two external references: an instrumented walkway (GaitRite) for step and stride length and marker-based motion capture for lower-extremity joint angles. Using data from 68 participants across two sites, it reports low Expected Calibration Error (ECE) values for spatial gait metrics, median step and stride errors of about 16 mm and 12 mm, and median kinematic errors of 1.5 to 3.8 degrees after per-participant bias correction. It also demonstrates that filtering steps by predicted uncertainty reduces median errors, and it argues that calibrated uncertainty allows users to identify unreliable outputs without concurrent ground-truth instrumentation.
Significance. The spatial validation is a genuinely independent external benchmark, and the finding that predicted uncertainty correlates with observed error in both spatial and kinematic measures is practically important for clinical gait analysis. The study addresses a real gap by testing calibration rather than only aggregate accuracy, on a diverse clinical population. However, the central kinematic calibration claim is currently undercut by the per-participant bias-correction procedure, which uses the same trials for fitting and evaluation; as written, the paper does not establish that raw markerless reconstructions provide calibrated joint-level confidence intervals in a deployment setting. The spatial filtering result is credible, but the kinematic claim needs additional analysis or substantial qualification.
major comments (3)
- [Section II-E3, Table V] The kinematic ECE and error magnitudes are computed after subtracting per-participant joint biases estimated from the same trials used for validation. This makes the reported ECE and median errors properties of residual fluctuations around a participant-specific offset, not of the raw probabilistic reconstructions. In deployment the marker-based reference is absent, so the bias is unknown; with the large between-participant standard deviations in Table IV (e.g., hip flexion 23.25 +/- 6.36 degrees, pelvis tilt 19.71 +/- 4.91 degrees), even population-mean correction would leave several degrees of systematic error and would break nominal coverage. Please report ECE and errors without any bias correction, or with an out-of-sample (e.g., leave-one-participant-out) bias estimation, so that the reader can see whether calibrated intervals survive the deployment condition.
- [Section V (Conclusion) and Abstract] The conclusion that the method provides 'calibrated confidence intervals at the level of each joint and moment of time for a participant' is not supported by the analysis as presented, because the calibration was evaluated on bias-corrected residuals. The evidence supports calibrated spatial intervals and a monotone uncertainty-error relationship for kinematics, but not calibrated joint-level intervals for a video-only pipeline. Please either qualify these deployment claims or provide the additional analysis requested above.
- [Section II-E1 (Kinematic ECE via HalfNormal CDF)] The PIT/HalfNormal calibration test implicitly assumes that the errors are zero-mean with the model's predicted scale. After per-participant bias subtraction the errors are zero-mean by construction, so the test cannot detect systematic offset miscalibration. If raw errors are used, this test would be a valid check of full calibration; please also report the coverage of the nominal confidence intervals before bias removal, or explicitly state that the kinematic calibration claim is limited to residual uncertainty after a bias-correction step that requires external data.
minor comments (5)
- [Section IV (Discussion)] The sentence stating that kinematic patterns 'other than pelvic obliquity' are well calibrated appears to be a mistake: Table V and Figure 5 show pelvis rotation (ECE 0.18) as the poorly calibrated joint, while pelvis obliquity has ECE 0.06. Please correct this.
- [Section II-E1] There is a typo: 'Probabilty Integral Transform' should read 'Probability Integral Transform.'
- [Abstract and Introduction] The abbreviation 'MMC' is used in a few places (e.g., 'As MMC moves from research to clinical deployment') where the text should read 'MMMC' for consistency.
- [Tables II and III] The captions contain formatting inconsistencies such as 'AT AL' and 'All Participants' versus 'AL' in column headers; please proofread the table captions and headers.
- [Throughout] Please standardize spelling of 'GaitRite' (the reference list uses 'Gaitrite') and the use of 'pelvis obliquity' versus 'pelvic obliquity'; both variants currently appear.
Circularity Check
Kinematic calibration is measured on residuals after subtracting per-participant biases fitted to the same validation trials, so the joint-level 'bias-corrected' ECE is partly forced by construction.
-
fitted input called prediction
[Section II-E3 (Joint Angle Errors and Validation); results in Tables IV and V; acknowledged in Section IV-C Limitations]
"Before performing our analysis, we removed these systematic biases from the probabilistic model data to ensure a valid comparison [3]. The systematic bias for each joint angle was calculated as the mean difference between the marker-based and probabilistic prediction, averaged over each trial. The final bias values, averaged per participant, were then subtracted from the probabilistic model’s predictions to establish a corrected reference for comparative validation against the marker-based data."
The per-participant bias subtracted before the kinematic error and ECE computations is fitted to the very same marker-based trials used as the validation reference. After subtraction, the reported median errors (1.5–3.8°) and ECE values (<0.1) describe only the zero-mean residual fluctuation around a participant-specific offset; the large systematic differences (e.g., hip flexion 23.25°±6.36°, pelvis tilt 19.71°±4.91°) are removed by construction. In deployment no marker-based system exists to estimate this bias, so the abstract's 'bias-corrected gait kinematics' calibration and the conclusion that the method provides 'calibrated confidence intervals at the level of each joint' without concurrent ground-truth instrumentation are not predictions from video alone.
full rationale
The spatial validation against the GaitRite instrumented walkway is an independent external benchmark: step and stride length ECE values are computed from the model's raw posterior samples without any per-participant adjustment against the walkway data. That portion of the paper is self-contained and not circular. The heavy reliance on the authors' prior work [6] is also not circular by itself, because the present paper's central evidence is the external comparison to GaitRite and marker-based Vicon data rather than a self-referential uniqueness argument. The significant circular step is confined to the kinematic calibration. The per-participant bias between marker-based and probabilistic joint angles is estimated from the same validation trials on which errors and ECE are subsequently reported, then subtracted before computing Table V. This makes the reported kinematic errors and calibration properties those of bias-corrected residuals, not of the raw video-only probabilistic output that would be available at deployment. The paper is honest in labeling the results as 'bias-corrected' and in listing the limitation, but the abstract and conclusion nonetheless present these bias-corrected kinematic ECE values as evidence that a user can identify unreliable outputs without concurrent ground-truth instrumentation. Because one of the two central validation claims partly reduces by construction to a fitted per-participant input, while the spatial GaitRite claim remains independently supported, a partial circularity score of 6 is appropriate rather than a higher score.
Assumptions & free parameters
free parameters (3)
- lambda_ece (calibration regularization weight) =
0.5
- Per-participant joint-angle bias offsets (8 joints) =
e.g., pelvis tilt 19.71 degrees, hip flexion 23.25 degrees (Table IV)
- Regularization weights lambda_theta, lambda_site, lambda_excess in Eq. 1 =
not reported in this paper
assumptions (5)
- domain assumption The variational posterior q_phi(theta_t) is a Gaussian with low-rank covariance, sufficient to capture the true posterior.
- domain assumption Ground truth references (GaitRite walkway and marker-based Vicon/SCGM) are accurate enough to serve as external validation.
- ad hoc to paper After per-participant bias removal, residual kinematic errors are zero-mean, so the PIT/HalfNormal calibration test is meaningful.
- domain assumption The internal ECE regularizer in Eq. 1, computed against high-confidence keypoints as pseudo-ground truth, improves external calibration.
- domain assumption GaitRite step and stride length definitions can be matched by tracking the calcaneus marker at the median stance time.
Cite this review
Pith. "Pith review of Calibrated Uncertainty for Trustworthy Clinical Gait Analysis Using Probabilistic Multiview Markerless Motion Capture." pith.science (2026). https://pith.science/paper/XJ52KO2N
@misc{pith2026260122412,
author = {Pith},
title = {Pith review of: Calibrated Uncertainty for Trustworthy Clinical Gait Analysis Using Probabilistic Multiview Markerless Motion Capture},
year = {2026},
howpublished = {\url{https://pith.science/paper/XJ52KO2N}},
note = {Machine review of arXiv:2601.22412}
}
read the original abstract
Video-based human movement analysis holds potential for movement assessment in clinical practice and research. However, the clinical implementation and trust of multi-view markerless motion capture (MMMC) require that, in addition to being accurate, these systems produce reliable confidence intervals to indicate how accurate they are for any individual. Building on our prior work utilizing variational inference to estimate joint angle posterior distributions, this study evaluates the calibration and reliability of a probabilistic MMMC method. We analyzed data from 68 participants across two institutions, validating the model against an instrumented walkway and standard marker-based motion capture. We measured the calibration of the confidence intervals using the Expected Calibration Error (ECE). The model demonstrated reliable calibration, yielding ECE values generally < 0.1 for both step and stride length and bias-corrected gait kinematics. We observed a median step and stride length error of ~16 mm and ~12 mm respectively, with median bias-corrected kinematic errors ranging from 1.5 to 3.8 degrees across lower extremity joints. Consistent with the calibrated ECE, the magnitude of the model's predicted uncertainty correlated strongly with observed error measures. These findings indicate that, as designed, the probabilistic model reconstruction quantifies epistemic uncertainty, allowing it to identify unreliable outputs without the need for concurrent ground-truth instrumentation.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[6]
Biomechanical Reconstruction with Confidence Intervals from Multiview Markerless Motion Capture
R. J. Cotton and F. Sinz, “Biomechanical reconstruction with confidence intervals from multiview markerless motion capture,”arXiv preprint arXiv:2502.06486, 2 2025
work page Pith review arXiv 2025
-
[1]
Opti- mizing mpjpe promotes miscalibration in multi-hypothesis human pose lifting,
P. A. Pierzchlewicz, M. Bashiri, R. J. Cotton, and F. H. Sinz, “Opti- mizing mpjpe promotes miscalibration in multi-hypothesis human pose lifting,” inICLR 2023 Affinity Workshop, 4 2023
work page 2023
-
[2]
Concurrent assessment of gait kinematics using marker-based and markerless motion capture,
R. M. Kanko, E. K. Laende, E. M. Davis, W. S. Selbie, and K. J. Deluzio, “Concurrent assessment of gait kinematics using marker-based and markerless motion capture,”Journal of Biomechanics, vol. 127, p. 110665, Oct. 2021
work page 2021
-
[3]
C. Antognini, A. Ortigas-Vásquez, C. Knowlton, M. Utz, A. Sauer, and M. A. Wimmer, “Comparison of markerless and marker-based motion analysis accounting for differences in local reference frame orientation,” Journal of Biomechanics, vol. 185, p. 112683, 2025
work page 2025
-
[4]
S. D’Souza, T. Siebert, and V . Fohanno, “A comparison of lower body gait kinematics and kinetics between theia3d markerless and marker-based models in healthy subjects and clinical patients,” Scientific Reports, vol. 14, no. 1, p. 29154, 2024. [Online]. Available: https://doi.org/10.1038/s41598-024-80499-8
-
[5]
Opencap: Human movement dynamics from smartphone videos,
S. D. Uhlrich, A. Falisse, L. Kidzi ´nski, J. Muccini, M. Ko, A. S. Chaudhari, J. L. Hicks, and S. L. Delp, “Opencap: Human movement dynamics from smartphone videos,”PLOS Computational Biology, vol. 19, no. 10, pp. 1–26, 10 2023. [Online]. Available: https://doi.org/10.1371/journal.pcbi.1011462
-
[7]
Mujoco: A physics engine for model-based control,
E. Todorov, T. Erez, and Y . Tassa, “Mujoco: A physics engine for model-based control,” in2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, pp. 5026–5033
2012
-
[8]
Myosuite – a contact-rich simulation suite for musculoskeletal motor control,
V . Caggiano, H. Wang, G. Durandau, M. Sartori, and V . Kumar, “Myosuite – a contact-rich simulation suite for musculoskeletal motor control,” 2022
work page 2022
Show all 21 references
-
[9]
Differentiable biomechanics unlocks opportunities for markerless motion capture,
R. J. Cotton, “Differentiable biomechanics unlocks opportunities for markerless motion capture,” in2025 International Conference On Re- habilitation Robotics (ICORR), 2025, pp. 44–51
2025
-
[10]
Differentiable biomechanics for markerless motion capture in upper limb stroke rehabilitation: A comparison with optical motion capture,
T. Unger, A. Sal Moslehian, J. D. Peiffer, J. Ullrich, R. Gassert, O. Lam- bercy, R. J. Cotton, and C. Awai Easthope, “Differentiable biomechanics for markerless motion capture in upper limb stroke rehabilitation: A comparison with optical motion capture,” 2024
2024
-
[11]
Probabilistic forecasts, calibration and sharpness,
T. Gneiting, F. Balabdaoui, and A. E. Raftery, “Probabilistic forecasts, calibration and sharpness,”Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 69, no. 2, pp. 243–268, 2007
2007
-
[12]
The shriners children’s gait model (scgm),
K. M. Kruger, P. Fischer, S. Augsburger, J. Feng, J. F. Girouard, D. L. Gregory, L. Johnson, B. A. MacWilliams, M. L. McMulkin, B. Nelson, S. Warshauer, P. Saraswat, and R. S. Chafetz, “The shriners children’s gait model (scgm),”Gait & Posture, vol. 110, pp. 84–109, 5 2024
2024
-
[13]
Sensing and Rehabilitation
I. Sensing and Rehabilitation. Multicameratracking. [On- line]. Available: https://github.com/IntelligentSensingAndRehabilitation/ MultiCameraTracking
-
[14]
Learning 3d human pose estimation from dozens of datasets using a geometry-aware autoencoder to bridge between skeleton formats,
I. Sárándi, A. Hermans, and B. Leibe, “Learning 3d human pose estimation from dozens of datasets using a geometry-aware autoencoder to bridge between skeleton formats,” inIEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023
2023
-
[15]
Movi: A large multi-purpose human motion and video dataset,
S. Ghorbani, K. Mahdaviani, A. Thaler, K. Kording, D. J. Cook, G. Blohm, and N. F. Troje, “Movi: A large multi-purpose human motion and video dataset,”PLOS ONE, vol. 16, no. 6, p. e0253157, 6 2021
2021
-
[16]
Optimizing trajectories and inverse kinematics for biomechanical analysis of markerless motion capture data,
R. J. Cotton, A. DeLillo, A. Cimorelli, K. Shah, J. D. Peiffer, S. Anar- wala, K. Abdou, and T. Karakostas, “Optimizing trajectories and inverse kinematics for biomechanical analysis of markerless motion capture data,” inIEEE International Consortium for Rehabilitation Robotic...
2023
-
[17]
Improved trajectory reconstruction for markerless pose estimation,
R. J. Cotton, A. Cimorelli, K. Shah, S. Anarwala, S. Uhlrich, and T. Karakostas, “Improved trajectory reconstruction for markerless pose estimation,” in45th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 3 2023
2023
-
[18]
Fast and robust multi-person 3d pose estimation and tracking from multiple views,
J. Dong, Q. Fang, W. Jiang, Y . Yang, H. Bao, and X. Zhou, “Fast and robust multi-person 3d pose estimation and tracking from multiple views,” inT-PAMI, 2021
2021
-
[19]
Multi- hypothesis 3D human pose estimation metrics favor miscalibrated dis- tributions,
P. A. Pierzchlewicz, R. J. Cotton, M. Bashiri, and F. H. Sinz, “Multi- hypothesis 3D human pose estimation metrics favor miscalibrated dis- tributions,” Oct. 2022
2022
-
[20]
Concurrent related validity of the gaitrite® walkway system for quantification of the spatial and temporal parameters of gait,
B. Bilney, M. Morris, and K. Webster, “Concurrent related validity of the gaitrite® walkway system for quantification of the spatial and temporal parameters of gait,”Gait & Posture, vol. 17, pp. 68–74, 1 2003
2003
-
[21]
The validity and reliability of the gaitrite system’s measurements: A pre- liminary evaluation,
A. L. McDonough, M. Batavia, F. C. Chen, S. Kwon, and J. Ziai, “The validity and reliability of the gaitrite system’s measurements: A pre- liminary evaluation,”Archives of Physical Medicine and Rehabilitation, vol. 82, pp. 419–425, 3 2001
2001
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.