Pith. sign in

REVIEW 4 major objections 6 minor 25 references

Quantitative Gait Analysis from Single RGB Videos Using a Dual-Input Transformer-Based Network

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A dual-image Transformer can estimate gait parameters from ordinary video, reducing GDI error by 10.6%.

desk verdict A small dual-image CNN that may improve GDI/knee-flexion MAE on the Kidzinski dataset, but the baseline comparison is uncontrolled enough that the headline gains aren't established. read the letter →

arxiv 2501.01689 v1 pith:CIUTYBHY submitted 2025-01-03 cs.CV

Hiep Dinh , Son Le , My Than , Minh Ho , Nicolas Vuillerme , Hieu Pham This is my paper · ORCID
classification cs.CV
keywords gaitanalysisDeviationIndex(GDI)kneeflexioncadencesingleRGBvideoposeestimationdual-inputTransformercerebralpalsy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Quantitative gait analysis normally requires a motion-capture laboratory, which is expensive and hard to staff. This paper claims that a compact dual-input convolutional Transformer network can estimate clinically useful gait parameters from an ordinary single-camera RGB video, by turning a 124-frame skeleton sequence into two grayscale images. On a public dataset of 2,212 videos from 1,138 children with cerebral palsy, the model reports mean absolute errors of 5.6450 for the Gait Deviation Index, 5.1203 degrees for knee flexion, and 0.1418 for cadence. Against the two published comparison models, this is a 10.6% lower GDI error and a 12.0% lower knee-flexion error, while cadence remains worse. If these numbers hold under identical evaluation conditions, the method offers a smaller, cheaper route to markerless gait assessment.

What carries the argument

The load-bearing object is the dual-pattern input: two grayscale images constructed from the same 124-frame skeleton sequence. The first image is a 128x64 matrix whose rows are the normalized (x,y) coordinates of 25 body landmarks, padded and duplicated to 128x128; the second is a 128x128 plot of the hip, knee, and ankle trajectories across all frames. Each image is processed by three convolutional blocks with max pooling, the resulting 32,768-dimensional feature maps are concatenated to a 65,536-unit vector, and four fully connected layers regress to a single scalar. In the paper's framing, the self-attention mechanism lets the model focus on relevant anatomical keypoints and their relationships over time, replacing the hand-crafted features and separate pose-estimation stages used by earlier pipelines.

What would settle it

A reader could re-train the 1D-CNN and STT baselines on the same 8:1:1 patient-exclusive split and dual-image preprocessing, and also shuffle frame order within the 124-frame clips before building the input images. If the reproduced baseline MAEs do not match the values of 6.5469 and 6.3137 (GDI) or 5.9129 and 5.8220 (knee flexion), the reported reductions do not hold; if shuffled inputs change predictions only slightly, the model is not actually using temporal gait dynamics.

Watch

Extended reading notes

Core claim

The paper's central finding is that encoding the temporal sequence of 2D body keypoints as two static 128x128 grayscale images is enough for a single CNN-based Transformer model to regress gait parameters from single-view RGB video. The first image arranges all 25 body landmarks across the 124 frames as rows, and the second plots only hip, knee, and ankle coordinates over the whole clip; the same network consumes both, concatenates their features, and outputs one gait metric. Measured against ground-truth laboratory values, the DPG model achieves mean absolute errors of 5.6450 (GDI), 5.1203 (knee flexion), and 0.1418 (cadence) on the test set. The authors interpret the lower GDI and knee-flexion errors as evidence that a single dual-pattern model can surpass the 1D-CNN and STT approaches for those two metrics while using fewer resources, and they attribute the cadence shortfall to the absence of explicit gait-cycle and stride detection.

Load-bearing premise

The claimed accuracy gains assume the two comparison models were evaluated on exactly the same train/test split and preprocessing; if their published numbers came from different settings, the reported improvements are not meaningful.

Editorial extensions

If this is right

  • Clinics with only a standard camera could obtain GDI and knee-flexion estimates within roughly 5-6 units or degrees of laboratory values, bypassing marker-based motion capture.
  • A single network replaces the multi-network pipeline of the earlier approach, reducing training and inference cost, which matters for telehealth and low-resource settings.
  • Cadence prediction at 0.1418 MAE is weaker than both baselines, so the method as presented is not yet a complete gait-assessment tool; adding stride and gait-cycle detection is the natural next step.
  • The dual-image encoding could be adapted to other clinical parameters derived from body landmarks, such as step length or trunk kinematics, with the same small input representation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper never checks whether discarding frame order matters: an ablation that shuffles the 124 frames before building the two images would show how much of the accuracy comes from temporal gait dynamics rather than from a static posture summary.
  • If the baseline MAE values were copied from the original publications rather than recomputed on the same patient-exclusive split and preprocessing, the 10.6% and 12.0% reductions could be an artifact of different evaluation settings; reproducing the baselines is the decisive test.
  • The same two-image encoding might generalize to other clinical video tasks, but robustness to occlusions, camera angles, and atypical gait patterns remains untested, so real-clinic performance could differ from lab-test error.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript proposes DPG, a dual-input convolutional network for estimating gait deviation index (GDI), knee flexion angle, and cadence from single-view RGB video. OpenPose keypoints from 124-frame clips are encoded as two grayscale images: one stacks normalized landmark coordinate rows over time and the other plots hip, knee, and ankle landmarks from all frames. A three-layer CNN processes each image, the resulting features are concatenated and regressed through four fully connected layers. Training and evaluation use the Gillette Children's dataset with an 8:1:1 patient-disjoint split. Table 1 reports MAE values of 5.6450, 5.1203, and 0.1418 for GDI, knee flexion, and cadence, respectively, and the text claims 10.6% and 12.0% improvements over 1D-CNN and STT for the first two metrics while cadence is worse. The authors state that code and trained models are released, but no repository link is given.

Significance. If the reported comparison were controlled, the result would be a modest but practically useful demonstration that a simple image-encoding network can estimate clinical gait metrics from RGB video with few parameters. The main strengths are the use of a real clinical dataset with patient-disjoint splitting and a clearly described preprocessing pipeline. However, the paper's central novelty claim, a 'dual-input Transformer' with self-attention, is not reflected in the architecture, which is described as a CNN with no attention mechanism. The significance therefore hinges on whether the empirical superiority claim survives a controlled baseline comparison; in its current form the evidence is not sufficient.

major comments (4)
  1. [Section 3.2 / Table 1] The headline claim of 10.6% reduction on GDI and 12.0% on knee flexion is not supported unless the 1D-CNN and STT baseline MAEs were obtained under exactly the same protocol. The manuscript reports the DPG split (8:1:1, patient-disjoint; 1,768/920 training, 212/106 validation, 232/112 test) and preprocessing, but never states whether the baseline numbers in Table 1 were reproduced with that split, the same 124-frame windowing with 31-frame overlap, the same OpenPose keypoint representation, and the same model-selection rule, or whether they were copied from [11] and [17]. Because the original dataset splits and preprocessing in those papers may differ, the percentage improvements may be artifacts of evaluation mismatch. Please rerun baselines under the identical pipeline or state precisely the source of each baseline number and justify comparability, and report variability over repeated runs.
  2. [Section 2.2 / Figure 2] The architecture described in Section 2.2 and Figure 2 consists of three convolutional layers followed by four fully connected layers. There is no self-attention, multi-head attention, positional encoding, or transformer block anywhere in the model description. The Abstract, Introduction, and title repeatedly call the model a 'dual-input Transformer', and the Introduction states that 'the self-attention mechanism enables the model to focus on relevant anatomical keypoints', which is contradicted by the presented method. Either the model must be changed to include an attention component, or the claims and name should be revised to describe a dual-input CNN; as written, the central modeling contribution is not accurately characterized.
  3. [Section 3.1 / preprocessing] The dual-input representation encodes the 124-frame sequence as two static images: one row-per-frame normalized keypoint image and one scatter of hip-knee-ankle coordinates over all frames. This choice discards the explicit temporal ordering in the second input and only implicitly preserves it in the first. No ablation compares this encoding to a sequence-based input or to using a single image, and no analysis shows that the two inputs contribute complementary information. Given that the claimed advantage over 1D-CNN is attributed to temporal modeling, the absence of such an ablation leaves the core design choice unvalidated.
  4. [Section 3.2 / Table 1] All results are single-point MAE values with no standard deviation, number of repeated runs, or statistical significance testing. Differences between methods, for example 6.3137 versus 5.6450 for GDI and 5.9129 versus 5.1203 for knee flexion, may be meaningful but cannot be assessed without variability estimates or paired tests across patients. The paper also does not specify the units or clinical interpretation of the cadence error, which makes the reported failure on cadence difficult to evaluate.
minor comments (6)
  1. [Title page / Header] There are typographical errors in the title page and author list, including 'Quantita tive' and 'Vuilerrme'; the phrase 'convolution Transformer' should likely be 'convolutional Transformer'.
  2. [Abstract / Section 3.2] The Abstract and contribution list mention step length as an estimated gait parameter, but Table 1 and Section 3.2 report results only for GDI, knee flexion, and cadence; the step length result is missing.
  3. [Figure 1 caption] Figure 1 is reused from Kidzinski et al. [11] for illustration purposes, but the manuscript does not include an explicit copyright or permission statement for the reused figure.
  4. [Section 3.2] The phrase 'Through thought experiments and comparisons' is unclear; it should be replaced with a more precise description such as 'Through qualitative analysis and experimental comparisons'.
  5. [References / Section 3.2] Reference [17] is a prior paper by Le and Pham, which appears to be from the same group as the current manuscript; if so, this relationship should be disclosed when STT is used as a baseline.
  6. [Section 1 / Conclusion] The statement that 'we released our codes and trained models' is not accompanied by a repository link or availability statement, so the claim cannot be verified by readers.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the only self-referential element is the STT baseline from prior work by a co-author, and it is not load-bearing because an external baseline supports the comparison.

full rationale

This is an empirical machine-learning paper that trains a regression model on keypoints extracted from RGB video, so the classical circular-derivation patterns do not apply: there is no fitted parameter renamed as a prediction, no uniqueness theorem imported from the authors' prior work, and no ansatz smuggled in via citation. The only self-referential element is the STT baseline [17], authored by H. Le and H. Pham, with Hieu Pham being a co-author of the present work. STT is used as a comparison in Table 1 and appears to be the reference for the stated 10.6% and 12.0% improvements. This self-citation is not load-bearing because the paper also compares against the external 1D-CNN baseline [11], and the proposed DPG model beats that external baseline on the same two metrics (GDI 5.6450 vs. 6.5469; knee flexion 5.1203 vs. 5.9129). The paper does not disclose whether the baseline numbers were reproduced under the current patient-disjoint 8:1:1 split, which is a legitimate reproducibility and comparability risk, but that is a correctness concern rather than circularity. The knee-flexion prediction is strongly related to the hip/knee/ankle landmarks in the second input image, but the ground-truth labels are derived from 3D motion capture while the input contains 2D projected keypoints, so the mapping is not exactly by construction; this is a novelty and baseline-omission concern, not a circular reduction. Overall, the central empirical claim has independent content through the external baseline, and no prediction reduces to its inputs by definition.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central empirical claim rests on the reliability of OpenPose keypoints, the correctness of the Gillette ground-truth labels, the preservation of temporal information in static images, and comparability of baseline numbers; none of these is validated in the paper.

free parameters (5)
  • Learning rate = 0.001
    Optimization hyperparameter fixed without a reported sensitivity study (Section 3.1).
  • Batch size = 32
    Training batch size chosen in Section 3.1; no ablation.
  • Epoch budget = 30 epochs with early stopping
    Training schedule; affects convergence and final model (Section 3.1).
  • Image encoding parameters = 128x64 rows duplicated to 128x128; 0-255 grayscale
    Manual preprocessing choices in Section 3.1 that define the input; no ablation shows they are optimal or that temporal structure survives.
  • Architecture widths = Conv 32/64/128; FC 512/256/128/64
    Hand-selected network capacity in Section 2.2; no ablation for these dimensions.
assumptions (5)
  • domain assumption OpenPose keypoint trajectories extracted from sagittal-plane video are accurate enough for clinical gait parameter estimation.
    The entire input pipeline depends on the quality of OpenPose detections; no validation of keypoint errors is provided (Section 3.1).
  • domain assumption The Gillette dataset's ground-truth gait parameters (GDI, knee flexion, cadence, step length) from motion capture are reliable and remain valid after preprocessing.
    The supervised labels are taken as ground truth (Section 3.1).
  • domain assumption Encoding 124 frames of 2D keypoints as two static 128x128 grayscale images preserves the temporal dynamics needed for gait regression.
    The method reduces time to space by plotting landmarks over frames; no ablation confirms temporal information is retained (Sections 2.2 and 3.1).
  • domain assumption The reported 1D-CNN and STT baseline numbers were obtained under comparable train/validation/test splits and evaluation protocols.
    The paper gives no implementation details for baselines; the comparison in Table 1 assumes comparability (Section 3.2).
  • domain assumption The camera view is sagittal and consistent across the dataset.
    The method is introduced for sagittal-plane video in Figure 1 and Section 3.1, but no check of camera angle consistency is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantitative Gait Analysis from Single RGB Videos Using a Dual-Input Transformer-Based Network." pith.science (2026). https://pith.science/paper/CIUTYBHY

@misc{pith2026250101689,
  author       = {Pith},
  title        = {Pith review of: Quantitative Gait Analysis from Single RGB Videos Using a Dual-Input Transformer-Based Network},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CIUTYBHY}},
  note         = {Machine review of arXiv:2501.01689}
}
read the original abstract

Gait and movement analysis have become a well-established clinical tool for diagnosing health conditions, monitoring disease progression for a wide spectrum of diseases, and to implement and assess treatment, surgery and or rehabilitation interventions. However, quantitative motion assessment remains limited to costly motion capture systems and specialized personnel, restricting its accessibility and broader application. Recent advancements in deep neural networks have enabled quantitative movement analysis using single-camera videos, offering an accessible alternative to conventional motion capture systems. In this paper, we present an efficient approach for clinical gait analysis through a dual-pattern input convolutional Transformer network. The proposed system leverages a dual-input Transformer model to estimate essential gait parameters from single RGB videos captured by a single-view camera. The system demonstrates high accuracy in estimating critical metrics such as the gait deviation index (GDI), knee flexion angle, step length, and walking cadence, validated on a dataset of individuals with movement disorders. Notably, our approach surpasses state-of-the-art methods in various scenarios, using fewer resources and proving highly suitable for clinical application, particularly in resource-constrained environments.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 23 canonical work pages

  1. [11]

    Biometric recognition by gait: A survey of modalities and features,

    P. Connor and A. Ross, “Biometric recognition by gait: A survey of modalities and features,” Computer Vision and Image Understanding, vol. 167, p. 1–27, Feb. 2018

    work page 2018

  2. [17]

    Deep neural networks enable quantitative movement analysis using single- camera videos,

    L. Kidzi ´nski, B. Yang, J. L. Hicks, A. Rajagopal, S. L. Delp, and M. H. Schwartz, “Deep neural networks enable quantitative movement analysis using single- camera videos,” Nature communications, vol. 11, no. 1, p. 4054, 2020

    work page 2020

  3. [1]

    INTRODUCTION Gait analysis, considered a fundamental biomechanics assess- ment method, first developed at the end of the 18th century [1]. It holds significant value in clinical settings worldwide, playing a crucial role in rehabilitation protocol planning, fall detection, injury prevention, sports science, and even the early detection of neurological dis...

    work page Pith review arXiv 2025

  4. [2]

    Problem Formulation The task is described as receiving an input sequence of mo- tion data X ∈ RT ×N ×2, where X = [x1, x2,

    METHODOLOGY 2.1. Problem Formulation The task is described as receiving an input sequence of mo- tion data X ∈ RT ×N ×2, where X = [x1, x2, . . . , xT ], and T represents the total number of frames in a video clip. Each frame xi captures a human posture at time step i, involving N joints, with each joint j(n) i represented by a 2D coordinate (x, y) in the...

    work page

  5. [3]

    EXPERIMENTS 3.1. Datasets and Experiment Settings To validate the proposed approach, we utilized a publicly accessible dataset from Gillette Children’s Specialty Health- care, which was gathered from 1994 to 2015. As described in [11], this dataset comprises 2,212 video recordings featur- ing 1,138 individuals diagnosed with cerebral palsy. These patients...

    work page 1994

  6. [4]

    Our method employs a dual-input Transformer ar- chitecture to estimate essential gait parameters from single- view RGB video

    DISCUSSION AND CONCLUSION In this paper, we introduced a novel approach for clinical gait analysis using a dual-pattern input convolutional transformer network. Our method employs a dual-input Transformer ar- chitecture to estimate essential gait parameters from single- view RGB video. Experimental results showed high accuracy in estimating critical metri...

    work page

  7. [5]

    This work also forms part of a broader translational and interdisciplinary GaitAlps research program

    ACKNOWLEDGMENT The research was supported by the VinUni-Illinois Smart Health Center and the French National Research Agency (ANR) in the framework of the Investissements d’avenir pro- gram (ANR-10-AIRT-05 and ANR-15-IDEX-02), and the MIAI @ Grenoble Alpes (ANR-19-P3IA-0003). This work also forms part of a broader translational and interdisciplinary GaitA...

    work page

  8. [6]

    As the data was already anonymized and accessible under an open license, ethical approval was deemed unnecessary

    COMPLIANCE WITH ETHICAL STANDARDS This study utilized publicly available human subject data from open access sources, as referenced in [11]. As the data was already anonymized and accessible under an open license, ethical approval was deemed unnecessary

    work page

Show all 25 references
  1. [7]

    The history of gait analysis before the advent of modern computers,

    R. Baker, “The history of gait analysis before the advent of modern computers,” Gait and Posture, vol. 26, no. 3, p. 331–342, Sep. 2007

    2007

  2. [8]

    Gait analysis in neurorehabilitation: From research to clinical practice,

    M. Bonanno, A. M. De Nunzio, A. Quartarone, A. Mil- iti, F. Petralito, and R. S. Calabr `o, “Gait analysis in neurorehabilitation: From research to clinical practice,” Bioengineering, vol. 10, no. 7, p. 785, Jun. 2023

    2023

  3. [9]

    Tools for observational gait analysis in pa- tients with stroke: A systematic review,

    F. Ferrarello, V . A. M. Bianchi, M. Baccini, G. Rub- bieri, E. Mossello, M. C. Cavallini, N. Marchionni, and M. Di Bari, “Tools for observational gait analysis in pa- tients with stroke: A systematic review,” Physical Ther- apy, vol. 93, no. 12, p. 1673–1685, Dec. 2013

    2013

  4. [10]

    Acceleration gait measures as proxies for motor skill of walking: A narrative review,

    P. Dasgupta, J. VanSwearingen, A. Godfrey, M. Red- fern, M. Montero-Odasso, and E. Sejdic, “Acceleration gait measures as proxies for motor skill of walking: A narrative review,”IEEE Transactions on Neural Systems and Rehabilitation Engineering , vol. 29, p. 249–261, 2021

    2021

  5. [12]

    Introduction to observational gait analysis,

    H. W. Wallmann, “Introduction to observational gait analysis,” Home Health Care Management Practice , vol. 22, no. 1, p. 66–68, Aug. 2009

    2009

  6. [13]

    A review of observa- tional gait assessment in clinical practice,

    B. Toro, C. Nester, and P. Farren, “A review of observa- tional gait assessment in clinical practice,” Physiother- apy Theory and Practice, vol. 19, no. 3, p. 137–149, Jan. 2003

    2003

  7. [14]

    The influence of tester experience on the reliability of 3d kinematic in- formation during running,

    J. Sinclair, J. Hebron, and P. Taylor, “The influence of tester experience on the reliability of 3d kinematic in- formation during running,” Gait and Posture , vol. 40, no. 4, p. 707–711, Sep. 2014

    2014

  8. [15]

    Real- time multi-person 2d pose estimation using part affinity fields,

    Z. Cao, T. Simon, S.-E. Wei, and Y . Sheikh, “Real- time multi-person 2d pose estimation using part affinity fields,” in Proceedings of the IEEE Conference on Com- puter Vision and Pattern recognition , 2017, pp. 7291– 7299

    2017

  9. [16]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems, 2017

    2017

  10. [18]

    Gait analysis methodology,

    A. Cappozzo, “Gait analysis methodology,” Human Movement Science, vol. 3, no. 1–2, p. 27–50, Mar. 1984

    1984

  11. [19]

    Markerless motion capture estimates of lower extremity kinematics and kinetics are compa- rable to marker-based across 8 movements,

    K. Song, T. J. Hullfish, R. S. Silva, K. G. Silbernagel, and J. R. Baxter, “Markerless motion capture estimates of lower extremity kinematics and kinetics are compa- rable to marker-based across 8 movements,” Feb. 2023

    2023

  12. [20]

    Two- dimensional video-based analysis of human gait using pose estimation,

    J. Stenum, C. Rossi, and R. T. Roemmich, “Two- dimensional video-based analysis of human gait using pose estimation,”PLOS Computational Biology, vol. 17, no. 4, p. e1008935, Apr 2021

    2021

  13. [21]

    Spatial temporal graph convolutional networks for skeleton-based action recog- nition,

    S. Yan, Y . Xiong, and D. Lin, “Spatial temporal graph convolutional networks for skeleton-based action recog- nition,” CoRR, vol. abs/1801.07455, 2018

    2018 arXiv

  14. [22]

    Spatial temporal transformer network for skeleton-based action recognition,

    C. Plizzari, M. Cannici, and M. Matteucci, “Spatial temporal transformer network for skeleton-based action recognition,” CoRR, vol. abs/2012.06399, 2020

    2012 arXiv

  15. [23]

    Learning to estimate criti- cal gait parameters from single-view rgb videos with transformer-based attention network,

    H. Le and H. Pham, “Learning to estimate criti- cal gait parameters from single-view rgb videos with transformer-based attention network,” in 2024 IEEE In- ternational Symposium on Biomedical Imaging (ISBI) . IEEE, 2024, pp. 1–5

    2024

  16. [24]

    Deeplabcut: markerless pose estimation of user-defined body parts with deep learning,

    A. Mathis, P. Mamidanna, K. M. Cury, T. Abe, V . N. Murthy, M. W. Mathis, and M. Bethge, “Deeplabcut: markerless pose estimation of user-defined body parts with deep learning,”Nature Neuroscience, vol. 21, no. 9, pp. 1281–1289, 2018

    2018

  17. [25]

    Two-step deep-learning identification of heel keypoints from video-recorded gait,

    K. Halvorsen, W. Peng, F. Olsson, and A. C. ˚Aberg, “Two-step deep-learning identification of heel keypoints from video-recorded gait,” Medical and Biological En- gineering and Computing, Sep 2024

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.