Pith. sign in

REVIEW 3 major objections 5 minor 18 references

Real-Time Fitness Exercise Classification and Counting from Video Frames

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A bidirectional LSTM that reads 30-frame windows of body landmarks, with joint angles added to raw coordinates, classifies four exercises with over 99% test accuracy and runs in real time from a webcam.

desk verdict A practical BiLSTM exercise classifier with genuinely external test sets, but the 99% headline rests on an unspecified split unit and the 'best-performing' claim is undercut by the paper's own gym results. read the letter →

arxiv 2411.11548 v1 pith:TBFIBVFO submitted 2024-11-18 cs.CV cs.AIcs.LG

Riccardo Riccio This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords exerciseclassificationBiLSTMposeestimationBlazelandmarksjointanglestemporalsequencerepetitioncountingreal-timevideoanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's claim is that exercise recognition should treat movement as a sequence, not as a bag of frames. A bidirectional LSTM that reads 30-frame windows of body landmarks, with 12 joint angles added to the raw (x, y, z) coordinates, can classify squats, push-ups, shoulder presses, and bicep curls in real time without the user manually selecting the exercise. The authors report 99.24% accuracy on their main test set, 95.05% on newly recorded home videos, and 87.91% on gym videos with harder angles and occlusions. They also report that the mixed angle-plus-coordinate input generalizes better than raw coordinates alone or angles alone, and that their model outperforms reimplemented prior approaches on the main and home test sets.

What carries the argument

The load-bearing mechanism is the 30-frame sliding window of pose features fed into a two-layer Bidirectional LSTM. Each window contains the (x, y, z) coordinates of 22 relevant BlazePose landmarks plus 12 joint angles, including hip-shoulder-elbow, shoulder-elbow-wrist, hip-knee-ankle, shoulder-hip-knee, knee-ankle-heel, and ankle-heel-foot-index angles on both sides. The bidirectional pass lets the model read each moment in light of both earlier and later frames, so exercises that look similar at a single instant, like the start of a squat and a shoulder press, are separated by how the angles evolve over time. The same window is used for classification and feeds the rule-based angle thresholds that count repetitions once the exercise is known.

What would settle it

Group every window by its source video ID, check that no video contributes windows to both training and test, then retrain from scratch with a video-level split and report test accuracy; if it drops well below 99%, the original claim did not measure generalization to unseen footage.

Watch

Extended reading notes

Core claim

The central discovery is that a BiLSTM fed 30-frame sequences of BlazePose landmark data, combining the 12 joint angles that are invariant to camera distance and body proportions with raw coordinates that help when landmarks are occluded, can recognize an exercise from its motion pattern alone. On the combined synthetic-and-real training set the model reaches over 99% accuracy on the held-out test split, and on two additional test sets that mimic real app use it reaches about 95% accuracy at home and about 88% in a gym. The paper presents this as evidence that temporal context plus perspective-invariant features solves the generalization problem that limits frame-wise and raw-coordinate classifiers. In the comparison experiments the mixed-feature BiLSTM scores highest on the main and home test sets; the paper concludes it is the best-performing approach.

Load-bearing premise

The entire 99% accuracy figure assumes the 70-15-15 split keeps whole videos apart; if 30-frame windows from the same video land in both training and test, the test set is not independent and the accuracy is inflated.

Editorial extensions

If this is right

  • A real-time fitness app can recognize which of four exercises a user is doing without any manual selection, using only a laptop or phone webcam.
  • Combining viewpoint-invariant joint angles with raw coordinates is a better default feature set for pose-sequence classifiers than either feature type alone.
  • Sequence models that read 30 frames beat frame-wise or short-window alternatives on home-style video, so temporal context is the main driver of the improvement.
  • The same architecture can be extended to more exercises simply by adding labeled videos and retraining, reusing the existing angle-plus-coordinate pipeline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 99% test figure is only trustworthy if the 70-15-15 split separates whole videos, not individual 30-frame windows; the paper does not say which, so the number should be re-checked with a video-level split.
  • The drop from 99% to 88% on gym video hints that occluded joints break the angle features; a natural extension is to mask missing landmarks explicitly or train with random landmark dropout.
  • Because repetition counting is hand-coded angle thresholds while classification is learned, the two modules can disagree; an end-to-end counter trained from the same windows would remove that manual layer.
  • A direct transfer test on videos from sources never seen in training, such as new online uploads, would show whether the model generalizes beyond the particular clips used to build the dataset.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This manuscript proposes a real-time exercise classifier based on BlazePose landmarks and a Bidirectional LSTM. The model consumes 30-frame windows of joint coordinates plus derived angles to classify four exercises (barbell bicep curl, push-up, squat, shoulder press). Training data combines the Kaggle Workout/Exercises Video Dataset, the InfiniteRep synthetic dataset, and additional online videos; the paper reports 99% accuracy on a 70-15-15 split and additional results on two external test sets (home and gym). The authors also reimplement two prior baselines, present a comparison table, and describe a Streamlit web application. The abstract claims both 'over 99%' test accuracy and that the proposed model is 'the best-performing one.'

Significance. The paper has several praiseworthy features: two external test sets recorded in home and gym conditions, ablations over feature types (raw coordinates, angles, invariant features), reimplementation of two literature baselines on the same data, and a public GitHub repository. If the data-splitting issue is resolved, the contribution is a useful practical system, though the methodological novelty is incremental: BiLSTM sequence classification over pose features is a standard recipe, and the main specific choices are the angle-plus-coordinate feature set and the 30-frame window. The external accuracies (about 95% on the home set and 88% on the gym set) are more credible and policy-relevant than the 99% headline and should be foregrounded.

major comments (3)
  1. [Section 3.3.4] The description 'the dataset is split into training, validation and, test sets (70-15-15)' does not specify whether the split is performed on whole videos or on the 30-frame windows constructed in Section 3.2. Because adjacent windows from the same video overlap by 29 frames, a window-level split would place near-duplicate landmark sequences in both training and test, which can produce an inflated test accuracy of 0.9924. The support of 791 in Tables 5 and 6 is consistent with window-level counting rather than video-level counting. Please state the split unit, use a video-level split (or otherwise guarantee that no window from a training video appears in the test set), report the number of train/validation/test videos, and verify that the 'additional videos from various free online sources' are disjoint from the Kaggle and InfiniteRep videos. If the split was window-level, re-run the evaluation and revise the abstract's 'over 99%' claim.
  2. [Abstract and Section 4.7 (Table 20)] The abstract's unconditional statement that 'the proposed model is the best-performing one' is not supported by the paper's own results. Table 20 reports Final Test Gym accuracy of 0.8856 for CNN with Soft Voting versus 0.8791 for the proposed BiLSTM with mixed features, while the proposed model is best on the held-out split and My Test Video. Please either restrict the 'best-performing' claim to the conditions where it holds (e.g., the held-out split and the home test set) or explicitly discuss the gym-setting result; as written, the claim overstates the evidence.
  3. [Section 4.6] The comparison with [5] is skipped, with the justification that earlier comparisons already demonstrate superiority of BiLSTM over LSTM and of mixed features over raw coordinates. Those earlier comparisons are against the authors' own LSTM and BiLSTM variants, not against the exact architecture, feature set, and 8-frame window of [5]. Consequently, the 'best-performing' claim is not established relative to all three prior approaches discussed in Section 2.2. Please either implement [5] or clearly state that the comparison covers only [4] and [6], and adjust the corresponding claims.
minor comments (5)
  1. [Table 16] In the Sequence-Level block, the Shoulder Press row reports precision 1.00, recall 1.00, but F1-score 0; this is presumably a typo and should be 1.00.
  2. [Table 2] The coordinate landmark list contains apparent typos: 'LEFT HOOT' should likely be 'LEFT FOOT', and 'RIGHT THUMB, RIGHT THUMB' appears to duplicate the same landmark instead of listing two distinct entries.
  3. [Section 4.4 / Tables 17 and 19] The support counts for 'Final Test Gym' are inconsistent across tables: Table 17 sums to 312 (90+76+58+88) while Tables 9 and 10 sum to 306 (90+70+58+88) for the same named set. Please reconcile the counts and make the evaluation protocol for each comparison explicit.
  4. [Table 20] The column labeled 'Test Set Accuracy' is ambiguous because Section 3.3.4 does not define whether the underlying split is video-level or window-level; after resolving the split unit, rename this column accordingly (e.g., 'Held-out Video Test' or 'Held-out Window Test').
  5. [Sections 4.3-4.4] The external test sets contain 202 and 306 window-level samples, but these windows are not independent; reporting per-video accuracy or confidence intervals would make the generalization claims more robust.

Circularity Check

1 steps flagged · score 4.0 of 10

The headline 99% test-set accuracy is partly a selection artifact: the same test set used to choose the best model and hyperparameters is then reported as the performance result.

  1. fitted input called prediction [Section 3.3.4, Model Evaluation]
    "The models were evaluated using the accuracy metric on the test set, and the best-performing model and hyperparameters were selected based on this metric."

    The abstract's headline 'accuracy of over 99% on the test set' is the value of the metric that was used as the model-selection objective on that same test set. Because the test set directly determines which model and hyperparameters are chosen, the reported test accuracy is an optimistically biased selection result, not an independent evaluation on unseen data. In other words, the 'prediction' of 99% accuracy is partially forced by the fact that the test set itself was used to pick the model. The separate home and gym test sets provide some independent evidence of generalization, but they do not validate the specific 99% test-set claim, which is the central quantitative result in the abstract.

full rationale

This is fundamentally an empirical training paper rather than a derivation from first principles, so most of the circularity patterns do not apply. There is no self-definitional reasoning, no load-bearing self-citation chain, no imported uniqueness theorem, and no ansatz smuggled in via citation. The main circularity-adjacent problem is evaluation leakage through model selection: Section 3.3.4 explicitly uses the test set to select the best-performing model and hyperparameters, and the same test-set accuracy is then reported as the headline result. This makes the 99% figure partly a fitted selection outcome. Separate concerns that are not, strictly speaking, circularity: the paper never states whether the 70-15-15 split is performed on whole videos or on overlapping 30-frame windows, which creates a potential data-leakage risk for the 99% claim; and Table 20 shows the CNN+soft-voting baseline achieving 0.8856 on Final Test Gym versus 0.8791 for the proposed BiLSTM, contradicting the abstract's unconditional 'best-performing' statement. These are correctness and consistency issues rather than circular derivations, so they do not raise the circularity score further. The presence of external home and gym test sets is genuine independent evidence and keeps the paper from being wholly circular.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

All five assumptions are domain-level choices about the data and the sensor, not mathematical axioms. Any one of them failing would degrade the claimed accuracy; the viewpoint-invariance and video-level split assumptions are the most consequential.

free parameters (4)
  • BiLSTM hyperparameters = units 73, dropout 0.2174, lr 0.0004, batch 54, epochs 73
    Selected by random search over the ranges in Table 3; the model's reported accuracy depends on this fitted configuration.
  • Window length = 30 frames
    Chosen by hand to balance temporal context and real-time constraints; no ablation is reported showing that 30 is optimal or that results are robust to this choice.
  • Feature set composition = 66 raw coordinates + 12 angles
    The set of 22 landmarks and 12 angles was selected manually; the paper reports ablations but no systematic feature selection.
  • Missing-landmark placeholder = [0.0, 0.0, 0.0]
    Frames with missing landmarks are either skipped or filled with zeros; the zero-fill convention is a free modeling choice that can distort angle features when landmarks are absent.
assumptions (5)
  • domain assumption Joint angles computed from BlazePose landmarks are invariant to camera perspective, user positioning, and body differences.
    Invoked in the abstract and Section 2.3 as the main reason mixed angle plus coordinate features generalize; it is asserted, not demonstrated with controlled viewpoint changes.
  • domain assumption Each video contains exactly one exercise type, so a 30-frame window can be labeled by the first frame's exercise.
    Stated in Section 3.2.1: 'This aggregation assumes that each video contains only one type of exercise.' Violations would inject label noise into training and test windows.
  • domain assumption Synthetic InfiniteRep avatar videos are representative of real human webcam usage.
    Used in Section 3.1 to enlarge the training set; the paper offers no evaluation of the domain gap between avatars and real users.
  • domain assumption BlazePose landmark estimates are accurate enough under the test conditions, including lighting, occlusion, and camera angle.
    All features derive from BlazePose output in Section 3.2.1; errors in landmark detection propagate to angles and coordinates.
  • domain assumption The train/validation/test split is performed on independent videos, not on overlapping windows from the same video.
    Required for the 99% test accuracy to be meaningful; the paper does not specify the split unit in Section 3.3.4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Real-Time Fitness Exercise Classification and Counting from Video Frames." pith.science (2026). https://pith.science/paper/TBFIBVFO

@misc{pith2026241111548,
  author       = {Pith},
  title        = {Pith review of: Real-Time Fitness Exercise Classification and Counting from Video Frames},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TBFIBVFO}},
  note         = {Machine review of arXiv:2411.11548}
}
read the original abstract

This paper introduces a novel method for real-time exercise classification using a Bidirectional Long Short-Term Memory (BiLSTM) neural network. Existing exercise recognition approaches often rely on synthetic datasets, raw coordinate inputs sensitive to user and camera variations, and fail to fully exploit the temporal dependencies in exercise movements. These issues limit their generalizability and robustness in real-world conditions, where lighting, camera angles, and user body types vary. To address these challenges, we propose a BiLSTM-based model that leverages invariant features, such as joint angles, alongside raw coordinates. By using both angles and (x, y, z) coordinates, the model adapts to changes in perspective, user positioning, and body differences, improving generalization. Training on 30-frame sequences enables the BiLSTM to capture the temporal context of exercises and recognize patterns evolving over time. We compiled a dataset combining synthetic data from the InfiniteRep dataset and real-world videos from Kaggle and other sources. This dataset includes four common exercises: squat, push-up, shoulder press, and bicep curl. The model was trained and validated on these diverse datasets, achieving an accuracy of over 99% on the test set. To assess generalizability, the model was tested on 2 separate test sets representative of typical usage conditions. Comparisons with the previous approach from the literature are present in the result section showing that the proposed model is the best-performing one. The classifier is integrated into a web application providing real-time exercise classification and repetition counting without manual exercise selection. Demo and datasets are available at the following GitHub Repository: https://github.com/RiccardoRiccio/Fitness-AI-Trainer-With-Automatic-Exercise-Recognition-and-Counting.

Figures

Figures reproduced from arXiv: 2411.11548 by the authors.

Figure 1
Figure 1. topology of the 33 key points [2] 2.2 Automatic Exercise Classification Automatic exercise classification in real-time involves the use of machine learn￾ing models to analyze data extracted from video frames or sensor inputs to identify the exercise a user is performing. This paper focuses on using fea￾tures that can be extracted from frames with a simple webcam, eliminating the need for additional sensors and relyi… view at source ↗
Figure 2
Figure 2. Images from the Datasets used for training and validation of the [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Images of the datasets used for additional testing (images are cropped [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: model architecture 3.3.4 Model Evaluation During the training phase, the dataset is split into training, validation and, test sets (70-15-15) to assess the model’s performance on unseen data. To ensure that the models generalized well beyond the training data, both the…
Figure 5
Figure 5. Figure 5: Confusion Matrices for LSTM and BiLSTM Models [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Learning Curves for LSTM and BiLSTM Models [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Confusion Matrices for LSTM and BiLSTM Models [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Confusion Matrices for LSTM and BiLSTM Models [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Auto Classify Mode page 5.1 Exercise Recognition in Practice The Auto Classify Mode serves as the primary showcase for the exercise recog￾nition model. In this mode, the BiLSTM model processes a 30-frame window of pose data every second, classifying the exercise in rea…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 14 canonical work pages

  1. [4]

    S., & Park, S

    Bang, G. S., & Park, S. B. (2024). Workout classification using a convolu- tional neural network in ensemble learning. Sensors, 24(10), 3133

    work page 2024

  2. [6]

    Talal, A., & Chen, C. (2020). Recognizing exercises and counting repeti- tions in real time. arXiv preprint arXiv:2005.03194

    work page Pith review arXiv 2020

  3. [5]

    Moran, A., Gebka, B., Goldshteyn, J., Beyer, A., Johnson, N., & Neuwirth, A. (2022). Muscle vision: Real-time keypoint based pose classification of physical exercises. arXiv preprint arXiv:2203.12111

    work page Pith review arXiv 2022

  4. [1]

    Dosovitskiy, A. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

    arXiv 2020

  5. [2]

    L., Zhang, F., & Grundmann, M

    Bazarevsky, V., Grishchenko, I., Raveendran, K., Zhu, T. L., Zhang, F., & Grundmann, M. (2020). BlazePose: On-device real-time body pose track- ing. ArXiv, abs/2006.10204

    arXiv 2020

  6. [3]

    Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C., Yong, M.G., Lee, J., Chang, W., Hua, W., Georg, M., & Grundmann, M. (2019). MediaPipe: A framework for building perception pipelines. ArXiv, abs/1906.08172

    arXiv 2019

  7. [7]

    InfiniteRep. (n.d.). Papers with Code. https://paperswithcode.com/ dataset/infiniterep

    work page

  8. [8]

    Soomro, K. (2012). UCF101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402

    arXiv 2012

Show all 18 references
  1. [9]

    Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., & Sheikh, Y. (2018). OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence , 43, 172–186

    2018

  2. [10]

    Abdillah, H. (n.d.). Workout fitness video . [Dataset]. Kaggle. https:// www.kaggle.com/datasets/hasyimabdillah/workoutfitness-video

  3. [11]

    Push-ups videos . (n.d.). Pexels. https://www.pexels.com/search/ videos/push-ups/

  4. [12]

    Push-up videos . (n.d.). Pixabay. https://www.pixabay.com/videos/ search/push-up/

  5. [13]

    Push-up videos . (n.d.). Shutterstock. https://www.shutterstock.com/ it/video/search/push-up

  6. [14]

    Push-up videos . (n.d.). Storyblocks. https://www.storyblocks.com/ video/search/push-ups

  7. [15]

    Home. (n.d.). Netron. https://netron.app/. 27

  8. [16]

    Home. (n.d.). Streamlit. https://streamlit.io/

  9. [17]

    Riccio, R. (n.d.). Thesis fitness app code [GitHub repos- itory]. GitHub. https://github.com/RiccardoRiccio/ Fitness-AI-Trainer-With-Automatic-Exercise-Recognition-and-Counting

  10. [18]

    API reference: Introduction. (n.d.). OpenAI. https://platform.openai. com/docs/api-reference/introduction. 28

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.