REVIEW 5 major objections 5 minor 32 references
Efficient Calisthenics Skills Classification through Foreground Instance Selection and Depth Estimation
T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The authors claim that calisthenics skill classification can be made faster and more accurate by cropping the athlete with a detector and classifying a depth patch, replacing the costly pose-estimation stage.
desk verdict The depth-patch pipeline is a sensible new empirical data point, but the 0.837-vs-0.815 superiority claim hangs on a split that the paper never describes. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the foreground instance selection step: a YOLOv10 person detector chooses the most prominent bounding box by a weighted score of detection confidence (0.6) and normalized area (0.4), enlarges smaller boxes more (5 to 15 percent), clips the box to the frame, and falls back to a 20-percent-smaller center crop when detection is missing or below 1 percent of the frame area. On the chosen crop, the depth-patch path runs Depth Anything V2 and feeds the resulting map to a pretrained EfficientNetV2 CNN; inference time is combined with accuracy through the WAITT metric, $\mathrm{WAITT} = \frac{A}{\mathrm{IT}^{\gamma}} + \alpha(1-A)$, with $\alpha=1$ and $\gamma=2$. This machinery is what lets the pipeline drop pose estimation entirely while still isolating the athlete from background clutter.
What would settle it
Run the depth-patch and RGB-patch pipelines and the skeleton baseline on a split that keeps all frames of each video together, then check whether depth patches still beat 0.815; if the margin disappears, the claimed superiority depends on the split rather than the method.
Extended reading notes
Core claim
The authors set out to show that skeleton extraction is not a necessary step for classifying static calisthenics skills. Their pipeline removes the background by asking YOLOv10 to find the athlete, crops the detected region (with a fallback center crop when no confident detection is found), and feeds that patch to an EfficientNetV2 classifier. The paper reports that the best variant, which first converts the RGB patch into a depth patch with Depth Anything V2, reaches 0.837 accuracy on a dataset of nine skills plus a background class, against 0.815 for the OpenPose-MLP skeleton baseline; on the same table the depth-patch variant also has the highest precision, recall, F1, and WAITT trade-off score. The RGB-patch variant is much faster, at 0.01 seconds per frame, and still reaches 0.792 accuracy, which the authors propose as the right choice for latency-limited settings.
Load-bearing premise
The reported comparison assumes that the 0.815 accuracy quoted from the earlier skeleton work was measured on the same train/test frame split as the new CNN experiments, with no frames from the same video in both sets.
Editorial extensions
If this is right
- The skeleton-estimation stage can be dropped for static calisthenics recognition: depth patches give 0.837 accuracy versus 0.815 for OpenPose-MLP while taking 0.176 seconds per frame rather than 0.383.
- For latency-critical settings, RGB-patch classification provides 0.792 accuracy at 0.01 seconds per frame, about 38 times faster than the skeleton baseline.
- Because the detector, depth estimator, and classifier are separate modules, replacing any component should improve the pipeline without retraining the whole system.
- Under the paper's WAITT settings, both the depth-patch and RGB-patch pipelines rank above the skeleton baseline, giving a single score for accuracy-versus-speed comparisons.
Reading between the lines
- The authors do not describe the train/test frame split; if frames from the same video appear in both sets, temporal leakage could inflate all accuracies, so a video-level split would settle whether the 0.022-point margin over the skeleton baseline is real.
- Because the recipe needs no joint annotations, the same detector-crop-depth-classifier stack is a plausible template for other static pose or fitness-movement classification tasks, such as yoga pose recognition.
- The modular design leaves room for a faster or more accurate depth estimator than Depth Anything V2 to be swapped in without changing the classifier; the paper does not test such an upgrade.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses calisthenics skill classification from images, proposing four CNN-based pipelines that avoid explicit pose estimation: classification from full RGB frames, full depth frames (from Depth Anything V2), RGB patches cropped with YOLOv10 person detection, and depth patches estimated on those cropped regions. On the authors' prior calisthenics dataset, EfficientNetV2 is selected as the best backbone, and the depth-patch configuration is reported to achieve the highest accuracy (0.837), the best WAITT score (4.314), and a lower average inference time than the OpenPose-MLP skeleton baseline (0.176 s vs 0.383 s). The paper also introduces a weighted accuracy-inference-time trade-off metric and includes feature-map visualizations arguing that depth inputs focus better on the foreground.
Significance. If the reported numbers are protocol-consistent, the paper offers a practical, modular alternative to skeleton-based calisthenics classification, with a plausible efficiency gain and a public code/weights link (github.com/antof27/rgb-based-pose-classification). The study covers several backbones, both pretrained and randomly initialized settings, and four input modalities, which is a useful empirical sweep for this niche task. The WAITT metric, while ad hoc, is a reasonable way to compare accuracy-latency trade-offs. However, the central comparison against the skeleton baseline depends on an unspecified train/test split and on a baseline accuracy taken from the authors' prior work, so the headline 'outperforms skeleton-based methods' is not yet established independently.
major comments (5)
- [Section 4 (Dataset)] The train/test split is never described. Section 4 reports 115,843 frames extracted from 839 videos at 24 fps but does not state whether the split is by video or by random frame. If frames from the same video occur in both training and test sets, temporally adjacent near-duplicates can leak, inflating all accuracies in Tables 1-4 and the 0.022-point margin in Table 4. Please specify the split protocol and, ideally, perform a video-disjoint split and re-report all numbers.
- [Section 4.4, Table 4] The OP-MLP baseline accuracy of 0.815 is taken from prior work [5] rather than re-evaluated under the protocol used for the new CNN experiments. Since the dataset, preprocessing, and augmentation details in [5] may differ from those used here, this comparison is only valid if both numbers are produced on the same train/test frames. Please re-run the OpenPose-MLP baseline on the same split used for the proposed approaches and report its metrics, or state explicitly why the prior numbers are directly comparable.
- [Tables 2-4] There is an internal inconsistency about which configuration achieves 0.837. Table 2 reports 0.837 for the non-pretrained Depth Patches model and 0.834 for the pretrained one, while Table 3 states 'All CNNs have been pretrained' and Table 4 labels the depth-patch row as pretrained with accuracy 0.837. Since the headline claim is that depth patches outperform the skeleton baseline, the exact configuration (randomly initialized vs ImageNet-pretrained) must be disambiguated and corrected in all tables.
- [Section 4.3, Table 3] The reported AVG IIT for RGB Patches (0.01 s) is not accompanied by a specification of what is included in this measurement. If this number covers only the CNN forward pass and excludes the YOLOv10 detection, cropping, and preprocessing, then the 38.3x speedup claimed in the abstract over the OP-MLP baseline is misleading. Please provide a component-wise timing breakdown and clarify whether the OP-MLP timing includes OpenPose inference and MLP forward pass in the same environment.
- [Section 4.4] No error bars, standard deviations, or significance tests are reported for the accuracy values in Table 4, so the claim that depth patches 'outperforms ... across all metrics' rests on a single run with a margin of 0.022 in accuracy. Please report multiple runs with variance or a paired test over the test set.
minor comments (5)
- [Abstract and Table 4] The abstract highlights '38.3x faster inference with RGB image patches', while Table 4 shows the depth-patch approach (the main accuracy winner) at 0.176 s versus 0.383 s, which is only about 2.2x faster. Please clarify which comparison is being claimed in the abstract to avoid overstatement.
- [Equation (1)] The WAITT metric depends on the hyperparameters alpha and gamma, which are set to 1 and 2 without a sensitivity analysis. Since the ranking of approaches can change with these values, a short sensitivity study would strengthen the efficiency claims.
- [Section 4.2] The paper states that 2.60% of frames required center cropping, but does not report how these frames are distributed across classes or whether accuracy changes when they are excluded. This information would help assess the robustness of the patch-extraction step.
- [Figure 3 caption] The caption says 'top: no athlete detected, bottom: incorrect person detected', but the figure contains multiple panels that are not individually labeled. Adding subfigure labels (a), (b), etc. would make the failure cases easier to interpret.
- [Reference [5]] Reference [5] is incomplete: 'Proceedings Copyright (2024)' does not identify the venue or give page/article identifiers. Please provide full bibliographic information.
Circularity Check
No circularity: the pipeline comparison is empirical and the accuracy numbers are measured, not derived from the inputs.
full rationale
The paper's claims are empirical pipeline comparisons. The depth-patch accuracy (0.837) and RGB-patch accuracy (0.792) are obtained by training EfficientNetV2 on the authors' dataset and evaluating classifiers; they are not computed from, nor forced by, any equation in the paper. The compared skeleton-based baseline (OP-MLP, 0.815 accuracy, 0.383s) is taken from the authors' prior work [5]. This is a self-citation, and the comparison would be stronger if the baseline were re-run under the same split, but it is not circular: the cited number is an external, falsifiable empirical result, not a parameter fitted in this paper, and the new accuracy values do not reduce to it by construction. WAITT (Eq. 1) is a newly defined evaluation metric, not a prediction derived from the model; it aggregates measured accuracy and measured inference time. The unresolved issues — the unpublished train/test split, potential temporal leakage from 24 fps frames of the same videos, and the inconsistent pretrained/non-pretrained labeling of the 0.837 depth-patch result (Tables 2-4) — are correctness and reproducibility concerns, not circularity. Under the stated rules, self-citation alone is not circularity, and no equation or fitted parameter is renamed as a prediction.
Assumptions & free parameters
free parameters (4)
- WAITT trade-off parameters =
alpha=1, gamma=2
- Athlete selection weights =
0.6 confidence, 0.4 area
- Patch enlargement and fallback rules =
5-15%, 1%, 20%
- YOLO confidence threshold =
0.2
assumptions (4)
- domain assumption DAV2 color-mapped relative depth separates athlete from background.
- domain assumption YOLOv10 detects athletes in calisthenics poses.
- ad hoc to paper ImageNet-pretrained CNN features transfer to color-mapped depth images.
- domain assumption Frames can be treated as independent samples.
Cite this review
Pith. "Pith review of Efficient Calisthenics Skills Classification through Foreground Instance Selection and Depth Estimation." pith.science (2026). https://pith.science/paper/LI6ZFWXU
@misc{pith2026250712292,
author = {Pith},
title = {Pith review of: Efficient Calisthenics Skills Classification through Foreground Instance Selection and Depth Estimation},
year = {2026},
howpublished = {\url{https://pith.science/paper/LI6ZFWXU}},
note = {Machine review of arXiv:2507.12292}
}
read the original abstract
Calisthenics skill classification is the computer vision task of inferring the skill performed by an athlete from images, enabling automatic performance assessment and personalized analytics. Traditional methods for calisthenics skill recognition are based on pose estimation methods to determine the position of skeletal data from images, which is later fed to a classification algorithm to infer the performed skill. Despite the progress in human pose estimation algorithms, they still involve high computational costs, long inference times, and complex setups, which limit the applicability of such approaches in real-time applications or mobile devices. This work proposes a direct approach to calisthenics skill recognition, which leverages depth estimation and athlete patch retrieval to avoid the computationally expensive human pose estimation module. Using Depth Anything V2 for depth estimation and YOLOv10 for athlete localization, we segment the subject from the background rather than relying on traditional pose estimation techniques. This strategy increases efficiency, reduces inference time, and improves classification accuracy. Our approach significantly outperforms skeleton-based methods, achieving 38.3x faster inference with RGB image patches and improved classification accuracy with depth patches (0.837 vs. 0.815). Beyond these performance gains, the modular design of our pipeline allows for flexible replacement of components, enabling future enhancements and adaptation to real-world applications.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[5]
Finocchiaro, A., Farinella, G.M., Furnari, A.: Calisthenics skills temporal video segmentation. Proceedings Copyright (2024)
work page 2024
-
[1]
In: Proceedings of the IEEE/CVF ICCV (2023)
Aniraj, A., Dantas, C.F., Ienco, D., Marcos, D.: Masking strategies for background bias removal in computer vision models. In: Proceedings of the IEEE/CVF ICCV (2023)
work page 2023
-
[2]
Advances in Neural Information Processing Systems (2024)
Bhatt, G., Das, D., Sigal, L., N Balasubramanian, V.: Mitigating the effect of incidental correlations on part-based learning. Advances in Neural Information Processing Systems (2024)
work page 2024
-
[3]
In: Proceedings of the IEEE conference on CVPR (2017)
Cao,Z.,Simon,T.,Wei,S.E.,Sheikh,Y.:Realtimemulti-person2dposeestimation using part affinity fields. In: Proceedings of the IEEE conference on CVPR (2017)
work page 2017
-
[4]
Fine-grained Visual Classification with High-temperature Refinement and Background Suppression
Chou, P., Kao, Y., Lin, C.: Fine-grained visual classification with high- temperature refinement and background suppression. arxiv 2023. arXiv preprint arXiv:2303.06442 (2023)
work page Pith review arXiv 2023
-
[6]
arXiv preprint arXiv:1807.02811 (2018)
Frazier, P.I.: A tutorial on bayesian optimization. arXiv preprint arXiv:1807.02811 (2018)
arXiv 2018
-
[7]
Freire-Obregón, D., Lorenzo-Navarro, J., Santana, O.J., Hernández-Sosa, D., Castrillón-Santana, M.: Towards cumulative race time regression in sports: I3d convnet transfer learning in ultra-distance running events. In: 2022 26th ICPR (2022)
work page 2022
-
[8]
In: International Conference on Image Analysis and Processing (2023)
Freire-Obregón, D., Lorenzo-Navarro, J., Santana, O.J., Hernández-Sosa, D., Castrillón-Santana, M.: A large-scale analysis of athletes’ cumulative race time in running events. In: International Conference on Image Analysis and Processing (2023)
work page 2023
Show all 32 references
-
[9]
In: 2023 IJCB (2023)
Freire-Obregón, D., Lorenzo-Navarro, J., Santana, O.J., Hernández-Sosa, D., Castrillón-Santana, M.: A large-scale re-identification analysis in sporting scenar- ios: The betrayal of reaching a critical point. In: 2023 IJCB (2023)
2023
-
[10]
In: Proceedings of the IEEE conference on CVPR (2018)
Güler, R.A., Neverova, N., Kokkinos, I.: Densepose: Dense human pose estimation in the wild. In: Proceedings of the IEEE conference on CVPR (2018)
2018
-
[11]
In: Proceedings of the IEEE conference on CVPR (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on CVPR (2016)
2016
-
[12]
Neurocomputing (2015) 12 Finocchiaro et al
He, L., Wang, G., Liao, Q., Xue, J.H.: Depth-images-based pose estimation using regression forests and graphical models. Neurocomputing (2015) 12 Finocchiaro et al
2015
-
[13]
In: Proceedings of the IEEE/CVF international conference on computer vision (2019)
Howard, A., Sandler, M., Chu, G., Chen, L.C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V., et al.: Searching for mobilenetv3. In: Proceedings of the IEEE/CVF international conference on computer vision (2019)
2019
-
[14]
arXiv preprint arXiv:2303.07399 (2023)
Jiang, T., Lu, P., Zhang, L., Ma, N., Han, R., Lyu, C., Li, Y., Chen, K.: Rtm- pose: Real-time multi-person pose estimation based on mmpose. arXiv preprint arXiv:2303.07399 (2023)
2023 arXiv
-
[15]
Sensors (2018)
Li, M., Hashimoto, K.: Accurate object pose estimation using depth only. Sensors (2018)
2018
-
[16]
Battleground Creative (2016)
Low, S.: Overcoming Gravity: A Systematic Approach To Gymnastics And Body- weight Strength (Second Edition). Battleground Creative (2016)
2016
-
[17]
In: Proceedings of the IEEE conference on CVPR Workshops (2017)
Park, S., Yong Chang, J., Jeong, H., Lee, J.H., Park, J.Y.: Accurate and efficient 3d human pose estimation algorithm using single depth images for pose analysis in golf. In: Proceedings of the IEEE conference on CVPR Workshops (2017)
2017
-
[18]
In: 2019 WACV (2019)
Parmar, P., Morris, B.: Action quality assessment across multiple actions. In: 2019 WACV (2019)
2019
-
[19]
In: Proceedings of the IEEE conference on CVPR Workshops (2017)
Parmar, P., Tran Morris, B.: Learning to score olympic events. In: Proceedings of the IEEE conference on CVPR Workshops (2017)
2017
-
[20]
IEEE transactions on pattern analysis and machine intelligence (2020)
Ranftl, R., Lasinger, K., Hafner, D., Schindler, K., Koltun, V.: Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE transactions on pattern analysis and machine intelligence (2020)
2020
-
[21]
International journal of computer vision (2015)
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al.: Imagenet large scale visual recog- nition challenge. International journal of computer vision (2015)
2015
-
[22]
IEEE Transactions on Pattern Analysis and Machine Intelligence (2013)
Shotton, J., Girshick, R., Fitzgibbon, A., Sharp, T., Cook, M., Finocchio, M., Moore, R., Kohli, P., Criminisi, A., Kipman, A., Blake, A.: Efficient human pose estimation from single depth images. IEEE Transactions on Pattern Analysis and Machine Intelligence (2013)
2013
-
[23]
In: Proceedings of the IEEE/CVF conference on CVPR (2019)
Si, C., Chen, W., Wang, W., Wang, L., Tan, T.: An attention enhanced graph convolutional lstm network for skeleton-based action recognition. In: Proceedings of the IEEE/CVF conference on CVPR (2019)
2019
-
[24]
Applied Sciences (2019)
Su, J.Y., Cheng, S.C., Chang, C.C., Chen, J.M.: Model-based 3d pose estimation of a single rgb image using a deep viewpoint classification neural network. Applied Sciences (2019)
2019
-
[25]
In: Proceedings of the IEEE conference on CVPR (2016)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the incep- tion architecture for computer vision. In: Proceedings of the IEEE conference on CVPR (2016)
2016
-
[26]
In: Interna- tional conference on machine learning
Tan, M., Le, Q.: Efficientnetv2: Smaller models and faster training. In: Interna- tional conference on machine learning. PMLR (2021)
2021
-
[27]
arXiv preprint arXiv:2312.15236 (2023)
Torón-Artiles, J., Hernández-Sosa, D., Santana, O.J., Lorenzo-Navarro, J., Freire- Obregón, D.: Classifying soccer ball-on-goal position through kicker shooting ac- tion. arXiv preprint arXiv:2312.15236 (2023)
2023 arXiv
-
[28]
Advances in Neural Information Processing Systems 37, 107984–108011 (2024)
Wang, A., Chen, H., Liu, L., Chen, K., Lin, Z., Han, J., et al.: Yolov10: Real-time end-to-end object detection. Advances in Neural Information Processing Systems 37, 107984–108011 (2024)
2024
-
[29]
arXiv preprint arXiv:2210.02748 (2022)
Wang, K., Machiraju, H., Choung, O.H., Herzog, M., Frossard, P.: Clad: A contrastive learning based approach for background debiasing. arXiv preprint arXiv:2210.02748 (2022)
2022 arXiv
-
[30]
In: Proceedings of the IEEE conference on CVPR (2017) Efficient Calisthenics Skills Classification 13
Xie,S.,Girshick,R.,Dollár,P.,Tu,Z.,He,K.:Aggregatedresidualtransformations for deep neural networks. In: Proceedings of the IEEE conference on CVPR (2017) Efficient Calisthenics Skills Classification 13
2017
-
[31]
In: Proceedings of the AAAI conference on ar- tificial intelligence (2018)
Yan, S., Xiong, Y., Lin, D.: Spatial temporal graph convolutional networks for skeleton-based action recognition. In: Proceedings of the AAAI conference on ar- tificial intelligence (2018)
2018
-
[32]
Advances in Neural Information Processing Systems (2024)
Yang, L., Kang, B., Huang, Z., Zhao, Z., Xu, X., Feng, J., Zhao, H.: Depth anything v2. Advances in Neural Information Processing Systems (2024)
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.