{"id":"7b85f6ac-3da1-4fe8-84d3-19e00b14f161","arxiv_id":"2507.12292","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Using YOLO athlete cropping and Depth Anything V2 depth maps, a CNN classifies calisthenics skills with 0.837 accuracy, slightly above a 0.815 OpenPose skeleton baseline.","lead":"The paper tests four ways to identify calisthenics poses from a single image, replacing slow skeleton tracking with athlete detection and depth maps. The best depth-patch model reaches 0.837 accuracy versus 0.815 for a skeleton baseline, while the fastest RGB-patch variant runs about 38 times faster but is less accurate.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 0.837-vs-0.815 superiority claim depends on an unreported train/test split and an inconsistent pretrained-status label; a video-disjoint, identically-split comparison could erase the margin.","rationale":"Good-faith read: the pipeline is modular and the component choices are reasonable; the feature-map analysis is only qualitative and not the main claim. The strongest claim is the empirical superiority over a prior skeleton-based model. For that claim, the single most load-bearing condition is comparability of the two evaluations. The paper does not document the split nor verify that the prior baseline number was computed on the same split. Because the dataset consists of frame sequences, random frame splitting would create near-duplicate train/test examples, which is known to inflate accuracy and can easily account for a 0.022 margin. The additional inconsistency between Table 2 and Table 3 about whether the best depth-patch model is pretrained further undermines confidence in the exact number, though it is fixable. This is exactly the kind of ambiguity that a CONDITIONAL verdict should demand. It is not an internal mathematical contradiction in the method; it is a missing evaluation-protocol detail, plus a small labeling inconsistency. If the authors supply the split and re-run OP-MLP on it, the conclusion may very well stand. Therefore I do not move the verdict; the reader's conditional verdict already captures the requirement.","tokens_in":9117,"tokens_out":4426,"duration_ms":49264,"concrete_test":"Run a video-disjoint split: hold out all frames from a fixed subset of the 839 videos (e.g., 20% of videos), train the four EfficientNetV2 variants and the OP-MLP model [5] on the remaining frames, and report per-video grouped accuracy plus 95% bootstrap confidence intervals. Also resolve the pretrained-status ambiguity by checking the released checkpoint and training configuration for the 0.837 depth-patch row. If the depth-patch accuracy falls to within noise of 0.815, or if the margin is not significant, the abstract's comparative claim should be weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in §4.4 (Table 4) is that the EfficientNetV2 depth-patch pipeline outperforms the skeleton-based OP-MLP baseline (0.837 vs 0.815). This comparison is load-bearing only if both numbers come from the same evaluation protocol. The paper never states the dataset split: §4 reports 115,843 frames extracted from 839 videos at 24 fps, but does not say whether train/test are split by frame or by video. If frames are split randomly, consecutive frames from the same video are near-duplicates and can appear in both training and test sets, so the reported accuracies (and the 0.022-point margin) may be inflated by temporal leakage. The OP-MLP accuracy is taken from prior work [5], whose split may differ from the one used for the CNN experiments; no code or checkpoint is used to re-evaluate it under the new protocol. There is also an internal inconsistency: Table 2 gives Depth Patches accuracy 0.837 for the randomly initialized model and 0.834 for the pretrained model, while Table 3 says 'All CNNs have been pretrained' yet reports 0.837, and Table 4 labels this row pretrained. The headline number is therefore ambiguous. Because WAITT (§4.3, Eq. 1) is computed from this same accuracy, the efficiency claim inherits the uncertainty.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses calisthenics skill classification from images, proposing four CNN-based pipelines that avoid explicit pose estimation: classification from full RGB frames, full depth frames (from Depth Anything V2), RGB patches cropped with YOLOv10 person detection, and depth patches estimated on those cropped regions. On the authors' prior calisthenics dataset, EfficientNetV2 is selected as the best backbone, and the depth-patch configuration is reported to achieve the highest accuracy (0.837), the best WAITT score (4.314), and a lower average inference time than the OpenPose-MLP skeleton baseline (0.176 s vs 0.383 s). The paper also introduces a weighted accuracy-inference-time trade-off metric and includes feature-map visualizations arguing that depth inputs focus better on the foreground.","tokens_in":9398,"tokens_out":4132,"duration_ms":44926,"significance":"If the reported numbers are protocol-consistent, the paper offers a practical, modular alternative to skeleton-based calisthenics classification, with a plausible efficiency gain and a public code/weights link (github.com/antof27/rgb-based-pose-classification). The study covers several backbones, both pretrained and randomly initialized settings, and four input modalities, which is a useful empirical sweep for this niche task. The WAITT metric, while ad hoc, is a reasonable way to compare accuracy-latency trade-offs. However, the central comparison against the skeleton baseline depends on an unspecified train/test split and on a baseline accuracy taken from the authors' prior work, so the headline 'outperforms skeleton-based methods' is not yet established independently.","major_comments":[{"comment":"The train/test split is never described. Section 4 reports 115,843 frames extracted from 839 videos at 24 fps but does not state whether the split is by video or by random frame. If frames from the same video occur in both training and test sets, temporally adjacent near-duplicates can leak, inflating all accuracies in Tables 1-4 and the 0.022-point margin in Table 4. Please specify the split protocol and, ideally, perform a video-disjoint split and re-report all numbers.","section":"Section 4 (Dataset)"},{"comment":"The OP-MLP baseline accuracy of 0.815 is taken from prior work [5] rather than re-evaluated under the protocol used for the new CNN experiments. Since the dataset, preprocessing, and augmentation details in [5] may differ from those used here, this comparison is only valid if both numbers are produced on the same train/test frames. Please re-run the OpenPose-MLP baseline on the same split used for the proposed approaches and report its metrics, or state explicitly why the prior numbers are directly comparable.","section":"Section 4.4, Table 4"},{"comment":"There is an internal inconsistency about which configuration achieves 0.837. Table 2 reports 0.837 for the non-pretrained Depth Patches model and 0.834 for the pretrained one, while Table 3 states 'All CNNs have been pretrained' and Table 4 labels the depth-patch row as pretrained with accuracy 0.837. Since the headline claim is that depth patches outperform the skeleton baseline, the exact configuration (randomly initialized vs ImageNet-pretrained) must be disambiguated and corrected in all tables.","section":"Tables 2-4"},{"comment":"The reported AVG IIT for RGB Patches (0.01 s) is not accompanied by a specification of what is included in this measurement. If this number covers only the CNN forward pass and excludes the YOLOv10 detection, cropping, and preprocessing, then the 38.3x speedup claimed in the abstract over the OP-MLP baseline is misleading. Please provide a component-wise timing breakdown and clarify whether the OP-MLP timing includes OpenPose inference and MLP forward pass in the same environment.","section":"Section 4.3, Table 3"},{"comment":"No error bars, standard deviations, or significance tests are reported for the accuracy values in Table 4, so the claim that depth patches 'outperforms ... across all metrics' rests on a single run with a margin of 0.022 in accuracy. Please report multiple runs with variance or a paired test over the test set.","section":"Section 4.4"}],"minor_comments":[{"comment":"The abstract highlights '38.3x faster inference with RGB image patches', while Table 4 shows the depth-patch approach (the main accuracy winner) at 0.176 s versus 0.383 s, which is only about 2.2x faster. Please clarify which comparison is being claimed in the abstract to avoid overstatement.","section":"Abstract and Table 4"},{"comment":"The WAITT metric depends on the hyperparameters alpha and gamma, which are set to 1 and 2 without a sensitivity analysis. Since the ranking of approaches can change with these values, a short sensitivity study would strengthen the efficiency claims.","section":"Equation (1)"},{"comment":"The paper states that 2.60% of frames required center cropping, but does not report how these frames are distributed across classes or whether accuracy changes when they are excluded. This information would help assess the robustness of the patch-extraction step.","section":"Section 4.2"},{"comment":"The caption says 'top: no athlete detected, bottom: incorrect person detected', but the figure contains multiple panels that are not individually labeled. Adding subfigure labels (a), (b), etc. would make the failure cases easier to interpret.","section":"Figure 3 caption"},{"comment":"Reference [5] is incomplete: 'Proceedings Copyright (2024)' does not identify the venue or give page/article identifiers. Please provide full bibliographic information.","section":"Reference [5]"}],"recommendation":"major_revision","confidential_remarks":"The central risk is protocol mismatch: the proposed models are evaluated under an unspecified split while the baseline number comes from the authors' prior paper. Requiring a video-disjoint split and a re-run of the baseline on the identical protocol is the key to making the headline claim valid. The pretrained/non-pretrained inconsistency in Tables 2-4 should also be resolved before the paper can be considered reliable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know two things before reading. First, the paper is a sensible modular pipeline comparison, and the depth-patch result is a genuinely new empirical data point for calisthenics classification. Second, the headline advantage over the skeleton baseline is not yet supported, because the evaluation protocol is under-specified.\n\nWhat is actually new: combining YOLOv10 athlete cropping, Depth Anything V2, and an ImageNet CNN for this task; the observation that depth patches beat full depth and full RGB consistently across five CNN backbones; and WAITT, a simple latency-accuracy trade-off metric that could be reused. The ablations are reasonably thorough for an applied paper, and a code/weights link is provided. The feature-map inspection is modest but consistent with the depth-patch story.\n\nThe soft spots are in the comparison, not the pipeline. First, the paper never states the train/test split. With 115,843 frames from 839 videos, a random frame split would let near-duplicate frames from the same video leak into both training and test, inflating all accuracies and potentially erasing the 0.022-point margin. The OpenPose/MLP baseline from prior work [5] is not shown to use the same split. Second, the pretrained label is inconsistent: Table 2 reports 0.837 for randomly initialized depth patches and 0.834 for pretrained, while Tables 3 and 4 label the 0.837 row as pretrained. The headline number is ambiguous. Third, there are no error bars or significance tests, and 0.837 versus 0.815 on a single split is a small margin. Fourth, the abstract's 38.3x speedup comes from the RGB-patch variant while the accuracy claim comes from depth patches, and it is unclear whether the 0.01s timing includes YOLO detection and preprocessing or only the CNN forward pass.\n\nNone of these are fatal. They are missing experimental details a referee would demand, and all are fixable. The depth-patch benefit over full depth is internally consistent, and the modular design is reasonable.\n\nAs an editor, I would send this to peer review rather than desk reject, asking for a video-disjoint split, a re-run of the skeleton baseline under the same protocol, confidence intervals, and clarification of the timing and pretrained labels. A reader working on sports analytics or lightweight action recognition would get value here; I would not cite the 0.837 versus 0.815 claim until the split question is answered.","headline":"The depth-patch pipeline is a sensible new empirical data point, but the 0.837-vs-0.815 superiority claim hangs on a split that the paper never describes.","tokens_in":9920,"tokens_out":3964,"would_cite":false,"duration_ms":41788,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The authors claim that calisthenics skill classification can be made faster and more accurate by cropping the athlete with a detector and classifying a depth patch, replacing the costly pose-estimation stage.","keywords":["calisthenics skill recognition","foreground instance selection","depth estimation","pose-free classification","EfficientNetV2","YOLOv10","Depth Anything V2"],"falsifier":"Run the depth-patch and RGB-patch pipelines and the skeleton baseline on a split that keeps all frames of each video together, then check whether depth patches still beat 0.815; if the margin disappears, the claimed superiority depends on the split rather than the method.","tokens_in":8903,"feed_emoji":"🤸","tokens_out":7875,"duration_ms":80069,"temperature":0.7,"pith_summary":"The paper tries to establish that calisthenics skill classification does not need a pose-estimation stage. It proposes a pipeline that first uses YOLOv10 to crop the athlete out of each frame and then, in the best configuration, runs Depth Anything V2 on the crop and feeds the resulting depth patch to an EfficientNetV2 classifier. On a ten-class calisthenics dataset this depth-patch configuration reaches 0.837 accuracy while the skeleton-based OpenPose-MLP baseline reaches 0.815, and it takes 0.176 seconds per frame against 0.383. RGB patches are the fastest at 0.01 seconds but slightly less accurate, so the paper argues the best accuracy-versus-speed trade-off is captured by its WAITT metric.","feed_headline":"Depth patches beat skeleton models for calisthenics","feed_subtitle":"Cropping the athlete and estimating depth gives 0.837 accuracy and runs about 2x faster than OpenPose.","key_machinery":"The load-bearing object is the foreground instance selection step: a YOLOv10 person detector chooses the most prominent bounding box by a weighted score of detection confidence (0.6) and normalized area (0.4), enlarges smaller boxes more (5 to 15 percent), clips the box to the frame, and falls back to a 20-percent-smaller center crop when detection is missing or below 1 percent of the frame area. On the chosen crop, the depth-patch path runs Depth Anything V2 and feeds the resulting map to a pretrained EfficientNetV2 CNN; inference time is combined with accuracy through the WAITT metric, $\\mathrm{WAITT} = \\frac{A}{\\mathrm{IT}^{\\gamma}} + \\alpha(1-A)$, with $\\alpha=1$ and $\\gamma=2$. This machinery is what lets the pipeline drop pose estimation entirely while still isolating the athlete from background clutter.","core_discovery":"The authors set out to show that skeleton extraction is not a necessary step for classifying static calisthenics skills. Their pipeline removes the background by asking YOLOv10 to find the athlete, crops the detected region (with a fallback center crop when no confident detection is found), and feeds that patch to an EfficientNetV2 classifier. The paper reports that the best variant, which first converts the RGB patch into a depth patch with Depth Anything V2, reaches 0.837 accuracy on a dataset of nine skills plus a background class, against 0.815 for the OpenPose-MLP skeleton baseline; on the same table the depth-patch variant also has the highest precision, recall, F1, and WAITT trade-off score. The RGB-patch variant is much faster, at 0.01 seconds per frame, and still reaches 0.792 accuracy, which the authors propose as the right choice for latency-limited settings.","pith_inferences":["The authors do not describe the train/test frame split; if frames from the same video appear in both sets, temporal leakage could inflate all accuracies, so a video-level split would settle whether the 0.022-point margin over the skeleton baseline is real.","Because the recipe needs no joint annotations, the same detector-crop-depth-classifier stack is a plausible template for other static pose or fitness-movement classification tasks, such as yoga pose recognition.","The modular design leaves room for a faster or more accurate depth estimator than Depth Anything V2 to be swapped in without changing the classifier; the paper does not test such an upgrade."],"forward_implications":["The skeleton-estimation stage can be dropped for static calisthenics recognition: depth patches give 0.837 accuracy versus 0.815 for OpenPose-MLP while taking 0.176 seconds per frame rather than 0.383.","For latency-critical settings, RGB-patch classification provides 0.792 accuracy at 0.01 seconds per frame, about 38 times faster than the skeleton baseline.","Because the detector, depth estimator, and classifier are separate modules, replacing any component should improve the pipeline without retraining the whole system.","Under the paper's WAITT settings, both the depth-patch and RGB-patch pipelines rank above the skeleton baseline, giving a single score for accuracy-versus-speed comparisons."],"supporting_citations":[{"why":"Supplies the calisthenics dataset and the OpenPose-MLP skeleton baseline with 0.815 accuracy that the paper compares against.","marker":"[5]"},{"why":"Provides Depth Anything V2, the monocular depth estimator used to produce the depth images and depth patches.","marker":"[32]"},{"why":"Provides YOLOv10, the person detector used for foreground instance selection and patch extraction.","marker":"[28]"},{"why":"Provides EfficientNetV2, the CNN classifier that carries the classification in all reported configurations.","marker":"[26]"},{"why":"Supplies ImageNet pretrained weights, which the paper uses for the best-performing classifier configurations.","marker":"[21]"}],"fun_headline_variants":["Depth patches beat skeleton models for calisthenics","Skip pose estimation: depth patches classify calisthenics","Calisthenics skills without skeleton extraction","Depth beats skeleton in calisthenics recognition"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported comparison assumes that the 0.815 accuracy quoted from the earlier skeleton work was measured on the same train/test frame split as the new CNN experiments, with no frames from the same video in both sets.","fun_headline_variants_meta":{"raw":{"variants":["Depth patches beat skeleton models for calisthenics","Skip pose estimation: depth patches classify calisthenics","Calisthenics skills without skeleton extraction","Depth beats skeleton in calisthenics recognition"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000216,"raw_usage":{"total_tokens":1441,"prompt_tokens":962,"completion_tokens":479,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":418}},"tokens_in":578,"tokens_out":479,"duration_ms":5192,"temperature":1.0,"reasoning_tokens":418,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:49:05.138073+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the depth-patch and RGB-patch pipelines and the skeleton baseline on a split that keeps all frames of each video together, then check whether depth patches still beat 0.815; if the margin disappears, the claimed superiority depends on the split rather than the method.","supporting_citations":[{"cited_title":"Proceedings Copyright (2024)","cited_arxiv_id":null,"evidence_quote":"Supplies the calisthenics dataset and the OpenPose-MLP skeleton baseline with 0.815 accuracy that the paper compares against."},{"cited_title":"Advances in Neural Information Processing Systems (2024)","cited_arxiv_id":null,"evidence_quote":"Provides Depth Anything V2, the monocular depth estimator used to produce the depth images and depth patches."},{"cited_title":"In: Interna- tional conference on machine learning","cited_arxiv_id":null,"evidence_quote":"Provides EfficientNetV2, the CNN classifier that carries the classification in all reported configurations."},{"cited_title":"International journal of computer vision (2015)","cited_arxiv_id":null,"evidence_quote":"Supplies ImageNet pretrained weights, which the paper uses for the best-performing classifier configurations."}],"review_version":1}