{"id":"050c302f-d0d8-4e23-bfe1-be4c3a139bac","arxiv_id":"2412.10573","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Using paired correct and incorrect exercise videos, ExeChecker's contrastively trained graph-attention model highlights the joints responsible for a wrong movement and beats a time-warping baseline in joint-level tests.","lead":"A computer vision system called ExeChecker watches videos of people doing rehabilitation exercises and points out which joints moved incorrectly. The authors show it can identify the right joints more often than an alignment-based baseline in initial tests, using a new paired dataset of correct and incorrect exercise performances.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The JoA metric in Eq. (5) ignores non-JoA joints, so Tables 2–3 do not actually demonstrate that ExeChecker localizes errors better than CTW.","rationale":"I considered the reader's weakest assumption, that the cross-spacetime attention map is a faithful proxy for which joints caused the error. That is a genuine concern and the paper does not validate it. However, I identified a more direct problem with the quantitative evidence: Eq. (5) defines a score that depends only on the attention received by true JoA joints, with no cost for highlighting unrelated joints. This is not a disagreement with consensus; it is a stated formula whose consequences are unambiguous. Even a model that gives high attention to every joint in the JoA set and equally high attention to every other joint scores 1.0. Thus the headline comparison in Tables 2 and 3 does not establish better localization. The qualitative figures show plausible examples, but they are not quantified. The proposed precision-aware metric would settle whether the reported advantage over CTW is real or an artifact. Because the reader's verdict was already CONDITIONAL and the reader listed the false-positive issue in the rationale, my read does not change the final verdict; it sharpens the specific condition under which the paper should be accepted. I do not claim the model is ineffective, only that the current evaluation cannot support the claim as written.","tokens_in":11269,"tokens_out":8308,"duration_ms":76993,"concrete_test":"Recompute the per-exercise results on ExeCheck and UI-PRMD using a precision-aware localization metric, e.g., for each test sequence rank joints by raw attention score and compute average precision or top-N intersection-over-union against the expert JoA set (with N = |JoA|), then average over test sequences. If ExeChecker no longer beats the CTW baseline on a majority of exercise-dataset pairs, the central comparison is an artifact of the metric.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central quantitative claim is that ExeChecker 'outperformed the baseline ... in identifying joints of physical relevance' (Abstract; Tables 2–3). The only quantification is S_JoA in Eq. (5): S_JoA = (1/N_e) * sum_i s_i * hat{s}_i, where s_i is 1 exactly for the expert-defined JoA joints and 0 otherwise. Non-JoA joints never enter the score. A model that highlights every true joint and also every false joint receives S_JoA = 1; a model that puts its highest attention on true joints but also high scores on many unrelated joints is not penalized. Min-max normalization amplifies this: if the maximum raw score happens to be on a true joint, that joint is normalized to 1 and all others to 0, regardless of how small the raw-score margin was. Consequently, Tables 2 and 3 measure something closer to recall of the JoA set than to localization accuracy. The 'outperforms CTW for 9 of 10 exercises' claim therefore does not logically follow, even if attention maps are perfectly faithful indicators of which joints produce an error.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces ExeChecker, a triplet-network framework built on a spatial-temporal graph attention transformer (STGAT) for rehabilitation exercise analysis. Given paired correct and incorrect exercise recordings, the model learns embeddings that separate the two classes and, at inference, uses the cross-spacetime attention map to score joints and highlight those most likely responsible for the error. The authors contribute the ExeCheck dataset of paired correct/incorrect RGB-D recordings for ten exercises with therapist-defined joint-of-attention (JoA) annotations, a JoA scoring metric (Eq. 5), and comparisons against a hop-adjusted canonical time warping (CTW) baseline on ExeCheck and UI-PRMD. They report that ExeChecker outperforms CTW on 9 of 10 ExeCheck exercises and on all but one UI-PRMD exercise.","tokens_in":11509,"tokens_out":5796,"duration_ms":53119,"significance":"If the quantitative claims held, ExeChecker would be a useful step toward interpretable, joint-level feedback in home rehabilitation, and the released dataset and annotations would be a valuable community resource. The design has real merits: JoA labels are not used during training, evaluation is performed on held-out subjects and on a second dataset, and the method is compared with a nontrivial alignment baseline. The paper also openly states limitations (restricted mistake types, planned user study). However, the evaluation metric and the experimental protocol currently do not support the central claim of superior joint localization; the significance is therefore conditional on a corrected evaluation.","major_comments":[{"comment":"The JoA score in Eq. (5) computes the average predicted score over only the expert-defined JoA joints; non-JoA joints enter the sum only with s_i=0, so false positives are never penalized. After min-max normalization, a model that assigns its maximum score to a single true JoA joint and arbitrarily high scores to all other joints receives S_JoA=1, regardless of how many irrelevant joints were highlighted. Tables 2 and 3 therefore measure recall of the JoA set rather than localization accuracy, and the abstract's claim that ExeChecker 'outperformed the baseline ... in identifying joints of physical relevance' is not supported by these numbers. Please re-evaluate with a metric that penalizes false positives (e.g., precision/recall at top-k or joint-level IoU) and report both components.","section":"Section 3.3, Eq. (5)"},{"comment":"All quantitative results are computed on a single held-out subject (Subject 7 for ExeCheck, Subject 10 for UI-PRMD) with one model run per configuration. Tables 2-3 report point estimates with no variance or significance tests, and several differences are small (e.g., ExeCheck 2-HKR 0.402 vs 0.356; UI-PRMD m02 0.454 vs 0.479, where CTW actually wins). Because the central claim is comparative, please report results across multiple train/test subject splits or repeated training runs, with confidence intervals or paired tests.","section":"Section 5.2, Tables 2 and 3"},{"comment":"The paper asserts without validation that the center-frame, head-averaged cross-spacetime attention map faithfully indicates which joints caused the incorrectness. If the attention map mainly captures general discriminative differences between the correct and incorrect sequences, the highlighted joints need not be the cause of the error, especially because the paired incorrect performances were instructed to differ at exactly the JoA joints. Please provide evidence for the faithfulness of the attention proxy, for example an ablation against another attribution method or a comparison of attention with kinematic error on held-out error types.","section":"Section 5.2, attention-to-joint mapping"}],"minor_comments":[{"comment":"Typos should be corrected: 'interpretablity' in the abstract and 'weaing' in Section 4.","section":"Abstract and Section 4"},{"comment":"The notation around Eq. (1) is unclear: please define sigma, W_h, X_tau, and the exact shape of A_tau, and fix the sentence 'A t τ is τ×N×N in dimension.'","section":"Section 3.1, Eq. (1)"},{"comment":"The min-max normalization is not defined for the case where all raw scores are equal; please specify how ties and zero-denominator cases are handled.","section":"Section 3.3"},{"comment":"The entry for 5-SA appears to read 'neck lShoulder' without a clear separator; please reformat the table so each joint is a distinct entry.","section":"Table 1"},{"comment":"Figures 3 and 4 are referenced but not included in the text, and exercises 6-9 are deferred to supplementary; please include all qualitative results or clearly mark them as supplementary-only.","section":"Section 5.4"},{"comment":"The concluding limitation statement is welcome, but the evaluation section should also discuss the construction confound: because participants were instructed to err at specific joints, the JoA labels coincide with the kinematic differences intentionally introduced.","section":"Section 6"}],"recommendation":"major_revision","confidential_remarks":"The core issue is the inappropriate evaluation metric; if the authors revise it to penalize false positives and add multi-split or repeated-run statistics, the paper could become acceptable. The dataset and code release plans are a useful contribution. No concerns about authorship or competing interests."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe thing to know: ExeChecker is a sensible application paper with a genuinely useful new dataset, but its main quantitative claim does not hold up as stated. The JoA metric in Eq. (5) only sums over the expert-defined joints, so it cannot distinguish a model that correctly localizes errors from one that simply assigns high attention everywhere. That makes Tables 2 and 3 a recall test, not a localization test.\n\nWhat's new and good: The ExeCheck dataset, 10 Parkinson's rehab exercises with paired correct/incorrect performances by 7 subjects and therapist-defined Joints of Attention, is a real contribution. The idea of using contrastive learning to get joint-level feedback is reasonable. The qualitative results are actually suggestive — the attention maps tend to light up on relevant joints and often catch subtle trunk or shoulder errors that CTW misses. The authors are also honest about the margin-loss instability and the bone-vector degradation.\n\nThe soft spots: First, the metric. Because non-JoA joints never enter the score, a model that highlights everything gets a perfect score as long as the max attention lands on a true joint. Min-max normalization then masks how flat the attention distribution is. So the \"outperforms CTW for 9 of 10 exercises\" claim is not supported. Second, the evaluation is on a single held-out subject per dataset, with no error bars or significance tests. That is thin evidence for a general claim. Third, the attention-to-joint mapping is simply assumed from prior work; the paper doesn't validate that the cross-spacetime attention at the center frame is a faithful indicator of which joints caused the error. Given the metric problem, that validation becomes even more important.\n\nNone of this kills the idea. The dataset and the qualitative evidence are worth having, and the paper is clearly written. But as it stands, the abstract overstates what has been demonstrated.\n\nThis paper deserves a serious referee — because of the dataset and the application, not because the current experiments are solid. The review should require a better metric that penalizes false positives, multi-subject evaluation with variance, and some check on the attention interpretation. The released data, when it appears, will make follow-up work easier.\n\nMy recommendation: send it to review, but expect a major revision.\n\nBest.","headline":"Useful dataset, but the JoA metric only measures recall, so the paper's central quantitative claim about joint localization is not supported.","tokens_in":12014,"tokens_out":2768,"would_cite":true,"duration_ms":26452,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ExeChecker claims that a triplet-trained graph attention transformer can identify which joints caused an incorrect rehabilitation exercise, by reading the model's cross-spacetime attention scores.","keywords":["movement assessment","contrastive learning","rehabilitation exercise","joint-level feedback","graph attention transformer","triplet loss","attention interpretability","skeleton-based pose"],"falsifier":"Take a correct exercise sequence and synthetically corrupt only one joint, such as the left knee, to create incorrect sequences; train ExeChecker and check whether the top-scoring joint is the corrupted one across many trials. Alternatively, mask the joints with the highest attention scores by replacing them with their correct positions and see whether the embedding still classifies the sequence as incorrect; if it does, the attention is not pointing at the cause.","tokens_in":11072,"feed_emoji":"🦴","tokens_out":7889,"duration_ms":68268,"temperature":0.7,"pith_summary":"ExeChecker claims that a contrastively trained graph attention transformer can tell a correct rehabilitation exercise from an incorrect one and, without per-joint error labels, point to the joints responsible for the mistake. The model is trained on triplets of paired correct and incorrect skeleton sequences, pushing correct movements together and correct-incorrect pairs apart in an embedding space. At inference, the network's cross-spacetime attention map, averaged over its eight heads and read from the central frame, is converted into per-joint scores that highlight the joints of attention. The paper introduces the ExeCheck dataset of ten therapist-selected exercises and a joint-of-attention scoring metric, and reports that ExeChecker beats the canonical-time-warping baseline on nine of ten ExeCheck exercises and on all but one UI-PRMD exercise. If the claim holds, home rehabilitation systems could replace vague feedback with concrete, joint-specific corrections.","feed_headline":"Triplet-trained AI points out the joints behind a wrong exercise","feed_subtitle":"Trained only on paired correct/incorrect videos, it beats time-warping baselines on most rehab exercises.","key_machinery":"The load-bearing object is the cross-spacetime attention map of a spatial-temporal graph attention transformer. At each timestamp, every joint attends not only to all joints in the current pose but also to all joints in $\\tau=3$ neighbouring poses, giving a $\\tau \\times N \\times N$ attention structure that is averaged over $H=8$ heads; for feedback, the map is read at the center frame and each joint's received attention is summed into a score. The other essential mechanism is the triplet ratio loss with in-triplet hard-negative mining and anchor swapping, which keeps the embedding training stable and forces the network to attend to the differences that separate correct from incorrect movement.","core_discovery":"The paper's central claim is that the cross-spacetime attention learned by a spatial-temporal graph attention transformer can localize the joints that cause an exercise to be incorrect, not merely separate correct from incorrect sequences. ExeChecker builds an embedding with a triplet ranking loss over correctly executed anchor and positive sequences and incorrectly executed negative sequences; because the same network that produces the embedding also emits attention over joints and neighbouring frames, the attention values can be read as per-joint importance. The authors define joints of attention for each exercise from therapist knowledge, average the eight attention heads, take the center frame of the sequence, and sum the attention each joint receives to obtain raw joint scores. On the collected ExeCheck dataset and on UI-PRMD, these scores match the therapist-defined joints of attention better than hop-adjusted canonical time warping on most exercises, and the qualitative examples show the model catching subtle trunk and shoulder errors that alignment-based scores miss.","pith_inferences":["One extension the paper leaves implicit: the per-frame attention could be aggregated over the time window where the error actually occurs instead of the single center frame, which may improve localization for exercises whose mistake happens early or late in the movement.","A direct stress test of the attention-as-explanation assumption would be to synthesize incorrect sequences by perturbing only one known joint; if ExeChecker's top-scoring joint tracks the perturbed joint across many such sequences, the attention map is causal rather than merely discriminative.","The same triplet recipe could be applied to other paired movement data such as sports technique or dance wherever a coach can record one correct and several flawed executions, turning expert demonstrations into weak supervision for joint-level feedback.","Because the embedding separates correct from incorrect sequences, the attention scores could also be used for continuous error grading, estimating how far a joint's motion is from the correct trajectory, which the paper does not attempt."],"forward_implications":["A home-rehabilitation system could give joint-specific feedback from a single RGB-D camera, using only paired correct/incorrect recordings and no per-joint labels at training time.","The joint-of-attention scoring metric gives a quantitative way to compare how well different methods localize movement errors, not just how well they classify them.","On both ExeCheck and UI-PRMD, attention-based scores beat hop-adjusted canonical time warping for most exercises, so attention interpretability is a usable alternative to pairwise temporal alignment.","The reprocessed UI-PRMD skeletons and new joint-of-attention annotations extend an existing public dataset to joint-level evaluation.","The method's sensitivity to subtle errors such as hunching shoulders or a wobbling trunk suggests it can flag the small, therapist-relevant mistakes that large pose differences do not expose."],"supporting_citations":[{"why":"Supplies the spatial-temporal graph attention transformer that generates the embeddings and the attention maps used for joint scoring.","marker":"[9]"},{"why":"Supplies the UI-PRMD dataset of paired correct and incorrect rehabilitation movements used for evaluation.","marker":"[25]"},{"why":"Supplies canonical time warping, the alignment-based baseline that ExeChecker is compared against.","marker":"[31]"},{"why":"Supplies the in-triplet hard-negative mining with anchor swap used to stabilize triplet training.","marker":"[1]"},{"why":"Supplies the triplet ratio loss that replaces the margin triplet loss after training failed to converge.","marker":"[8]"},{"why":"Supplies the off-the-shelf 3D pose estimator used to reprocess UI-PRMD RGB videos into cleaner skeleton sequences.","marker":"[30]"},{"why":"Supplies the original triplet loss formulation on which the contrastive training objective is based.","marker":"[26]"}],"fun_headline_variants":["AI pinpoints joints that make exercises wrong","Contrastive learning finds the joints behind exercise mistakes","ExeChecker: where exactly did your form go wrong?","Joint-level error locator for rehab exercises","Triplet loss AI identifies culprit joints in exercise"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The joint-level feedback stands on the assumption that the attention values, averaged over eight heads and read only from the center frame, identify the joints that caused the error rather than merely joints that differ between correct and incorrect videos.","fun_headline_variants_meta":{"raw":{"variants":["AI pinpoints joints that make exercises wrong","Contrastive learning finds the joints behind exercise mistakes","ExeChecker: where exactly did your form go wrong?","Joint-level error locator for rehab exercises","Triplet loss AI identifies culprit joints in exercise"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000317,"raw_usage":{"total_tokens":1763,"prompt_tokens":883,"completion_tokens":880,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":807}},"tokens_in":499,"tokens_out":880,"duration_ms":6694,"temperature":1.0,"reasoning_tokens":807,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:50:39.752667+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a correct exercise sequence and synthetically corrupt only one joint, such as the left knee, to create incorrect sequences; train ExeChecker and check whether the top-scoring joint is the corrupted one across many trials. Alternatively, mask the joints with the highest attention scores by replacing them with their correct positions and see whether the embedding still classifies the sequence as incorrect; if it does, the attention is not pointing at the cause.","supporting_citations":[{"cited_title":"In: Similarity- based pattern recognition: Third International Workshop, SIMBAD 2015, Copen- hagen, Denmark, October 12-14, 2015","cited_arxiv_id":null,"evidence_quote":"Supplies the triplet ratio loss that replaces the margin triplet loss after training failed to converge."},{"cited_title":"Expert Systems with Applications232, 120683 (2023).https://doi.org/10.1016/j.eswa.2023.120683","cited_arxiv_id":null,"evidence_quote":"Supplies the spatial-temporal graph attention transformer that generates the embeddings and the attention maps used for joint scoring."},{"cited_title":"Data3(1) (2018).https://doi.org/10.3390/ data3010002, 15 pages","cited_arxiv_id":null,"evidence_quote":"Supplies the UI-PRMD dataset of paired correct and incorrect rehabilitation movements used for evaluation."},{"cited_title":"Advances in Neural Information Processing Systems (NIPS)22(2009), 9 pages","cited_arxiv_id":null,"evidence_quote":"Supplies canonical time warping, the alignment-based baseline that ExeChecker is compared against."},{"cited_title":"In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2014)","cited_arxiv_id":null,"evidence_quote":"Supplies the original triplet loss formulation on which the contrastive training objective is based."}],"review_version":1}