{"id":"1c0cd421-288f-4070-b2ed-f005d23a0de8","arxiv_id":"1908.08928","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A simple K-nearest-neighbors classifier using centered skeleton poses achieved the highest accuracy (83%) on CAD-60 in this head-to-head comparison with SVM and growing neural gas methods.","lead":"This paper compares four machine learning methods for recognizing human activities from skeleton joint data using the CAD-60 dataset. It finds that a simple nearest-neighbor classifier with centered skeletons gives the best accuracy in their tests, outperforming more complex neural gas networks.","discovery_kind":"replication","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed 83.02% 'global accuracy' is computed under two incompatible definitions (Section IV vs Appendix A); until Table I is recomputed with the pose-weighted formula, the KNN-vs-GWR ranking is not established.","rationale":"The reader's concern about manual hyperparameter tuning is real but partly disarmed by the paper's scoping: the conclusion says 'on our implementation', and the authors explicitly flag that tuning might improve GWR/GNG. The metric inconsistency is more fundamental because it affects the number that purportedly supports the claim under any tuning. Section IV's definition is a pose-weighted global accuracy; Appendix A's formula is an unweighted macro-average. For imbalanced CAD-60, these differ. The concrete test can settle it from the released code. I therefore keep the conditional verdict, but the condition should include recomputing Table I under the stated definition and reporting subject-level variability, not merely additional hyperparameter searches.","tokens_in":12540,"tokens_out":6190,"duration_ms":65803,"concrete_test":"Run the released MATLAB code (github.com/frederico-klein/cad-gas) to regenerate per-scene per-subject confusion matrices, then recompute the four methods under both definitions: (i) pose-weighted global accuracy as in Section IV; (ii) macro-average as in Appendix A. Compare the resulting Table I. The concern is settled if 1NN centred-mirrored remains above GWR normalised under both definitions by the same margin; if the margin collapses or ordering flips, the central claim fails. As a secondary check, compute a permutation or paired bootstrap over the four subjects for the 83.02 vs 79.00 difference.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table I is the sole quantitative support for the central claim. Section IV defines the reported 'overall global accuracy' as the sum of the traces of all scene/subject confusion matrices divided by the total number of poses. Appendix A instead defines the reported values as A_method = (1/S) * sum_s (1/P) * sum_p tr(C_{s,p}) / sum sum C_{s,p}, i.e. an unweighted macro-average over scenes and subjects. These two definitions coincide only if every scene-subject pair has the same number of poses, which is not true for CAD-60. If the appendix formula produced Table I, the headline 83.02% is not a global accuracy in the Section IV sense, and the true pose-weighted accuracy could be materially different. Since the conclusion is a comparison of overall accuracy, the metric must be unambiguous; a ranking based on a macro-average can change when recomputed as pose-weighted. This concern is independent of the acknowledged hyperparameter-tuning limitation and is checkable from the released code.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper compares four classifiers for human activity recognition from Kinect skeleton data on the CAD-60 dataset: SVM, k-nearest neighbors (KNN), and two hierarchical neural gas methods (GNG and GWR). Using a leave-one-subject-out protocol and a by-scene evaluation, the authors report global accuracies under three preprocessing conditions (none, centering and mirroring, and centering plus mirroring plus normalization). Their central empirical claim, stated in the conclusion, is that the simplest method—1NN with centered skeletons—achieves the best overall accuracy (83.02%) on their implementation, outperforming the more complex GWR and GNG approaches. The paper also discusses internal and external validity, acknowledges the limitations of manual hyperparameter tuning and the stochasticity of gas methods, and provides the MATLAB code as a reproducibility artifact.","tokens_in":12709,"tokens_out":2027,"duration_ms":22701,"significance":"If the comparison is valid, the finding that a simple 1NN baseline beats hierarchical neural gas classifiers on CAD-60 under a standard leave-one-subject-out protocol is a useful and potentially cautionary result for the HAR community, where complex learned representations are often assumed to be necessary. The paper's strengths include: (i) the use of a standard evaluation protocol (leave-one-subject-out, by-scene), (ii) explicit reporting of per-class precision/recall with standard deviations and honest discussion of failure cases, (iii) acknowledgment of the tuning asymmetry between methods rather than overselling the gas classifiers, and (iv) release of the code. The conclusion, however, rests entirely on the numeric ranking in Table I, and that ranking is currently undermined by an inconsistency in the definition of the reported accuracy metric and by the acknowledged non-exhaustive tuning of the gas methods. The result is potentially important but needs a more defensible quantitative basis before it can be accepted as stated.","major_comments":[{"comment":"The definition of the reported global accuracy is inconsistent and this is load-bearing for the central claim. Section IV states that the overall global accuracy is 'the sum of the traces of the confusion matrices of all scenes and all subjects divided by the total number of poses.' Appendix A instead defines A_method as a double average: first over the four subjects via Eq. (3), then over scenes via Eq. (4), with each scene-subject confusion matrix trace normalized by that scene-subject's pose count. These two formulas coincide only if every scene-subject pair contains the same number of poses, which is not the case in CAD-60. If Table I was produced with the Appendix A formula, the headline 83.02% is a macro-average, not the pose-weighted global accuracy described in Section IV, and the ranking in Table I could change when recomputed with the Section IV definition. The authors must recompute Table I with a single, clearly stated formula and report whether the KNN-vs-GWR ordering survives the correct pose-weighted computation.","section":"Section IV vs. Appendix A"},{"comment":"The comparison is not yet conclusive because the gas-based methods were not optimized to the same extent as the other classifiers, and the authors explicitly concede this. Section III-G states that GNG 'was not optimised, as the method its implementation is much slower than all others,' and Section IV-A acknowledges that 'we cannot guarantee for some of the models ... that the same algorithms with a few tweaks are not going to perform much better.' Since the paper's conclusion is that simpler methods are preferable to more complex ones, the burden is on showing that the GWR and GNG results are representative of those methods' capabilities. The authors should either (i) provide evidence that the chosen hyperparameters are near the achievable optimum (e.g., a small sensitivity study around the reported values), or (ii) soften the conclusion to a claim about 'these implementations with these settings' rather than about the methods in general.","section":"Section III-G and IV-A"},{"comment":"The global accuracies in Table I are reported without error bars, confidence intervals, or significance tests, and Table II shows extreme per-class variability (e.g., 55.47% ± 39.08% for drinking water in the bedroom). With only four test subjects, the 3.9-percentage-point gap between 1NN (83.02%) and the best GWR result (79.00%) under the same preprocessing may not be statistically meaningful. The conclusion 'the simple KNN ... gives results that are on our implementation the best accuracy overall' requires at least a paired test across the four folds (e.g., per-subject accuracies for 1NN vs. the best gas variant) to rule out that the ranking is driven by one subject or by stochastic gas training. Reporting per-subject global accuracies would be a minimal, feasible addition.","section":"Table I and Section V"}],"minor_comments":[{"comment":"The row 'Global average' in Table II appears to repeat the Bathroom 'Brushing teeth' precision value (94.42% ± 11.18%) instead of an actual global average over all scenes; this should be corrected.","section":"Table II"},{"comment":"There is a typo in Section III-C: 'the skeleton has its mid-sagittal plane passing the origin and its normal vector on X-axis direction' should probably read '... passing through the origin and with its normal vector along the X-axis direction.' The sentence is understandable but should be cleaned up.","section":"Section III-C"},{"comment":"The text refers to 'Fig. 1' (the combined confusion matrix) and describes it, but no figure content is present in the manuscript text; the figure must be included in the submission.","section":"Section IV"},{"comment":"The phrase 'an inclusion parameter aT = 1 and nodes = ∞ , the GNG and the GWR would degenerate into a KNN' is unclear without a definition of aT and nodes in that paragraph; a brief explanation of how these parameters control node insertion would help the reader.","section":"Section III-F"},{"comment":"Several references (e.g., [4], [6], [17], [31], [32]) lack complete bibliographic information such as year and page numbers; the reference list should be completed for reproducibility.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is honest and the code release is commendable, but the core quantitative claim currently rests on a metric-definition ambiguity that is checkable from the code and must be resolved. The tuning asymmetry is acknowledged by the authors themselves and is not a fabrication, but it does materially limit the strength of the conclusion. If the recomputed pose-weighted Table I still shows 1NN ahead, and the authors add a per-subject breakdown or significance check, I would support acceptance; otherwise the claim needs to be scaled back to a comparison of particular implementations under non-optimized settings."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a straightforward empirical comparison of KNN, SVM, GNG, and GWR on CAD-60 skeleton data, with code released. The result is plausible and the paper is honest about its limitations. But there is a load-bearing inconsistency in how the reported 'global accuracy' is computed, and until that is resolved the central claim should be treated as provisional.\n\nWhat's new: a head-to-head comparison under leave-one-subject-out, with several preprocessing variants, showing a simple 1NN with centered and mirrored skeletons beating the neural gas implementations in this setup. That is genuinely useful for practitioners who need a baseline. The paper ships code, which is real credit. The citation pattern is reasonable; citing their own earlier GWR work is fine given they are replicating that architecture.\n\nWhat's good: the authors use the standard evaluation protocol, report per-scene results, and openly discuss the manual hyperparameter search, the chaotic nature of gas methods, and the risk of cherry-picking. That candor is welcome.\n\nThe main problem: Section IV says the overall accuracy is the sum of traces of all scene/subject confusion matrices divided by total poses, which is pose-weighted. Appendix A gives a different formula: average over scenes of average over subjects of trace/sum. These coincide only if every scene-subject pair has the same number of poses, which CAD-60 does not satisfy. If Table I was produced with the Appendix formula, the 83.02% figure is a macro-average, and the true pose-weighted number could be materially different. The ranking between KNN and GWR could change. This is checkable from the released code, but as written the paper does not disambiguate.\n\nSecondary issues: no error bars or significance tests on the global accuracies; GNG was explicitly not optimized; the dataset is small (4 subjects, 12 actions); and the accuracies are below state of the art, though that is not a flaw per se.\n\nVerdict: the paper deserves a serious referee because the empirical comparison is reproducible and the negative result about neural gas methods is worth reporting. It needs revision: recompute Table I with a single clearly defined metric, report pose-weighted numbers, and ideally add significance tests or at least error bars. The authors' own caveat about hyperparameters is enough to soften the 'diminishing returns' conclusion; the metric issue is the one thing that could sink the ranking.","headline":"Useful, code-releasing baseline comparison on CAD-60, but the headline accuracy metric is ambiguous and the KNN-vs-gas ranking may not survive a pose-weighted recomputation.","tokens_in":13295,"tokens_out":2649,"would_cite":false,"duration_ms":24760,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A plain nearest-neighbour classifier on hip-centred mirrored skeleton poses beats three more complex activity-recognition methods in leave-one-subject-out tests.","keywords":["human activity recognition","skeleton poses","RGB-D","k-nearest neighbours","growing neural gas","growing when required neural gas","support vector machines","leave-one-subject-out"],"falsifier":"Run GNG and GWR on the same centred-and-mirrored skeleton features under leave-one-subject-out cross-validation with a systematic hyperparameter search over node counts, epochs, thresholds, and sliding-window lengths; if either model exceeds 83.02% global accuracy, the central claim that the simple KNN is the best method fails. A second check would be to repeat the comparison on a larger multi-subject data-set and see whether the KNN margin persists under real unseen-user conditions.","tokens_in":12288,"feed_emoji":"🦴","tokens_out":7443,"duration_ms":73807,"temperature":0.7,"pith_summary":"This paper asks which classifier is best for recognizing indoor activities from skeleton poses, with assistive robots for older adults as the motivation. It replicates a hierarchical growing-when-required neural gas architecture and compares it against support vector machines, k-nearest neighbours, and a growing neural gas on the CAD-60 data-set under leave-one-subject-out evaluation. The central finding is that the simplest method, 1-nearest-neighbour on hip-centred, mirrored skeletons, gives the highest overall accuracy at 83.02%, outperforming all gas-based and SVM alternatives. The paper concludes that complex methods show diminishing returns and that simple classifiers, possibly combined with object or sound cues, are a preferable starting point. If this holds, a robot caregiver could recognize daily activities with a classifier simple enough to update on every new pose.","feed_headline":"1NN on skeleton poses hits 83%, beating neural-gas models","feed_subtitle":"A replication finds hip-centred nearest-neighbour classification beats neural-gas and SVM baselines.","key_machinery":"The argument is carried by the preprocessing pipeline and the evaluation protocol rather than by a single theorem. Each skeleton is represented as a vector of $3J$ joint coordinates; centring the pose at the hips removes translation, and concatenating the pose with its mirror image across the X-axis doubles the data and encodes left-right symmetry. An optional further step normalises the skeleton by the neck-to-torso distance. For the gas classifiers, prototype poses are learned and then labelled with a 1-nearest-neighbour rule, while for KNN the full labelled training set is kept. Global accuracy is computed as the mean over scenes of the mean over subjects of the confusion-matrix trace divided by the number of poses, and the paper reports per-scene precision and recall for the winning classifier.","core_discovery":"The central discovery is a replication result: under leave-one-subject-out evaluation on the CAD-60 data-set, the 1-nearest-neighbour classifier applied to skeleton poses that are centred at the hips and mirrored across the sagittal plane reaches 83.02% global accuracy. The hierarchical growing-when-required neural gas reaches 79%, the growing neural gas reaches 78.37%, and the support vector machine reaches 67.27% under the same preconditioning, making the simplest method the most accurate. The paper interprets this as evidence of diminishing returns: the added machinery of gas-based prototype learning did not translate into better accuracy in this implementation, so simpler classifiers deserve priority in activity recognition for assistive robotics.","pith_inferences":["The mirroring step nearly doubles the training set, so an ablation without mirroring would show whether the gain comes from symmetry augmentation or from centring alone.","The KNN margin may depend on the dense 30 fps sampling of the data-set; testing with temporally subsampled poses would reveal whether single-frame nearest-neighbour still holds its advantage.","The simplicity argument could be tested on newer data-sets with more joints and larger subject pools, where the noisy 15-joint skeleton representation that KNN tolerates may behave differently."],"forward_implications":["A caregiver robot can recognize common indoor activities from a single current skeleton pose, with classification updates possible every 33 ms or faster.","The best classifier's accuracy drops to 80.36% when all twelve actions are considered together rather than restricted to a scene, so scene context is an important lever for performance.","Centring skeleton poses at the hips should be a default preprocessing step, since it improved every classifier in the comparison.","Gas-based classifiers did not justify their tuning cost and stochasticity on this data-set, making simpler pipelines a defensible first choice for similar skeleton-based activity recognition.","Per-subject variability remains large for some activities, so real deployments should be validated across new users rather than only on average accuracy."],"supporting_citations":[{"why":"Supplies the RGB-D data-set with four subjects and twelve actions whose skeleton poses are classified, and anchors the leave-one-subject-out evaluation protocol.","marker":"[18]"},{"why":"Defines the hierarchical growing-when-required neural gas architecture and the centring-and-mirroring preprocessing that the paper replicates as its main baseline.","marker":"[1]"},{"why":"Introduces the additional neck-to-torso normalisation step that the paper compares against centring and mirroring alone.","marker":"[7]"},{"why":"Provides the data-set's collection of state-of-the-art results from which the paper takes its reference accuracies and preconditioning strategies.","marker":"[19]"},{"why":"Earlier modular GWR implementation by the same group that grounds the paper's GWR implementation and expected performance.","marker":"[21]"},{"why":"Supplies the original growing-when-required neural gas algorithm description used to build the replicated GWR classifier.","marker":"[30]"},{"why":"Systematic comparison of supervised classifiers used to justify choosing KNN and SVM as the classic baselines.","marker":"[25]"}],"fun_headline_variants":["Simple 1NN beats neural gas on skeleton pose recognition","1NN on hip-centered poses hits 83%, tops neural gas","Hip-centered 1NN wins activity recognition at 83%","Skeleton pose study: 1NN beats complex neural gas models","Replication: 1NN bests neural-gas and SVM in action recognition"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's strongest result depends on the gas classifiers being tested with representative parameters, and the authors state that they tuned them manually and cannot rule out that better-tuned GNG or GWR models would perform much better.","fun_headline_variants_meta":{"raw":{"variants":["Simple 1NN beats neural gas on skeleton pose recognition","1NN on hip-centered poses hits 83%, tops neural gas","Hip-centered 1NN wins activity recognition at 83%","Skeleton pose study: 1NN beats complex neural gas models","Replication: 1NN bests neural-gas and SVM in action recognition"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000549,"raw_usage":{"total_tokens":2539,"prompt_tokens":779,"completion_tokens":1760,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":395,"completion_tokens_details":{"reasoning_tokens":1669}},"tokens_in":395,"tokens_out":1760,"duration_ms":11265,"temperature":1.0,"reasoning_tokens":1669,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:18:19.973767+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run GNG and GWR on the same centred-and-mirrored skeleton features under leave-one-subject-out cross-validation with a systematic hyperparameter search over node counts, epochs, thresholds, and sliding-window lengths; if either model exceeds 83.02% global accuracy, the central claim that the simple KNN is the best method fails. A second check would be to repeat the comparison on a larger multi-subject data-set and see whether the KNN margin persists under real unseen-user conditions.","supporting_citations":[{"cited_title":"Unstructured human activity detection from rgbd images,","cited_arxiv_id":null,"evidence_quote":"Supplies the RGB-D data-set with four subjects and twelve actions whose skeleton poses are classified, and anchors the leave-one-subject-out evaluation protocol."},{"cited_title":"A human activity recognition system using skeleton data from RGBD sensors,","cited_arxiv_id":null,"evidence_quote":"Introduces the additional neck-to-torso normalisation step that the paper compares against centring and mirroring alone."},{"cited_title":"Interactive telemedicine: effects on professional practice and health care outcomes,","cited_arxiv_id":null,"evidence_quote":"Provides the data-set's collection of state-of-the-art results from which the paper takes its reference accuracies and preconditioning strategies."},{"cited_title":"available at: http://pr.cs","cited_arxiv_id":null,"evidence_quote":"Earlier modular GWR implementation by the same group that grounds the paper's GWR implementation and expected performance."},{"cited_title":"A self-organising network that grows when required,","cited_arxiv_id":null,"evidence_quote":"Supplies the original growing-when-required neural gas algorithm description used to build the replicated GWR classifier."}],"review_version":1}