{"id":"248f4974-ab6e-42e1-b8ab-17a78de6b3a1","arxiv_id":"2411.15655","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A benchmark of existing EMG feature descriptors and classifiers, reporting 97% top accuracy on Grabmyo and 94.95% on FORS-EMG.","lead":"This paper tests combinations of muscle-signal features and machine learning models for recognizing hand gestures. It reports 97 percent accuracy for a dilated convolutional network on one dataset and 94.95 percent for a random forest on another, which could guide prosthetic and human-computer interface design.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation-split ambiguity (Section IV-B) is the load-bearing risk: an 80/20 window-level split would leak temporally overlapping windows and inflate all reported accuracies.","rationale":"This is a straightforward empirical benchmark paper, and its central claim is only as strong as its evaluation protocol. The reader's weakest assumption correctly identifies the 80/20 split as the load-bearing premise: no subject-level or session-level separation is stated, and the 50% overlapping-window design makes random window splitting especially dangerous. My independent reading of Section III and Section IV-B confirms that the split description is the single point on which the headline accuracies depend. I considered whether other issues, such as the 'novel' feature descriptors being cited to Khushaba et al. or the apparent text/table inconsistency in the FORS-EMG KNN description, are more central; they are not. Low novelty and minor prose errors do not invalidate an empirical comparison, whereas an invalid split would invalidate the entire quantitative contribution. The paper does have some independent support: it uses two public datasets and reports full 10-model comparison tables, which are checkable. It also cites specific prior feature-extraction sources, which is helpful for reproducibility. However, no code is released and no variance is reported, so the only way to resolve the main threat is to clarify or re-run the evaluation. The reader's CONDITIONAL verdict is therefore appropriate and I do not recommend moving it: the condition is exactly that the split protocol be disclosed and, if needed, corrected to avoid leakage. My concrete test is deliberately narrow: leave-one-subject-out with non-overlapping windows would settle whether the reported rankings reflect generalization or memorization.","tokens_in":12770,"tokens_out":3419,"duration_ms":33742,"concrete_test":"Obtain the exact split procedure or re-run the experiments with leave-one-subject-out cross-validation, ensuring that no training window overlaps in time with any test window within the same session. Reproduce the Grabmyo fTDD + 1D Dilated CNN condition and the FORS-EMG TSD + Random Forest condition from Tables III and VII under this protocol, reporting mean accuracy and standard deviation over subjects. If the 97% and 94.95% figures drop materially, or if the best-model ranking changes, the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is an empirical ranking: 97% with 1D Dilated CNN plus fused time-domain descriptors on Grabmyo, and 94.95% with Random Forest plus temporal-spatial descriptors on FORS-EMG. For this claim to stand, the 80/20 split must measure generalization to unseen recordings, not memorization of near-duplicate windows. Section IV-B states only that 'The data sets were divided into 80% for training and 20% for testing,' without specifying subject-level, session-level, or non-overlap constraints. This is decisive because Section III uses 600 ms windows with 50% overlap, so consecutive windows share 300 ms of raw signal. Grabmyo has 43 participants recorded on days 1, 8, and 29, while FORS-EMG has 19 participants with five repetitions per gesture. A random window-level split can place overlapping windows from the same contraction in both training and test sets, allowing the model to interpolate the exact signal rather than learn gesture-generalizable features. This would inflate every accuracy in Tables III-VIII, especially the deep models. The paper also reports no error bars or repeated runs, so the ranking is fragile even apart from leakage: Table VII shows Random Forest and AdaBoost both at 94.95%, a difference of 0.00 that is not statistically interpretable. If the split is actually subject-independent and temporally non-overlapping, the main threat disappears; as written, the evaluation protocol is underspecified and the headline generalization claim is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper benchmarks three feature extraction approaches (fused time-domain descriptors, temporal-spatial descriptors, and wavelet transform-based features) combined with several machine learning classifiers (LDA, SVM, KNN, random forest, voting/bagging/boosting ensembles) and two deep models (1D dilated CNN and 1D dilated CNN-LSTM) for sEMG-based hand gesture recognition on two recent public datasets, Grabmyo and FORS-EMG. The headline results are 97% accuracy for the 1D dilated CNN with fused time-domain descriptors on Grabmyo and 94.95% accuracy for random forest with temporal-spatial descriptors on FORS-EMG; the authors conclude these are the best combinations among those compared. The evaluation uses 600 ms windows with 50% overlap, an 80/20 train/test split, and standard metrics reported on the test set.","tokens_in":13015,"tokens_out":2714,"duration_ms":26098,"significance":"If the empirical ranking is valid, this is a useful benchmark for practitioners selecting feature/classifier combinations for myoelectric control, especially because it uses two recent (2024) public datasets and covers a wider model family than most prior comparisons. The paper also reports precision, recall, and F1 alongside accuracy, which is valuable for class-imbalance assessment. However, the central claim is a purely empirical ranking and is only as strong as the evaluation protocol; the current manuscript does not provide enough detail on how the train/test split was constructed to establish that the reported accuracies reflect generalization to unseen subjects or even unseen recordings.","major_comments":[{"comment":"The evaluation protocol is under-specified in a way that is load-bearing for every reported accuracy. Section IV-B states only that 'The data sets were divided into 80% for training and 20% for testing' without saying whether the split respects subject, session, or recording boundaries. Since Section III uses 600 ms windows with 50% overlap, a random window-level split can place highly overlapping or near-duplicate windows from the same contraction in both training and test sets, allowing the model to memorize the signal rather than learn gesture-generalizable features. This concern is amplified by the structure of both datasets: Grabmyo has 43 subjects recorded on days 1, 8, and 29, and FORS-EMG has 19 subjects with five repetitions per gesture. Please clarify whether the split was subject-independent and temporally non-overlapping; if it was not, the experiments should be rerun with a subject-independent split, and the abstract/conclusions should be revised accordingly.","section":"Section IV-B"},{"comment":"No variance or uncertainty is reported for any of the accuracy numbers, and no repeated runs or statistical tests are described. This is particularly important because many of the reported differences are very small: in Table VII, Random Forest and AdaBoost both report 94.95% accuracy, yet the text and abstract single out Random Forest as the best. Without confidence intervals, paired significance tests, or multiple-seed statistics, the claimed ranking is not statistically grounded. Please report mean plus/minus standard deviation over repeated runs (or an equivalent uncertainty measure) and, where the top accuracies are close, a paired test.","section":"Tables III-VIII and Section V"},{"comment":"The results narrative in Section V-B is internally inconsistent and should be corrected. The text says 'Among traditional machine learning models, K-Nearest Neighbors (KNN) obtained the highest accuracy of 89.05%,' but Table VII reports KNN at 89.9% and Random Forest at 94.95%, so KNN is not the highest-accuracy traditional model and 89.05 does not match the table. Since the central claim depends on identifying the best-performing model, this inconsistency between the text and the table needs to be resolved, even though the table itself appears to support the Random Forest claim.","section":"Section V-B, Table VII"}],"minor_comments":[{"comment":"The comparison with existing work [39] is not apples-to-apples: the text states that [39] uses only five basic hand gestures, while the present work uses all gesture classes, in addition to different features and classifiers. The '6.96% improvement' conclusion is therefore not a controlled comparison and should be phrased as indicative only.","section":"Section V-A"},{"comment":"Table I lists features such as COV and TKEO as part of the fTDD table, but the text attributes those to TSD; the table header says 'SUMMARY OF FTDD AND TSD FEATURES,' which is confusing. Please separate or clearly label which features belong to fTDD and which to TSD.","section":"Section III-A and Table I"},{"comment":"The paper does not report the total number of windows, the number of windows per class, or the class balance after feature extraction. This information would help the reader assess whether the high accuracies could be driven by class imbalance or by a very large overlap between adjacent windows.","section":"Section IV-B"},{"comment":"In the third paragraph of Section V-B, the sentence 'KNN showed substantial improvements. Specifically, compared to LDA (63.3%), KNN improved accuracy by 40.66%' appears to use 89.05 as the KNN value even though the table gives 89.9. Please keep the text and tables consistent.","section":"Section V-B"}],"recommendation":"major_revision","confidential_remarks":"The central empirical claim is potentially useful, but the manuscript currently lacks the evaluation-protocol detail needed to rule out leakage from overlapping windows and the statistical support needed to distinguish top-ranked models. I do not see a circularity or novelty-disclosure issue; the main work is documentation and the provision of a subject-independent evaluation. If the authors can provide the split details (or rerun with a subject-independent split) and add uncertainty estimates, the paper would be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a straightforward empirical benchmark, not a methods paper. The new bits are accuracy numbers on two recent public datasets (Grabmyo, FORS-EMG); the feature descriptors are not new—they are Khushaba's fTDD and TSD, plus standard wavelet features, and the paper says so in Section III. The authors run a wide set of classifiers across three feature families and report accuracy/precision/recall/F1 on both datasets. That is useful work for practitioners who want a feature/model comparison on these particular datasets.\n\nThe soft spot is the evaluation split. Section IV-B says only that data were divided 80/20 for training/testing. With 600 ms windows at 50% overlap, a random split at window level would put temporally overlapping windows from the same contraction into both train and test. Grabmyo is 43 participants across days 1/8/29; FORS-EMG has 19 participants with five repetitions per gesture. If the split is not subject- or session-disjoint, the reported 97% and 94.95% reflect memorization of near-duplicate windows, not generalization. The paper does not state the split strategy, so the main claim is not established as written. This is fixable—state the split, repeat with subject-independent partitions, report means and standard deviations over multiple runs.\n\nOther issues are smaller. No code or data released. The abstract and contributions call the feature sets 'novel' when they are cited to Khushaba et al.; that should be corrected. There are small internal inconsistencies, e.g., the FORS-EMG TSD text says KNN got 89.05% while Table VII shows 89.9%, and Table VII shows Random Forest and AdaBoost both at 94.95%, a tie that is meaningless without variance. No error bars anywhere.\n\nI would not take the headline numbers at face value yet, but the paper is worth refereeing: the question it asks is practical, the datasets are public and recent, and the evaluation-protocol concern is addressable in revision rather than fatal. A serious referee should ask for a precise description of the split, subject-independent results, and code/data release.","headline":"Useful benchmark on two recent sEMG datasets, but the underspecified 80/20 split makes the headline accuracies unverified.","tokens_in":13622,"tokens_out":1721,"would_cite":false,"duration_ms":15720,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"97% hand-gesture accuracy from sEMG with dilated CNN","keywords":["sEMG signal classification","hand gesture recognition","fused time-domain descriptors","temporal-spatial descriptors","wavelet transform features","1D dilated CNN","random forest","myoelectric control"],"falsifier":"Re-run the same feature and classifier grid using a person-disjoint split, and for Grabmyo a session-disjoint split, with all windows from any given participant kept out of training when that participant is in the test set. If the dilated CNN's 97% and the random forest's 94.95% fall substantially under such held-out-user evaluation, the paper's central claim about best configurations does not generalize.","tokens_in":12494,"feed_emoji":"🖐️","tokens_out":8000,"duration_ms":66412,"temperature":0.7,"pith_summary":"Surface electromyographic (sEMG) signals record the electrical activity of muscles, and classifiers trained on them can translate forearm muscle patterns into hand gestures for prosthetic and wearable interfaces. This paper benchmarks three feature-extraction families—fused time-domain descriptors, temporal-spatial descriptors, and wavelet-transform features—against ten machine and deep-learning classifiers on two public datasets. Its central claim is that the best tested configuration is a 1D dilated CNN with fused time-domain descriptors, reaching 97% accuracy on the 16-gesture Grabmyo dataset, and a random forest with temporal-spatial descriptors, reaching 94.95% accuracy on the FORS-EMG dataset. These numbers beat the compared SVM baseline by about 4 and 31 percentage points respectively, so the paper is proposing concrete recipe choices for sEMG gesture control.","feed_headline":"97% hand-gesture accuracy from sEMG with dilated CNN","feed_subtitle":"Best model-plus-feature pairs for prosthetic and wearable gesture control.","key_machinery":"The load-bearing machinery is the sliding-window feature cascade. Band-pass filtered sEMG (20–450 Hz) is cut into 600 ms windows with 50% overlap, and each window is reduced to one of three descriptor sets: fused time-domain descriptors (fTDD), temporal-spatial descriptors (TSD), or five-level Symlet-8 wavelet features. fTDD combines log-transformed spectral moments with sparsity, irregularity factor, and waveform length ratio; TSD adds coefficient of variation and Teager-Kaiser energy to within-channel and pairwise between-channel features. These descriptors are then fed to the classifier grid, where the 1D dilated CNN uses increasing dilation rates to see multi-scale temporal patterns, and the random forest builds a hundred-tree ensemble. The winner on each dataset is the feature-classifier pair that best exploits the temporal or spatial structure in that dataset's signals.","core_discovery":"On the paper's own terms, the discovery is empirical: the configuration that wins is different on each dataset. For Grabmyo, a 1D dilated CNN fed with fused time-domain descriptors—power spectral moments, sparsity, irregularity factor, and waveform length ratio computed from 600 ms windows with 50% overlap—achieves 97% accuracy with 0.97 precision, recall, and F1, outperforming every classic model and also edging out the dilated CNN-LSTM at 96.53%. For FORS-EMG, a random forest using temporal-spatial descriptors—time-domain features supplemented by coefficient of variation, Teager-Kaiser energy, and pairwise between-channel statistics—achieves 94.95%, tied by AdaBoost, and clearly above the deep models on that dataset. The paper positions this as a benchmark that compares recent 2024 datasets with the same feature and classifier pipeline, and it reports the winning pairs as the practical choices among the tested alternatives.","pith_inferences":["If the reported evaluation is taken at face value, the practical takeaway is that feature engineering still matters: a deliberately designed descriptor set can let a classical tree ensemble match or beat deep networks, as the random forest does on FORS-EMG.","A subject-disjoint or session-disjoint split would be a stricter test of generalization to new users; because the paper describes only an 80/20 split, its accuracy numbers may overstate cross-user performance.","The 50% window overlap means adjacent training windows share muscle activity; testing with non-overlapping windows would separate true gesture discrimination from temporal smoothing."],"forward_implications":["On Grabmyo, fused time-domain descriptors plus the 1D dilated CNN give 97% accuracy, beating SVM by 4.17 points and LDA by 15.13 points on the same feature set.","On FORS-EMG, temporal-spatial descriptors plus random forest reach 94.95%, beating SVM by 30.74 points and LDA by 49.97 points.","Ensemble variants—voting, bagging, and AdaBoost—consistently sit at or above their single-model counterparts, which is most pronounced on the smaller FORS-EMG dataset.","Wavelet-transform features are competitive, reaching roughly 94–96% depending on classifier, but never surpass the winning fTDD and TSD combinations in this comparison."],"supporting_citations":[{"why":"Supplies the Grabmyo multi-day sEMG dataset from 43 participants used for the 97% result.","marker":"[24]"},{"why":"Supplies the FORS-EMG sEMG dataset from 19 participants used for the 94.95% result.","marker":"[25]"},{"why":"Defines the fused time-domain descriptors and the log-transform/correlation feature construction used by the winning Grabmyo model.","marker":"[29]"},{"why":"Defines the temporal-spatial descriptor framework, including within-channel and pairwise between-channel features used by the winning FORS-EMG model.","marker":"[30]"},{"why":"Supplies the wavelet feature extraction routine that the paper adapts for its wavelet-transform baseline.","marker":"[31]"},{"why":"Provides the prior Grabmyo result, 90.69% on five gestures with LDA/SVM, that the paper compares its 97% result against.","marker":"[39]"}],"fun_headline_variants":["sEMG hand gesture recognition: dilated CNN hits 97%","Dataset-dependent winners: CNN for Grabmyo, forest for FORS-EMG","Machine learning on sEMG: 97% for gestures via dilated CNN","Benchmarking sEMG gesture classifiers: CNN and forest lead"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the 80/20 train/test split keeps each person's recordings entirely on one side; the paper does not say that it separates by subject or session, so the reported accuracies could reflect memorizing individual muscle signatures rather than recognizing gestures for new users.","fun_headline_variants_meta":{"raw":{"variants":["sEMG hand gesture recognition: dilated CNN hits 97%","Dataset-dependent winners: CNN for Grabmyo, forest for FORS-EMG","Machine learning on sEMG: 97% for gestures via dilated CNN","Benchmarking sEMG gesture classifiers: CNN and forest lead"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000976,"raw_usage":{"total_tokens":4154,"prompt_tokens":963,"completion_tokens":3191,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":3110}},"tokens_in":579,"tokens_out":3191,"duration_ms":20060,"temperature":1.0,"reasoning_tokens":3110,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:03:48.888509+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same feature and classifier grid using a person-disjoint split, and for Grabmyo a session-disjoint split, with all windows from any given participant kept out of training when that participant is in the test set. If the dilated CNN's 97% and the random forest's 94.95% fall substantially under such held-out-user evaluation, the paper's central claim about best configurations does not generalize.","supporting_citations":[{"cited_title":"Multi-day dataset of forearm and wrist electromyogram for hand gesture recognition and biometrics,","cited_arxiv_id":null,"evidence_quote":"Supplies the Grabmyo multi-day sEMG dataset from 43 participants used for the 97% result."},{"cited_title":"FORS-EMG: A Novel sEMG Dataset for Hand Gesture Recognition Across Multiple Forearm Orientations","cited_arxiv_id":"2409.07484","evidence_quote":"Supplies the FORS-EMG sEMG dataset from 19 participants used for the 94.95% result."},{"cited_title":"A fusion of time-domain descriptors for improved myoelectric hand control,","cited_arxiv_id":null,"evidence_quote":"Defines the fused time-domain descriptors and the log-transform/correlation feature construction used by the winning Grabmyo model."},{"cited_title":"A framework of temporal-spatial descriptors-based feature extraction for improved myoelectric pattern recognition,","cited_arxiv_id":null,"evidence_quote":"Defines the temporal-spatial descriptor framework, including within-channel and pairwise between-channel features used by the winning FORS-EMG model."},{"cited_title":"getmswtfeat,","cited_arxiv_id":null,"evidence_quote":"Supplies the wavelet feature extraction routine that the paper adapts for its wavelet-transform baseline."},{"cited_title":"Machine learning-based feature extraction and classification of emg signals for intuitive prosthetic control,","cited_arxiv_id":null,"evidence_quote":"Provides the prior Grabmyo result, 90.69% on five gestures with LDA/SVM, that the paper compares its 97% result against."}],"review_version":1}