{"id":"1b101634-0b1e-4879-9a72-e41f36727e88","arxiv_id":"2504.14708","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"XMANet, a cross-layer mutual attention network originally designed for image classification, is applied to EMG spectrograms and scalograms, yielding small and inconsistently reported accuracy improvements over baseline CNNs.","lead":"This paper applies a fine-grained attention network called XMANet to recognize hand gestures from forearm muscle signals. It reports small accuracy gains over standard CNNs on two EMG datasets, but the headline numbers in the abstract do not match the tables in the paper.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Without error bars or repeated runs, the claimed 0.2–5% gains are indistinguishable from training noise, and the paper's own FORS STFT table already contains a negative result; 'consistent improvement' is therefore unverified.","rationale":"The reader's weakest assumption is exactly the main vulnerability: a single validation-loss-selected checkpoint is treated as a reliable estimator of test accuracy, and the reported differences are small enough to be explained by training noise. I agree that this is load-bearing. My independent reading also found that the abstract's percentage format (relative improvements) is used inconsistently with some in-text absolute values, and that Section 5.2 does not state whether the 70/15/15 split is subject/session-disjoint despite the use of 50%-overlapping windows; both deserve clarification. However, the single-run evaluation is the condition that, if unmet, makes the central claim unsupported even for the exact numbers printed. The proposed multi-seed rerun with paired comparisons directly determines whether the reported differences are systematic, so the reader's CONDITIONAL verdict remains appropriate.","tokens_in":15105,"tokens_out":10192,"duration_ms":96288,"concrete_test":"Ask the authors to release training code and random seeds, then rerun every condition in Tables 1–6 with at least 5 independent seeds and report mean ± std test accuracy plus a paired significance test (e.g., Wilcoxon signed-rank over the 5 runs). Pay specific attention to FORS-EMG STFT EfficientNetB0 (48.08 vs 48.18). If the negative result persists, or if most Grabmyo deltas are within one standard deviation, the consistent-improvement claim fails; if the positive direction is stable and significant, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.2 reports that the best model was selected by validation loss and evaluated once on the test set; no seeds, repeated runs, confidence intervals, or significance tests are given for any table. All headline improvements in Section 6 are therefore point estimates from a single training run. This matters because the differences are small: on Grabmyo the matched gains are roughly 1.4–5.1% relative at 90–99% accuracy (Tables 2–3), and Table 1 contains deltas of 0.03–0.49 percentage points. On FORS-EMG STFT, Table 5 already contains a counterexample to the word 'consistently': XMANet(EfficientNetB0) is 48.08% versus 48.18% for EfficientNetB0. With no estimate of run-to-run variance, the observed pattern of small positive differences could be produced by initialization and stochastic training rather than by the mutual-attention mechanism. The central claim that XMANet consistently improves performance is thus not yet supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes XMANet, a fine-grained classification model that applies cross-layer mutual attention among CNN experts, for EMG-based hand gesture recognition. Raw EMG is converted into stacked STFT spectrograms and wavelet scalograms, which are then classified by XMANet built on top of ResNet50, DenseNet121, MobileNetV3, and EfficientNetB0 backbones. Experiments on the Grabmyo and FORS-EMG datasets report accuracy, precision, recall, and F1, and the abstract and conclusion claim that XMANet consistently improves performance over all baselines on both time-frequency representations.","tokens_in":15307,"tokens_out":5272,"duration_ms":45540,"significance":"If the central claim were supported, the paper would provide a simple, architecture-agnostic way to improve EMG gesture recognition from time-frequency images, and the systematic comparison of STFT versus wavelet representations across four backbones on two public datasets would be a useful empirical contribution. The paper is transparent in reporting per-model accuracies and uses standard preprocessing and evaluation protocols. However, the methodological novelty is limited because XMANet is essentially the cross-layer mutual attention learning method of Liu et al. [24] applied to EMG spectrograms and scalograms; the contribution is empirical rather than architectural. The reliability of the empirical claim is the main issue: the evidence consists of single training runs without error bars or significance tests, and the paper contains a negative result in Table 5 that is not acknowledged. If the experimental protocol is strengthened, the study could be a valid application-oriented contribution.","major_comments":[{"comment":"The headline improvement percentages in the abstract and conclusion (e.g., 1.72%, 4.38%, 5.10%, 2.53% for STFT on Grabmyo) are relative percentage increases, not absolute accuracy gains. For example, XMANet(ResNet50) at 93.03% versus ResNet50 at 91.46% is a gain of 1.57 percentage points, which is a 1.72% relative improvement. Elsewhere in Section 6.1 (Table 1 discussion) the text uses absolute percentage points, e.g., 'improved accuracy by 0.424% relative.' The paper never defines which quantity is being reported, so the quantitative claims in the abstract and conclusion are ambiguous and the two usages are inconsistent.","section":"Abstract, Section 6.1, Tables 2 and 3"},{"comment":"All results come from a single training run per configuration: the model with the best validation loss is saved and evaluated once on the test set, with no seeds, repeated runs, confidence intervals, or statistical significance tests reported. Many of the claimed improvements are very small (e.g., 0.03 percentage points in Table 1 and 0.21 percentage points in Table 5), and without an estimate of run-to-run variability these differences cannot be distinguished from training noise. Since the paper's central claim is that XMANet consistently improves accuracy, this lack of statistical support is load-bearing.","section":"Section 5.2 and Section 6"},{"comment":"The claim that XMANet 'consistently improves performance' is contradicted by the paper's own results in Table 5: on FORS-EMG STFT, XMANet(EfficientNetB0) obtains 48.08% accuracy versus 48.18% for the EfficientNetB0 baseline, a decrease. The surrounding text does not acknowledge this negative result, and the summary in Section 6.2 reports only the positive gains. The claim of consistency must either be removed or qualified, and the negative case should be explained or shown to be within the run-to-run variation.","section":"Table 5, Section 6.2"}],"minor_comments":[{"comment":"The sentence listing the four pre-trained architectures says 'ResNet50, MobileNetV3, and DenseNet121' but omits EfficientNetB0, which is nevertheless used in all experiments. Also, the text says 'DenseNet121 model achieves an accuracy of 98.44%' while Table 1 reports 99.44%.","section":"Section 5.2, Section 6.1"},{"comment":"The min-max normalization formula is written as '(α,β) = ˜Ωn(α,β) - min( ˜Ωn)/max( ˜Ωn) - min( ˜Ωn)', which is missing parentheses and is not mathematically well-formed; it should be (˜Ωn - min(˜Ωn)) / (max(˜Ωn) - min(˜Ωn)).","section":"Equation (11)"},{"comment":"The description of Grabmyo does not state how many EMG channels are used, while FORS-EMG is described as eight channels; channel count is important for reproducing the stacked spectrogram and scalogram images.","section":"Section 4, Section 5.1"},{"comment":"The list of improvements in the text mixes comparisons against different baselines without a consistent convention; for instance, 'the proposed XMANet(ResNet50) model outperforms the baseline ResNet50, EfficientNetB0, MobileNetV3, and DenseNet121 models with improvements of approximately 5.04%, 15.44%, 34.73%, and 2.53%' is confusing because each number is relative to a different baseline. A single table of baseline-vs-proposed pairwise differences would improve readability.","section":"Section 6.2"},{"comment":"The reference to 'Zhao et al. [17]' appears to mismatch the cited paper by W. Zhang and J. Zhang; the authors should verify all reference-author name alignments.","section":"Section 2.2"}],"recommendation":"major_revision","confidential_remarks":"The stress-test note from the reader is on point: the single-run evaluation and the unacknowledged negative result in Table 5 are central weaknesses, not peripheral issues. The paper's method is a direct application of [24] to EMG time-frequency images, so the empirical validation must be strong enough to justify the contribution. Adding repeated runs with seeds, reporting confidence intervals or significance tests, harmonizing the percentage metric, and addressing the EfficientNetB0 STFT result on FORS-EMG are necessary changes. I do not see evidence of circularity or unsupported invention; the issues are about experimental evidence and reporting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is an applied benchmark paper that takes the XMANet architecture from Liu et al. 2023 and runs it on STFT spectrograms and CWT scalograms of EMG data. The combination is new, the datasets are public, and the comparisons are standard. But the paper oversells its results. The headline improvements are mostly small, the abstract numbers don't match the tables, and there are no repeated runs or significance tests. One variant actually underperforms its baseline, which contradicts the word 'consistently.'\n\nWhat's genuinely useful: the authors test four backbones (ResNet50, DenseNet121, MobileNetV3, EfficientNetB0) on two distinct EMG datasets and report accuracy/precision/recall/F1. If reproducible, this gives the community reference numbers for a known architecture on these tasks. The preprocessing pipeline (0.6s windows, 50% overlap, stacked spectrograms/scalograms) is described in enough detail to follow.\n\nThe soft spots are real. First, the numerical inconsistencies: the abstract claims a 1.72% gain for XMANet(ResNet50) on Grabmyo STFT, but Table 2 shows 93.03 vs 91.46 = 1.57. Similar mismatches appear elsewhere. Second, there is no estimate of run-to-run variance. The best model is selected by validation loss and evaluated once; differences of 0.2–2 points are exactly the kind that vanish across seeds. Third, Table 5 has a negative result: XMANet(EfficientNetB0) at 48.08% against a 48.18% baseline. That is not fatal by itself, but it kills the 'always improves' phrasing. Fourth, the method is adopted, not invented. The authors cite Liu et al. [24], but the abstract and contribution list present XMANet as their own without explicitly saying it is a direct application. That needs to be fixed.\n\nOn balance, the empirical direction is plausible and the larger gains on FORS-EMG (e.g., DenseNet121 on wavelet: 60.49 vs 55.31) deserve a closer look. But the paper as written does not establish reliable gains.\n\nRecommendation: send to peer review with a major-revision bar. The work is sufficiently substantive for referee time, and the issues are reparable with multi-seed results, corrected numbers, and honest attribution. I would not cite it in its current form.","headline":"A thin but honest application of an existing fine-grained attention network to EMG time-frequency images; the central claim of consistent gains is not yet supported by the evidence.","tokens_in":15885,"tokens_out":3277,"would_cite":false,"duration_ms":27678,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Mutual-attention layers consistently improve EMG gesture models","keywords":["Electromyography","EMG gesture recognition","Fine-grained classification","Cross-layer mutual attention","Time-frequency analysis","Short-Time Fourier Transform","Wavelet scalogram","CNN ensembles"],"falsifier":"Re-run each baseline and each XMANet variant on the same train/validation/test splits with multiple random seeds (e.g., 10 seeds) and compute the mean and standard deviation of test accuracy; if the accuracy differences between XMANet and its corresponding baseline shrink to within one standard deviation across seeds, the central claim of consistent improvement collapses.","tokens_in":14872,"feed_emoji":"🖐️","tokens_out":2526,"duration_ms":24507,"temperature":0.7,"pith_summary":"This paper tries to establish that fine-grained classification, implemented as cross-layer mutual attention learning, improves EMG-based hand gesture recognition when applied to time-frequency images. It introduces XMANet, which treats each CNN layer as an expert that exchanges attention-guided regions with other layers, and tests it on STFT spectrograms and wavelet scalograms. Across two public datasets, Grabmyo and FORS-EMG, XMANet reports higher accuracy than four standard CNN baselines, with the largest gains on the harder FORS-EMG dataset. The sympathetic reading is that fine-grained, multi-scale features carry information that ordinary CNNs miss, and that the mutual-attention mechanism recovers this information.","feed_headline":"Mutual-attention layers consistently improve EMG gesture models","feed_subtitle":"Adding cross-layer attention experts to CNNs lifts accuracy on STFT and wavelet EMG images across two public datasets.","key_machinery":"The central object is XMANet, which builds a hierarchy of N experts, each aggregating convolutional layers from the shallowest to a successively deeper point in a backbone CNN. Each expert produces a prediction from a global-max-pooled descriptor, and also generates a class-activation-map attention region that is cropped and used as an augmented training sample for the other experts. Training proceeds in a multi-step schedule: the deepest expert is trained first, then shallower experts, then a joint pass on the fused attention map, then a joint pass on the raw image. At inference, the network averages predictions from all experts on both the raw image and the overall attention map, yielding 2×(N+1) scores.","core_discovery":"XMANet, a cross-layer mutual attention learning network, consistently outperforms standard CNN baselines on EMG gesture recognition when the input is a time-frequency image (STFT spectrogram or wavelet scalogram). The paper reports accuracy improvements over ResNet50, DenseNet121, MobileNetV3, and EfficientNetB0 on both the Grabmyo and FORS-EMG datasets, with the largest relative gains on FORS-EMG, where wavelet scalograms and STFT spectrograms leave more room for improvement. The core claim is that fine-grained features extracted from shallow-to-deep layers, combined through mutual attention and attention-based data augmentation, yield robust and accurate EMG classification.","pith_inferences":["The reported gains, especially the small ones (0.03 to 0.5 percentage points on Grabmyo segmented images), may lie within run-to-run variance; repeated-seed experiments with confidence intervals would clarify the true effect size.","The method's benefit appears larger when the baseline accuracy is lower, suggesting that mutual attention mainly helps by recovering information that standard CNNs discard on harder inputs, not by re-ranking already-easy examples.","A natural extension is to apply XMANet directly to raw or lightly preprocessed EMG waveforms instead of spectrogram images, which would remove the time-frequency conversion step and potentially preserve temporal resolution.","Because the attention regions are generated per predicted class, the method could be adapted for weakly supervised localization of the informative muscle-activation patterns, connecting gesture recognition with interpretability for prosthetics."],"forward_implications":["If XMANet's gains hold, the same cross-layer mutual-attention recipe can be applied to other backbones and other physiological signal modalities, not just EMG.","The larger improvements on the FORS-EMG dataset suggest the method is most valuable when the time-frequency representation is noisy or the classification task is harder.","The attention-region-based data augmentation provides a built-in way to focus models on discriminative parts of the spectrogram or scalogram, which could improve interpretability.","The paper's consistent improvements across four architectures imply that the mechanism is architecture-agnostic, so practitioners can retain their preferred CNN and wrap XMANet around it.","The two-pass inference strategy (raw plus attention map) is a simple test-time augmentation that could be combined with other ensembling methods."],"supporting_citations":[{"why":"Supplies the cross-layer mutual attention learning method that XMANet adapts to EMG time-frequency images.","marker":"[24]"},{"why":"Provides the Grabmyo multi-day forearm EMG dataset and its experimental protocol used for evaluation.","marker":"[30]"},{"why":"Provides the FORS-EMG dataset with multiple forearm orientations, used as the second evaluation benchmark.","marker":"[31]"},{"why":"Demonstrates a fine-grained feature approach on facial attribute classification, motivating the application of fine-grained learning to EMG.","marker":"[47]"}],"fun_headline_variants":["Cross-layer attention boosts EMG gesture accuracy","XMANet improves EMG recognition over CNNs","Mutual attention nets beat CNN baselines on EMG datasets","Wavelet and STFT images help EMG gesture learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that a single run of each model, chosen by best validation loss, gives a reliable estimate of test accuracy, so that differences of a few tenths of a percentage point are meaningful rather than random noise.","fun_headline_variants_meta":{"raw":{"variants":["Cross-layer attention boosts EMG gesture accuracy","XMANet improves EMG recognition over CNNs","Mutual attention nets beat CNN baselines on EMG datasets","Wavelet and STFT images help EMG gesture learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000619,"raw_usage":{"total_tokens":2935,"prompt_tokens":1074,"completion_tokens":1861,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":690,"completion_tokens_details":{"reasoning_tokens":1797}},"tokens_in":690,"tokens_out":1861,"duration_ms":12895,"temperature":1.0,"reasoning_tokens":1797,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:41:32.565307+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run each baseline and each XMANet variant on the same train/validation/test splits with multiple random seeds (e.g., 10 seeds) and compute the mean and standard deviation of test accuracy; if the accuracy differences between XMANet and its corresponding baseline shrink to within one standard deviation across seeds, the central claim of consistent improvement collapses.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the cross-layer mutual attention learning method that XMANet adapts to EMG time-frequency images."},{"cited_title":"Pradhan, J","cited_arxiv_id":null,"evidence_quote":"Provides the Grabmyo multi-day forearm EMG dataset and its experimental protocol used for evaluation."},{"cited_title":"Manzoor, A","cited_arxiv_id":null,"evidence_quote":"Demonstrates a fine-grained feature approach on facial attribute classification, motivating the application of fine-grained learning to EMG."}],"review_version":1}