{"id":"82cb6851-0e89-4b93-aabf-73c26a068520","arxiv_id":"2507.03814","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"DeepSHAP-based channel selection on an alpha-power CNN allows a compact TCN to match near full-64-channel accuracy with 32 EEG channels.","lead":"This paper proposes SHAP-AAD, a two-stage method that uses DeepSHAP explanations to pick the most important EEG channels and then trains a smaller temporal network for auditory attention detection. It reports that 32 channels can match 64-channel accuracy, which could make EEG-based hearing aids more wearable.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation leaks information through overlapping windows and test-set channel selection; without trial-disjoint validation, the reported 32-channel accuracy is not an unbiased estimate, so the central claim is unsubstantiated.","rationale":"The paper's headline is an empirical performance claim. The necessary condition is that the evaluation protocol yields unbiased accuracy estimates. That condition is directly contradicted by two explicit design choices: overlapping windows combined with random CV, and use of test-set samples for SHAP-based channel selection. Overlapping windows inflate accuracy because the model can memorize EEG segments that recur in training and test windows; test-set channel selection inflates accuracy because feature selection is an oracle that uses test labels. Both are known sources of optimistic bias in EEG decoding evaluations. Without reporting results under a trial-disjoint, train-only-selection protocol, the 32-channel parity could likely be an artifact. This is more load-bearing than the transferability assumption: even if the CNN ranking did transfer perfectly to the TCN, the reported numbers would still be unreliable. I therefore agree with the rejection verdict but emphasize the leakage concern as the primary basis. The concrete test would settle whether the central claim survives a clean evaluation.","tokens_in":587,"tokens_out":3577,"duration_ms":100767,"concrete_test":"Re-run the full pipeline on the DTU dataset with trial-disjoint folds: assign entire 50-s trials to train/validation/test (no shared windows), compute SHAP values using only training/validation windows, then train the TCN with top-32 selected channels. Compare top-32 vs 64-channel mean accuracy with per-subject error bars. If top-32 accuracy remains within 1–2 points of 64-channel under this clean protocol, the claim survives; if not, the leakage explains the reported parity.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim—that top-32 channels match 64-channel accuracy—holds only if the reduced-channel TCN accuracy is an unbiased estimate of generalization. The protocol violates this. Section III.A segments data into 10-s windows with 50% overlap, and Section IV.A uses random CV without trial-disjoint grouping, so overlapping windows from the same 50-s trial can appear in both training and test folds. Additionally, Section III.B.3 states that '100 test samples of the test set' are used to compute SHAP values and averaged across folds to choose channels. Thus the channel ranking is fit using labels and data from the test set, and the subsequent TCN evaluation is on data that has already influenced feature selection. Both leaks inflate accuracy. The reported 79.21% vs 81.06% figure is therefore not a valid estimate of prospective performance; the load-bearing condition of unbiased evaluation is not established. Even if DeepSHAP ranking transfers perfectly to the TCN, the paper's experimental design cannot support the claim as reported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SHAP-AAD, a two-stage framework for EEG-based auditory attention detection (AAD). In the first stage, a CNN trained on topographic alpha-band power images is interpreted with DeepSHAP to rank the 64 EEG channels by importance, and the top-k channels are selected. In the second stage, a compact temporal convolutional network (TCN) is trained directly on raw EEG waveforms from the selected channels. Experiments on the public DTU dataset report that using 32 channels yields average accuracy of 79.21% versus 81.06% with all 64 channels, and that some subjects retain high accuracy with as few as 8 channels. The central claim is that DeepSHAP-guided channel reduction can match the accuracy of the full-channel setup while reducing sensor count.","tokens_in":6448,"tokens_out":6543,"duration_ms":74524,"significance":"The problem addressed is practically relevant: reducing EEG channel count could improve wearability and comfort of AAD systems. The use of DeepSHAP for explainability-driven channel selection appears novel in this application, and the lightweight TCN architecture is a reasonable choice for deployment. The paper is clearly structured and uses a public dataset, which materially aids reproducibility. However, the empirical evidence is compromised by two independent forms of leakage—overlapping decision windows randomly split across train and test, and SHAP-based channel selection performed on test data—so the reported accuracies are not unbiased estimates of prospective performance. No error bars or statistical significance tests are provided. Consequently, the central claim is not established in the current manuscript; the value of the work hinges on a re-evaluation with a clean experimental protocol.","major_comments":[{"comment":"The evaluation protocol leaks temporal information: Section III.A segments the continuous EEG into 10-s windows with 50% overlap, and Section IV.A splits the data into training (80%), validation (10%), and test (10%) using random cross-validation without grouping by trial. As a result, overlapping windows from the same 50-s trial can appear in both training and test folds, and the reported test accuracy is inflated by strong temporal autocorrelation. All accuracy figures, including the headline 79.21% vs. 81.06% comparison in Section IV.C and Fig. 6, are therefore not valid estimates of generalization. A trial-disjoint split (e.g., grouping all windows of a 50-s trial into the same fold) is required before any conclusion about the method's performance can be drawn.","section":"III.A and IV.A"},{"comment":"The channel selection procedure uses the test set: the text states that 'after each cross-validation fold, a 100 test samples of the test set were used to compute the SHAP values,' and these values are averaged across folds to produce the global importance map that determines the top-k channels. Because the test data are used to choose the channels on which the TCN is later trained and evaluated, the evaluation is not independent of the feature-selection step. The reported reduced-channel accuracy is optimistically biased, and this bias is load-bearing for the paper's central claim. The channel ranking must be derived exclusively from training data (e.g., a nested cross-validation loop or a separate selection set).","section":"III.B.3"},{"comment":"The channel ranking is computed from the alpha-power topographic CNN, but the final classifier is a TCN operating on raw waveforms. The paper asserts that channels important for the image-based CNN are the same channels that maximize the raw-waveform TCN's accuracy, but no evidence is provided for this transfer. Since the two models use fundamentally different input representations, this assumption is load-bearing: without it, the reported performance of the reduced-channel TCN is not explained by the SHAP-based selection. A concrete control experiment, such as comparing the selected channels against random or fixed channel subsets on the TCN, or computing SHAP values on the TCN itself, is needed to validate the transfer.","section":"III.B vs. III.C"}],"minor_comments":[{"comment":"The text says 'five different EEG channel configurations: 64, 60, 48, 32, 16, and 8 channels,' which lists six configurations; moreover, the 60-channel setting is not discussed in the results. Please correct the count and either add or remove the 60-channel configuration.","section":"IV.C"},{"comment":"The phrase 'a 100 test samples' should be '100 test samples.' Also, it is unclear whether the 200 training samples used as the DeepSHAP background are drawn from each fold's training set or from the full training set, and whether the 100 test samples are the same across folds; please clarify.","section":"III.B.3"},{"comment":"The SHAP importance maps in Figures 3 and 4 would benefit from axis labels, a colorbar with units, and a clear statement of the subject and windowing parameters used; currently the figures are described only in the caption. In Figure 5, the subject indices on the x-axis should be explicitly defined.","section":"Figures 3 and 4"},{"comment":"No comparison with existing channel-selection methods for AAD (e.g., the approaches in Refs. [13]–[15]) is provided. Since the introduction positions the work against these methods, the experimental section should include them as baselines to substantiate the claimed advantage.","section":"IV.C and Table I"},{"comment":"Only mean accuracies over 10 folds are reported, with no standard deviations or confidence intervals. Even after fixing the leakage issues, the claim that 32 channels are 'comparable' to 64 channels would be much stronger if accompanied by measures of variability across folds and subjects.","section":"IV.A"}],"recommendation":"major_revision","confidential_remarks":"The experimental flaws identified in the major comments are severe enough that the current results cannot be trusted, but they are fixable in principle because the dataset is public and the method is fully described. I recommend major revision rather than outright rejection, with the understanding that acceptance should require a complete re-analysis using trial-disjoint cross-validation, channel selection restricted to training folds, and appropriate uncertainty reporting. If the authors cannot or do not redo the experiments, the paper's central claim remains unsupported and it should be rejected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is sensible and new in a narrow way: using DeepSHAP to rank EEG channels for auditory attention detection is a fair extension of explainable AI to a practical wearability problem. The two-stage design—selecting channels with an alpha-power topographic CNN, then training a raw-waveform TCN on those channels—is clearly motivated, and the paper is honestly written. Reporting model size and computation cost is a plus, and using the public DTU dataset makes the work reproducible in principle. I agree with the reader that the central claim is plausible but unsubstantiated as evaluated.\n\nThe evaluation has two real leaks. First, 10-second windows with 50% overlap are randomly split into train and test, so overlapping windows from the same 50-second trial can appear on both sides. Second, Section III.B.3 says 100 test samples are used to compute SHAP values, averaged across folds, and those values choose the channels for the TCN. The test set therefore influences feature selection, and the subsequent accuracy is not an independent estimate. The reported 79.21% vs. 81.06% is not a valid prospective number; it is inflated by both leaks.\n\nThe transfer assumption is also worth flagging, though it is a concern about interpretation rather than inflation: channel importance is computed for a CNN trained on alpha-power images, then applied to a TCN trained on raw waveforms. The paper gives no evidence that the same channels are optimal for both representations. That is load-bearing for explaining why the reduced-channel TCN works, not for whether the numbers are biased.\n\nWhat is missing is proportionate to the claim: no comparison with the group-LASSO, decoder-magnitude, or greedy utility methods cited in the introduction, no error bars or statistical tests, no code. The prior channel selection methods are directly relevant, and omitting them makes the contribution look smaller and the evaluation less persuasive.\n\nIf the protocol were fixed—trial-disjoint splits, SHAP computed only from training folds, comparisons to prior methods—the core idea might survive. As submitted, the evidence does not support the headline result. This is a paper for AAD and BCI researchers, and it deserves a serious referee because the question is worth answering and the approach is new. I would send it back for major revision, not desk reject it, and would want to see a clean re-evaluation before trusting any of the accuracy figures.","headline":"The idea is a reasonable new application of DeepSHAP to EEG channel selection for AAD, but the reported accuracy numbers are not trustworthy because the evaluation leaks information through overlapping windows and test-set channel selection.","tokens_in":673,"tokens_out":2276,"would_cite":false,"duration_ms":34656,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Shapley-value attribution method can rank EEG channels by importance, letting a 32-channel system match the accuracy of a 64-channel setup at detecting which speaker a listener attends to.","keywords":["electroencephalography","auditory attention detection","channel reduction","DeepSHAP","Shapley values","temporal convolutional network","explainable AI","EEG"],"falsifier":"A reader could settle the transfer claim by training the same TCN on the top-32 channels from the DeepSHAP image-CNN ranking, on 32 channels chosen by greedy backward elimination on the TCN itself, and on random 32-channel subsets; the central claim fails if the greedy set does not clearly beat the DeepSHAP set or if random sets match it.","tokens_in":6033,"feed_emoji":"🧠","tokens_out":7251,"duration_ms":77475,"temperature":0.7,"pith_summary":"The paper tries to show that explainable-AI feature attribution can replace expensive 64-channel EEG setups for auditory attention detection, the task of deciding which of two competing speakers a listener is following. It ranks EEG channels with DeepSHAP applied to a CNN trained on alpha-power scalp maps, keeps the top channels, and trains a lightweight temporal convolutional network on the raw signals of those channels. On a public 18-subject dataset, the 32-channel system reaches 79.21% average accuracy against 81.06% for the full 64-channel setup, and some subjects retain satisfactory accuracy with 8 channels. If this holds, hearing-aid-style devices could use far fewer electrodes without a large loss in detection quality.","feed_headline":"32 EEG channels rival 64-channel attention detection","feed_subtitle":"Shapley-value channel ranking keeps accuracy at 79.21% versus 81.06%, a step toward wearable hearing aids.","key_machinery":"The load-bearing object is the DeepSHAP importance map: DeepSHAP combines Shapley-value game theory with DeepLIFT backpropagation to assign each pixel of a 32x32 alpha-power topographic image a contribution score, which is averaged over samples and cross-validation folds and mapped back to electrode locations. These scores produce a global channel ranking from which the top-k electrodes are kept; a two-layer dilated temporal convolutional network then classifies attention from raw waveforms of only those channels. The ranking is the mechanism that turns an explainability output into a channel-selection procedure.","core_discovery":"The paper's central claim is that DeepSHAP-derived channel importance transfers across representations: a CNN that classifies attention from 32x32 topographic images of alpha-band power assigns per-pixel Shapley values that, averaged across samples and cross-validation folds, rank the 64 scalp channels, and the top-k of that ranking can be fed as raw time series to a compact two-layer dilated TCN that detects attention with accuracy close to the full montage. Quantitatively, the paper reports 81.06% mean accuracy with 64 channels and 79.21% with 32, with further degradation at 16 and 8 channels and notable subject-to-subject variation. The paper argues that moderate reduction to 32 channels preserves most discriminative information and that the TCN avoids the interpolation artifacts image representations would introduce after channel removal.","pith_inferences":["The authors' transfer assumption could be tested directly by computing DeepSHAP on the TCN over raw waveforms instead of on the image CNN; if the rankings disagree, the reported accuracy of the reduced TCN may be an artifact of the particular CNN used for selection.","The same selection pipeline could be applied to other EEG tasks, such as sleep staging or motor imagery, wherever a stable topographic summary exists, but its success would hinge on the same cross-representation transfer.","A subject-specific rather than global ranking might shrink the required channel count further, since the paper's own data show large inter-subject variability in how much accuracy drops at 8 or 16 channels.","The paper does not compare against exhaustive or greedy channel selection on the TCN itself, so the claim that DeepSHAP adds value over random or heuristic reduction is not yet established."],"forward_implications":["A 32-channel wearable EEG device could plausibly support auditory attention detection with accuracy within roughly two percentage points of a full 64-channel laboratory setup.","Channel sets can be chosen per subject from the global importance map, so individual head geometries or attention-related topographies can guide electrode placement.","Model size and computation drop modestly with channel count, from 0.18M parameters and 220 MFLOPs at 64 channels to 0.15M and 183 MFLOPs at 32, making real-time inference easier on embedded hardware.","The two-stage design separates selection, image-based CNN, from classification, waveform-based TCN, so either stage could be swapped independently in later systems."],"supporting_citations":[{"why":"Supplies the DeepSHAP algorithm that computes the channel importance scores.","marker":"[16]"},{"why":"Defines Shapley values, the cooperative-game-theory notion DeepSHAP approximates.","marker":"[17]"},{"why":"Provides DeepLIFT, the backpropagation strategy DeepSHAP builds on.","marker":"[19]"},{"why":"Supplies the public EEG and audio dataset used for training and evaluation.","marker":"[20]"}],"fun_headline_variants":["32 EEG channels nearly match 64 in attention detection","DeepSHAP cuts EEG channels in half, accuracy barely drops","Shapley-guided channel reduction keeps AAD accuracy high","Fewer EEG channels, similar attention detection accuracy","EEG channel count halved with just 1.85% accuracy drop"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The channel ranking produced by a model that looks at maps of brain-wave power over the scalp must transfer to a different model that reads the raw waveform of each electrode; if that transfer fails, the reported accuracy of the reduced-channel system is not explained by the selection method.","fun_headline_variants_meta":{"raw":{"variants":["32 EEG channels nearly match 64 in attention detection","DeepSHAP cuts EEG channels in half, accuracy barely drops","Shapley-guided channel reduction keeps AAD accuracy high","Fewer EEG channels, similar attention detection accuracy","EEG channel count halved with just 1.85% accuracy drop"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000148,"raw_usage":{"total_tokens":1160,"prompt_tokens":884,"completion_tokens":276,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":500,"completion_tokens_details":{"reasoning_tokens":193}},"tokens_in":500,"tokens_out":276,"duration_ms":3464,"temperature":1.0,"reasoning_tokens":193,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:01:07.571848+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader could settle the transfer claim by training the same TCN on the top-32 channels from the DeepSHAP image-CNN ranking, on 32 channels chosen by greedy backward elimination on the TCN itself, and on random 32-channel subsets; the central claim fails if the greedy set does not clearly beat the DeepSHAP set or if random sets match it.","supporting_citations":[{"cited_title":"Explaining models by propagating shapley values of local components,","cited_arxiv_id":null,"evidence_quote":"Supplies the DeepSHAP algorithm that computes the channel importance scores."},{"cited_title":"A value for n-person games,","cited_arxiv_id":null,"evidence_quote":"Defines Shapley values, the cooperative-game-theory notion DeepSHAP approximates."},{"cited_title":"Learning important features through propagating activation differences,","cited_arxiv_id":null,"evidence_quote":"Provides DeepLIFT, the backpropagation strategy DeepSHAP builds on."}],"review_version":1}