{"id":"5ddbb6b1-6e7e-4477-9049-b7f696c972ac","arxiv_id":"2501.05525","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MECASA, an additive self-attention architecture built on CAS-ViT, reports peak accuracies of 75.07% (EEG), 86.52% (fNIRS), and 87.34% (fused) on the SMR Hybrid BCI dataset.","lead":"A new deep learning model called MECASA classifies rest versus movement from combined EEG and fNIRS brain signals, reporting higher accuracies than five baseline models on a public motor-execution dataset. The paper also documents that fusing the two modalities outperforms either alone, with the upsampled optical-density fNIRS representation giving the best single-modality results.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed consistent superiority rests on an invalid evaluation: overlapping one-second epochs are randomly split, so train and test share temporal context and the accuracy margins in Table 1 are not trustworthy.","rationale":"The reader's weakest assumption—random splitting of overlapping epochs causes data leakage—is exactly the load-bearing concern. The central claim is a comparative performance claim, and comparative performance claims are only meaningful under an evaluation protocol that preserves independence between training and test samples. The manuscript itself provides the grounds for concern: Section 4A describes a random 70/15/15 split of the epoch pool, and Section 4B describes five-fold cross-validation, without clarifying how trials or subjects are partitioned. Given 0.5-second overlap between consecutive epochs, a random split almost guarantees that near-duplicate windows from the same trial appear in both training and test sets. This does not merely inflate all accuracies uniformly; it can differentially benefit models that are better at exploiting the leakage, so the reported ranking of MECASA against baselines is not established. The proposed concrete test—trial-wise grouped cross-validation—directly addresses this by removing the overlap leakage and testing whether the claimed consistent advantage persists. I found no other single concern that is more load-bearing: the architecture and the O(N) CASA module are plausible, and the ablation studies are secondary to the main comparison. Because the reader already conditioned the verdict on exactly this issue, my assessment does not move the verdict; it reinforces the need for a re-evaluation under a clean split before the central claim can be accepted.","tokens_in":13092,"tokens_out":3171,"duration_ms":34531,"concrete_test":"Recompute Table 1 using trial-wise grouped evaluation: assign all epochs originating from the same 12-second trial to the same fold (e.g., stratified five-fold cross-validation grouped by trial, and optionally a leave-one-subject-out variant), retraining MECASA and all baselines under identical preprocessing and hyperparameters. If the accuracy margins shrink to within overlapping confidence intervals or MECASA no longer ranks first across all three modalities, the reported consistent superiority is an artifact of temporal leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—MECASA consistently outperforms baselines across EEG, fNIRS, and fused modalities—depends entirely on the validity of the accuracy comparison. Section 4A states that the 41,228 one-second epochs (0.5 s overlap) per modality were split 70/15/15 for training/validation/test, while Section 4B simultaneously claims stratified five-fold cross-validation; the two protocols are contradictory, and neither is described as subject-wise or trial-wise. Because epochs are cut from 12-second trials with 50% overlap, a random split of the epoch pool places adjacent windows from the same trial into both training and test sets. Those windows share half their signal, so the test set is not independent: a model can effectively observe test content during training. The same-subject overlap across splits further inflates accuracy through memorization of subject-specific patterns. All methods in Table 1 are likely evaluated under the same leaky protocol, but that does not make the relative comparison valid—different architectures can exploit the leakage to different degrees, so MECASA's reported margin (e.g., 75.07 vs. 72.40 for EEG) may be an artifact of how well each model memorizes overlapping training neighbors. The claim of consistent fusion gains is likewise untestable until a clean split is used.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MECASA, a deep-learning architecture for classifying rest versus task states in a motor-execution paradigm using hybrid EEG-fNIRS data. MECASA adapts the CAS-ViT convolutional additive self-attention module, giving an O(N) attention complexity, and combines separate EEG and fNIRS streams with a late fusion network. The authors evaluate MECASA on the SMR Hybrid BCI dataset against EEGNet, TSception, Transformer, LMDA, and ConvNeXT, and report accuracy with 95% confidence intervals, plus ablations on embedding dimensions and fNIRS data representations. The central claims are that MECASA consistently outperforms all baselines across EEG, fNIRS, and fused modalities, and that fusion consistently improves accuracy over single modalities.","tokens_in":13351,"tokens_out":4892,"duration_ms":46763,"significance":"If the claims were supported, MECASA would be a practically useful decoder: it uses a public dataset, compares against several established baselines, provides ablation experiments, and the CASA module has attractive O(N) complexity for real-time BCI use. The manuscript therefore addresses a relevant problem in multimodal BCI classification. However, the central claims are empirical and currently rest on an evaluation protocol that is internally contradictory and potentially leaky, and on accuracy differences that are not statistically corroborated. The strengths of the paper — a clear architecture description and broad baseline coverage — are undermined by these evaluation issues, so the significance of the claimed consistent superiority cannot be assessed until the evaluation is corrected.","major_comments":[{"comment":"The evaluation protocol is contradictory and does not establish independent test samples. Section 4A states that the 41,228 one-second epochs per modality were split into 70% training, 15% validation, and 15% test sets, while Section 4B states that stratified five-fold cross-validation was employed. These two protocols cannot both be the final evaluation procedure, and no details are given about how they are reconciled. More importantly, the epochs are cut from 12-second trials with a 0.5-second overlap, but the split is described only as a random split of the epoch pool. Adjacent epochs from the same trial and the same subject therefore appear in both training and test sets, sharing half of their signal. This temporal and subject-level overlap allows the models to memorize test content during training, and different architectures can exploit this leakage to different degrees. The relative ranking in Table 1 is thus not a trustworthy basis for the claim that MECASA consistently outperforms the baselines. The authors need to re-run the evaluation with trial-wise or subject-wise splits that keep overlapping epochs in the same partition, and report the actual protocol used.","section":"§4A–4B"},{"comment":"No significance tests are reported, and the EEG results do not even show non-overlapping confidence intervals. For EEG, MECASA achieves 75.07±3.89 while EEGNet achieves 72.40±0.52; the lower bound of the MECASA interval overlaps the upper bound of the EEGNet interval, so the headline 'consistently outperforms' claim is not supported for EEG at the 95% level, even before considering the data-split problem. Similar overlap exists for other comparisons (for example, in Table 2, fNIRS accuracy for embedding dims 48-56 is 87.57±0.69 versus 86.52±1.38 for 64-128). The paper should report pairwise significance tests or proper confidence intervals for the accuracy differences, and the text should be adjusted to state which comparisons are actually statistically distinguishable.","section":"Table 1"},{"comment":"The claim that fusion consistently improves accuracy compared to single modalities is directly contradicted by the reported LMDA results. In Table 1, LMDA has EEG accuracy 69.60±0.80, fNIRS accuracy 74.47±0.54, but fused accuracy 57.47±0.50, which is far lower than either unimodal result. The abstract and conclusion state that 'fusion consistently improving accuracy compared to single-modality approaches,' but the paper's own data show at least one counterexample. The evaluation section hedges with 'generally resulted in improved accuracy,' but the abstract and conclusion overstate the finding. This inconsistency must be resolved either by correcting the claim or by explaining why the LMDA fusion result is excluded from the general pattern.","section":"Table 1, Abstract, Conclusion"},{"comment":"The ablation conclusion that embedding dimensions of 64-128 'consistently outperform other configurations' is not supported by the tabulated results. For fNIRS, the 48-56 configuration achieves 87.57±0.69, which is higher in mean accuracy than 64-128's 86.52±1.38, and the confidence intervals overlap. The claim of consistency appears only in the discussion and is not a safe reading of Table 2. The authors should either restrict the claim to the EEG column, perform a statistical comparison, or otherwise temper the conclusion.","section":"§4E, Table 2"}],"minor_comments":[{"comment":"There is a duplicated section heading: 'B Classification Algorithms for Motor Execution' appears twice, once without a period and once as 'B. Classification Algorithms for Motor Execution.' One of these should be removed or renumbered.","section":"§2"},{"comment":"There is a stray heading 'E Ablation Study' immediately before Table 3; this appears to be a formatting artifact and should be removed.","section":"§4E/Table 3"},{"comment":"The caption for Figure 2 reads 'Convolution Adaptive Separable Attention,' but the text uniformly refers to the module as 'Convolutional Additive Self-Attention (CASA).' The caption should be made consistent with the text.","section":"Figure 2 caption"},{"comment":"The classifiers subsection states that the work uses established classifiers 'for EEG-based emotion classification,' but the task in this paper is motor-execution rest-versus-task classification. This looks like a copy-paste from a prior paper and should be corrected.","section":"§4C"},{"comment":"The implementation details do not state how baseline hyperparameters were chosen or whether they were tuned on the validation set. Since the main claim is comparative, the paper should state the hyperparameter selection procedure for every model, or at least note that the same training budget and optimizer settings were used.","section":"§4B"},{"comment":"The notation for the architecture is introduced only informally: the feature-map channel counts C1 and C2, and the meaning of 'Ch×T' for the input, are not precisely defined. A short notation table or explicit values for C1 and C2 would improve reproducibility.","section":"§3C"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant and active area, and the architecture itself is a reasonable adaptation of CAS-ViT to EEG-fNIRS data. The main problem is empirical: the contradictory and potentially leaky evaluation protocol makes the central comparative claim unsupported. This is fixable within the paper's scope by re-running the experiments with trial-wise/subject-wise splits, adding significance tests, and correcting the overstated fusion and ablation conclusions. I would not reject outright, but the authors need to substantially revise the evaluation and the claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper is a solid engineering adaptation of CAS-ViT's additive self-attention to hybrid EEG-fNIRS classification, but the accuracy claims are not supported as presented because the evaluation protocol is ambiguous and likely leaks temporal overlap between neighboring epochs into the test set. A clean trial-wise or subject-wise split is needed before any of the point estimates in Table 1 can be trusted.\n\nWhat's actually new: MECASA is a dual-stream network where each modality is processed by a CAS-ViT-style backbone with the CASA module, and the streams are concatenated and passed through a small fusion head. That specific combination is new, and the O(N) attention complexity is a real practical advantage for BCI use. The ablations on embedding dimensions and fNIRS data representations (OD10, HbT, OD128) are useful practical data points. The paper cites CAS-ViT, EfficientViT, SwiftFormer, and the baselines properly; the two baselines from the authors' own prior work are legitimate, though independent implementations would be safer.\n\nThe soft spots are in the evaluation and they are load-bearing. Section 4A says the 41,228 one-second epochs were split 70/15/15; Section 4B says stratified five-fold cross-validation was used. Both cannot be true, and neither is described as trial-wise or subject-wise. With a 0.5-second overlap between adjacent windows, a random split of the epoch pool puts overlapping segments from the same trial into both training and test sets. That breaks the independence assumption and inflates accuracy; worse, different architectures exploit that leakage differently, so the few-point margin MECASA shows over EEGNet or TSception is not a valid relative comparison. The 95% CIs overlap for EEG, and no significance tests are provided. The ablation also selects the best embedding dims and fNIRS representation on the test set, which is selection-on-test. No code or split metadata are released, so the numbers cannot be independently checked.\n\nNone of this makes the work fraudulent; the architecture is sensible and the claims are plausible. It is a case of a poorly controlled evaluation rather than a fabricated result.\n\nRecommendation: I would send this to a serious referee, with the explicit expectation that the authors re-run everything on a leave-one-subject-out or trial-wise split, add significance testing, and release the split details. If the accuracy gains survive that, the paper is a modest but useful contribution to the EEG-fNIRS BCI literature. If they do not, the architecture note is still worth a short paper, but the headline 'consistently outperforms' would have to go.","headline":"The architecture is a reasonable adaptation of CAS-ViT to EEG-fNIRS fusion, but the leaked evaluation protocol makes the accuracy claims untrustworthy until a clean split is shown.","tokens_in":13909,"tokens_out":3643,"would_cite":false,"duration_ms":33183,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that fusing EEG and fNIRS with a convolutional additive self-attention network, MECASA, outperforms standard baselines on rest-versus-task motor-execution classification, reaching 87.34% fused accuracy.","keywords":["Motor execution","EEG-fNIRS fusion","Additive self-attention","Brain-computer interface","CASA module","Rest versus task classification","SMR Hybrid BCI dataset"],"falsifier":"Retrain and evaluate MECASA under a strict trial-wise or subject-wise split (for example, leave-one-subject-out or holding out whole trials) with all hyperparameters unchanged, and check whether fused accuracy still exceeds the single-modality results by the reported margin.","tokens_in":12867,"feed_emoji":"🧠","tokens_out":6381,"duration_ms":54811,"temperature":0.7,"pith_summary":"This paper argues that combining EEG and fNIRS signals improves rest-versus-task classification in a motor-execution brain-computer interface, and that the proposed MECASA architecture does this better than existing baselines. On the SMR Hybrid BCI dataset, MECASA reports 75.07% accuracy for EEG, 86.52% for fNIRS, and 87.34% for the fused data, with fusion consistently beating each single modality across most compared methods. The authors also find that fNIRS alone outperforms EEG alone, and that the best results come from embedding dimensions of 64-128 for EEG and upsampled optical density (OD128) for fNIRS. If the results hold, the architecture offers a practical decoder that exploits complementary temporal and hemodynamic information at linear attention cost.","feed_headline":"EEG-fNIRS fusion reaches 87.3% motor-task accuracy","feed_subtitle":"MECASA's additive self-attention tops EEGNet and Transformer baselines at O(N) cost.","key_machinery":"The load-bearing component is the Convolutional Additive Self-Attention (CASA) module, which computes similarity between queries and keys as a sum of convolutional context mappings, $\\text{Sim}(Q,K)=\\Phi(Q)+\\Phi(K)$, with $\\Phi$ composed of Sigmoid-based channel attention and spatial attention, and forms output as $O=\\Gamma(\\Phi(Q)+\\Phi(K))\\cdot V$. Because the similarity uses convolutions instead of the quadratic $QK^\\top$ product, the module scales as $O(N)$. Two separate MECASA streams process EEG and fNIRS signals, and a fusion network concatenates their penultimate-layer features before two fully connected layers produce the binary rest-versus-task output.","core_discovery":"The central claim is that replacing softmax self-attention with a convolutional additive similarity function produces a model, MECASA, that outperforms EEGNet, TSception, Transformer, LMDA-Net, and ConvNeXT on EEG, fNIRS, and their fusion for motor-execution classification. On the SMR Hybrid BCI dataset, the reported accuracies are 75.07±3.89% for EEG, 86.52±1.38% for fNIRS, and 87.34±0.42% for the fused representation, with fusion improving over single-modality accuracy for every compared method except LMDA. The paper further claims that fNIRS yields higher accuracy than EEG alone and that the choice of fNIRS representation matters, with OD128 outperforming OD10 and total hemoglobin.","pith_inferences":["The reported accuracy gap likely depends on the evaluation split; if random epoch splitting allows temporally overlapping epochs from the same 12-second trial to land in both training and test sets, all methods' numbers may be inflated by correlation leakage.","A strict trial-wise or subject-wise cross-validation experiment, keeping everything else fixed, would settle whether the fusion advantage is a genuine architectural effect or a split artifact.","The additive-attention design could transfer to other long physiological time series where quadratic attention is prohibitive, provided the evaluation protocol is leakage-free."],"forward_implications":["If the results hold, hybrid EEG-fNIRS decoding for motor execution can be improved by fusing the two modalities at the feature level rather than relying on either signal alone.","The linear-complexity CASA module makes the architecture a candidate for real-time BCI use, where quadratic self-attention is often too slow.","The fNIRS ablation suggests that upsampling optical density to 128 Hz is a more informative input representation than lower-rate optical density or total hemoglobin.","The embedding-dimension ablation points to a practical default of 64-128 dimensions for EEG-based variants of this architecture."],"supporting_citations":[{"why":"Supplies the SMR Hybrid EEG-fNIRS dataset with 15 subjects and the 12-second rest/task motor-execution trials on which every reported result is measured.","marker":"(53)"},{"why":"Introduces the CAS-ViT architecture whose convolutional additive self-attention (CASA) module MECASA adapts as its core mechanism.","marker":"(48)"},{"why":"Provides the EEGNet baseline that MECASA is compared against on EEG, fNIRS, and fused data.","marker":"(55)"},{"why":"Provides the TSception baseline, the strongest comparable method on fNIRS and near-MECASA on fusion in the reported table.","marker":"(56)"},{"why":"Provides the LMDA-Net baseline, which is the one compared method where fused accuracy drops below fNIRS alone.","marker":"(59)"},{"why":"Provides the standard Transformer baseline used to show that softmax self-attention underperforms the additive variant.","marker":"(60)"},{"why":"Supplies the Modified Beer-Lambert law used to convert raw fNIRS signals into optical density and total hemoglobin, the preprocessing variants tested in the ablation.","marker":"(54)"},{"why":"Provides the ConvNeXT baseline in the comparison table and the prior EEG application that the authors reuse as a comparator.","marker":"(58)"}],"fun_headline_variants":["MECASA fusion reaches 87.3% on motor-execution BCI","Additive self-attention lifts EEG-fNIRS BCI to 87.3%","Fused EEG-fNIRS with MECASA hits 87.3% accuracy","MECASA's fusion nets 87.3% on motor execution"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that one-second epochs cut from the same 12-second trial are independent when the dataset is randomly split into training, validation, and test sets; if overlapping epochs from the same trial appear in more than one partition, the reported accuracies are inflated by temporal correlation.","fun_headline_variants_meta":{"raw":{"variants":["MECASA fusion reaches 87.3% on motor-execution BCI","Additive self-attention lifts EEG-fNIRS BCI to 87.3%","Fused EEG-fNIRS with MECASA hits 87.3% accuracy","MECASA's fusion nets 87.3% on motor execution"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000922,"raw_usage":{"total_tokens":3983,"prompt_tokens":1005,"completion_tokens":2978,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":621,"completion_tokens_details":{"reasoning_tokens":2890}},"tokens_in":621,"tokens_out":2978,"duration_ms":19423,"temperature":1.0,"reasoning_tokens":2890,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:14:06.429335+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain and evaluate MECASA under a strict trial-wise or subject-wise split (for example, leave-one-subject-out or holding out whole trials) with all hyperparameters unchanged, and check whether fused accuracy still exceeds the single-modality results by the reported margin.","supporting_citations":[],"review_version":1}