{"id":"d994d238-8d45-44a2-be91-37a2a8331b03","arxiv_id":"2506.11178","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"BrainMAP combines atlas-guided subgraph selection with multimodal attention-gated fusion to classify and localize brain disease efficiently.","lead":"BrainMAP is a machine-learning framework that identifies disease-related brain regions from fMRI and DTI scans and uses only those regions to classify neurodegenerative diseases like Alzheimer's and Parkinson's. It targets faster, less memory-hungry diagnosis that could be practical in clinics with limited hardware.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The atlas-mask selection in Algorithm 1 uses full-data labels and is not nested inside the five-fold evaluation; the reported ACC/AUC are likely optimistically biased, so the efficiency/localization claims rest on unverified numbers.","rationale":"The reader's weakest assumption identifies the same load-bearing flaw, and my reading of Sections IV-A and V-B agrees with it. The central claim of BrainMAP is that atlas-guided subgraph filtering localizes disease-relevant regions while reducing computation without hurting accuracy. The evidence for the accuracy component is Table I, and the protocol that produces Table I is vulnerable to label leakage because Algorithm 1 selects masks using the full label vector Y before cross-validation. This is not a stylistic or marginal issue: it concerns whether the evaluated model is a valid predictor for unseen subjects. Because the paper does not describe nesting the selection inside folds, and because no code is provided to demonstrate otherwise, the primary numerical results cannot be accepted as stated. I nevertheless do not see a reason to move the verdict in a different direction: the reader's REJECT verdict is consistent with this concern being unresolved. The concrete test I propose would settle the issue by rerunning the reported experiments under a fully nested selection protocol; if the numbers hold, the concern would be retired, while if they drop, the headline claim would need substantial revision. The recommendation is UNCHANGED because the reader already reached REJECT and this analysis does not change that assessment.","tokens_in":9837,"tokens_out":2227,"duration_ms":30277,"concrete_test":"Re-run the full BrainMAP pipeline with AGSF mask selection nested inside each of the five folds: for each fold, run Algorithm 1 using only the training subjects' labels to choose U, then evaluate on the held-out fold, and report the fold-averaged ACC/AUC for ADNI and PPMI. If either dataset's ACC/AUC drops materially relative to Table I, the reported numbers are inflated by label leakage and the headline accuracy-preservation claim is not supported. As a secondary check, confirm that the SVD and class-prototype computation in Section IV-C also uses only the current training fold.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV-A's Atlas-Guided Subgraph Filtering selects the ROI subset U by computing ACC_k = average random-forest accuracy on (R^(k), Y), where Y is the full label vector, and then keeps masks with ACC_k > ACC_0. Section V-B states that predictive performance is evaluated with five-fold cross-validation but never says this mask selection is performed inside each training fold. If U is chosen once on all subjects before splitting, then the test-fold labels have already influenced which ROIs are retained. This is a classic feature-selection leak: the reported 82.3% ACC / 91.5% AUC on ADNI and 86.2% ACC / 93.8% AUC on PPMI in Table I would not be estimates of generalization to unseen subjects, because the test data were used to select the subgraph. The leak is material because the whole contribution is that concentrating on disease-relevant subgraphs preserves accuracy while cutting computation; the accuracy side of that claim is exactly what the Table I numbers are supposed to demonstrate. The concern is not merely about exaggeration of a secondary number: it directly supports the headline 'without compromising predictive accuracy.' The ablation in Table II partially corroborates that AGSF is not the accuracy driver on ADNI (w/o AGSF: 82.6 ± 3.1 vs BrainMAP: 82.3 ± 3.5), but this does not repair the evaluation protocol for the main results. A second ambiguity is that Section IV-C computes SVD and class prototypes on 'training rows' without specifying whether those rows are restricted to the current CV fold; if not, the feature-distillation step introduces another label-leak pathway. The paper also provides no code or reproducibility artifact, which prevents checking the pipeline, but the selection-bias issue is identifiable directly from the text.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BrainMAP, a multimodal graph learning framework for brain disease classification and localization. It first uses an atlas-guided subgraph filtering (AGSF) step that selects a subset of ROIs by comparing random-forest accuracies computed on labeled data against a whole-brain baseline, then fuses fMRI and DTI features via cross-node attention and gated interaction, applies SVD-based feature distillation, and classifies with a lightweight GCN. On ADNI and PPMI, the paper reports higher accuracy and AUC than most baselines and claims over 50% reduction in computational overhead. The central evaluation protocol has a potential label-leakage issue in the atlas-mask selection step, and the main results are reported without error bars, with a selective efficiency comparison, so the headline claims are not yet supported.","tokens_in":10186,"tokens_out":6822,"duration_ms":79903,"significance":"If the headline results survived a properly nested cross-validation, BrainMAP would be a useful practical contribution: it combines localization with efficiency, uses standard atlas-based ROIs, and reports fast per-epoch training. The paper also includes an ablation study and parameter sensitivity checks, which are positive features. However, because the primary accuracy and efficiency claims rest on a non-nested feature-selection procedure and on point estimates without variability, the significance of the work cannot be assessed from the current evidence. No code or detailed experimental logs are provided, further limiting reproducibility.","major_comments":[{"comment":"The atlas-mask selection is performed on the full labeled dataset before the five-fold split. Algorithm 1 computes ACC[k] from RFscore(R(k), Y) with the full label vector Y and uses ACC[k] > baseline to define S and U; Section V-B only states that predictive performance is evaluated with five-fold cross-validation and does not state that U is re-estimated inside each training fold. If U is fixed using all subjects, the held-out test folds have already influenced which ROIs are retained, so the reported ACC/AUC in Table I (82.3/91.5 on ADNI, 86.2/93.8 on PPMI) are optimistically biased and do not estimate generalization to unseen subjects. This leak is load-bearing because the abstract and conclusion claim that accuracy is preserved without being compromised. The ablation in Table II does not repair this issue: the w/o AGSF row uses R0 and is leak-free, but the BrainMAP row is not. The authors must repeat the whole pipeline with the AGSF selection nested in each training fold and report the resulting unbiased estimates.","section":"Section IV-A, Algorithm 1; Section V-B"},{"comment":"The main classification results report fold averages without standard deviations, confidence intervals, or statistical tests. Table I lists no variability for ACC or AUC, so the reader cannot judge whether BrainMAP's differences from baselines, for example 82.3 versus 81.2 for Cross-GNN on ADNI, are meaningful. Table II provides ± values, but the main table does not. Please report mean ± standard deviation over folds for all methods and, ideally, pairwise significance tests.","section":"Section V-B, Table I"},{"comment":"The feature distillation step computes SVD and class prototypes on 'training rows,' but the manuscript does not specify whether these are the training rows within each fold or rows from the full dataset. If the SVD and prototype statistics are estimated on the full data before splitting, this is a second form of label leakage because test rows contribute to the prototypes for their class labels, and the Bernoulli-drop distribution also depends on the labels. Please state explicitly that all data-dependent transformations, including AGSF selection, SVD, prototypes, and masking, are fit only on training folds and applied to test folds.","section":"Section IV-C; Section V-B"},{"comment":"The efficiency claim is selective. Table I shows that GAT achieves per-epoch times of 0.0009 s on both datasets, roughly 27 times faster than BrainMAP's 0.0243 s on ADNI, with lower memory (1.1 GB versus 1.5 GB). The abstract's 'more than 50% reduction in computational overhead' and 'outperforms state-of-the-art methods in computational efficiency' are not true against all compared methods; they hold only against multimodal baselines and against BrainMAP's own w/o AGSF variant. Please specify the intended comparator class, report all methods' parameter counts, and clarify whether times are per epoch or per sample.","section":"Section V-C, Table I"},{"comment":"The localization claim is central to the paper's title and contribution, yet Section V-F provides only qualitative visual examples from ADNI and no quantitative evaluation of whether the selected regions match known disease effects or are stable across cross-validation folds. Please add a quantitative localization evaluation, such as overlap with independently established disease-related ROI sets, consistency of selected ROIs across folds, or comparison with a null selection procedure.","section":"Section V-F, Figures 4 and 5"}],"minor_comments":[{"comment":"The symbol K is used for both the number of candidate masks in Section III and the feature-group removal count in Section V-B; please rename one of them to avoid ambiguity.","section":"Section III vs. Section V-B"},{"comment":"The denominator in the attention-weight equation is formatted as 'N′s X e^...', which is visually ambiguous; please rewrite it as an explicit summation over j′.","section":"Section IV-B, Eq. (4)"},{"comment":"The union operation in 'U ← S k∈S Rk' is missing the union symbol; it should be 'U ← ∪_{k∈S} R_k'.","section":"Algorithm 1, line 9"},{"comment":"The ablation subsection refers to 'GIF' when the module is called AGIF elsewhere; please use consistent naming.","section":"Section V-E"},{"comment":"The paper reports a single AUC value for three-class datasets (ADNI: NC/MCI/AD; PPMI: NC/prodromal/PD); please specify the multiclass AUC averaging scheme used.","section":"Section V-B, Table I"},{"comment":"The parameter studies do not state whether the reported accuracy values are computed on training, validation, or test folds; please clarify the protocol for these sweeps.","section":"Section V-D, Figures 2 and 3"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the primary evaluation protocol has a clear label-leakage issue in the atlas-mask selection, and the current Table I numbers cannot be interpreted as estimates of generalization. Unless the authors can provide correctly nested cross-validation results that preserve the reported accuracy and efficiency, the paper's central claims will remain unsupported. I would be willing to review a revised version with corrected protocols and full reporting of variability. The work is not ready for publication in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take on arXiv:2506.11178 (BrainMAP). It's a sensible engineering pipeline: pick disease-relevant atlas regions with random forest accuracy, fuse fMRI and DTI via cross-node attention and gated mixing, distill with SVD, classify with a GCN. Each piece is known, but the specific combination and the efficiency framing are new enough to be publishable if the evaluation holds. The writing is clear, and the ablation is more honest than most: Table II shows that removing AGSF actually improves ADNI accuracy by 0.3%, which suggests the module's real payoff is memory and time, not accuracy.\n\nThe problem is the evaluation protocol. Section IV-A selects the ROI subgraph U using ACC_k computed on the full labeled dataset, and Section V-B says performance is evaluated with five-fold CV but never says the selection is nested inside folds. If U is chosen once on all subjects before splitting, the test folds have already influenced which ROIs are kept. That is a classic feature-selection leak, and it directly hits the paper's headline claim of 'without compromising predictive accuracy.' The reported 82.3/91.5 on ADNI and 86.2/93.8 on PPMI would not be honest estimates of generalization. The ablation partially softens this: on ADNI, BrainMAP without AGSF gets 82.6 ± 3.1, so the leak may not be inflating accuracy much there. But it doesn't repair the protocol for the main results or for PPMI, and the paper never reports a nested version. A second ambiguity is that the SVD distillation in Section IV-C uses 'training rows' without saying whether those are restricted to the current fold.\n\nMissing error bars on Table I, a selective efficiency comparison (GAT is faster, several baselines use less memory), and no code are secondary. The core problem is the leak.\n\nIf the authors rerun with selection inside each CV fold, and ideally release code, this could be a solid methods paper for the brain-graph community. As is, the central numbers don't support the conclusions. I'd send it to review with a request for major revision — the architecture has merit and the fix is straightforward.","headline":"A sensible multimodal brain-graph pipeline undone by a likely label leak in the atlas-mask selection, which is not nested inside the cross-validation; the accuracy claims need re-running before they can be trusted.","tokens_in":10728,"tokens_out":1585,"would_cite":false,"duration_ms":18638,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that focusing brain-graph learning on atlas-defined disease-relevant subgraphs cuts computational overhead by more than half, and that a two-stage attention-gated fusion of fMRI and DTI preserves or improves…","keywords":["Brain disease","Graph learning","Lightweight model","Localization","Multimodal fusion","fMRI","DTI","Atlas-guided subgraph filtering"],"falsifier":"Re-run the whole pipeline with atlas-mask selection nested inside each cross-validation fold, so masks are chosen only from training subjects and then evaluated on the held-out fold; if ADNI ACC/AUC falls materially below 82.3%/91.5% or PPMI below 86.2%/93.8%, the headline numbers depend on label information used at selection time rather than on the subgraph mechanism itself.","tokens_in":9653,"feed_emoji":"🧠","tokens_out":10405,"duration_ms":112392,"temperature":0.7,"pith_summary":"BrainMAP is a multimodal graph-learning framework for detecting Alzheimer's and Parkinson's disease from paired fMRI and DTI brain graphs. It tries to establish that localizing disease-relevant brain subgraphs before training is not just an interpretability aid but an efficiency mechanism: an atlas-guided filter keeps only ROIs whose pooled features separate patient groups better than the full 90-node graph, and a two-stage fusion then combines the surviving functional and structural signals. The reported results are 82.3% accuracy and 91.5% AUC on ADNI and 86.2% accuracy and 93.8% AUC on PPMI, with per-epoch runtimes of 0.0243 s and 0.0023 s, respectively, well below multimodal baselines. If correct, this is a practical path toward running multimodal brain-graph diagnosis on resource-constrained clinical hardware while outputting a concrete disease-localization map.","feed_headline":"BrainMAP halves brain-graph compute without hurting accuracy","feed_subtitle":"An atlas filter picks disease-relevant regions, then fusion of fMRI and DTI yields 91.5% ADNI AUC and 93.8% PPMI AUC.","key_machinery":"The load-bearing mechanism is the Atlas-Guided Subgraph Filtering (AGSF) module paired with the Attention-Gated Interaction Fusion (AGIF) module. AGSF uses the AAL atlas's predefined subsystems, pools each subject's node features within each mask, trains a random forest on those pooled features against the disease labels, and forms the reduced ROI set $U$ as the union of masks whose mean accuracy exceeds the all-nodes baseline. AGIF then projects the filtered fMRI and DTI features into a shared space, computes scaled-dot-product cross-node attention from each fMRI parcel to the DTI tracts, and for each parcel forms agreement, disparity, gated-mixture, and raw-modality terms that are concatenated into a single embedding. SVD-based feature distillation, adopted from the paper's reference [19], removes the largest and smallest loading components and applies Bernoulli dropout before the embedding is fed to a sparse GCN. Because all subsequent attention and convolution steps run on $|U|$ nodes instead of all $N=90$ ROIs, the complexity of the dominant cross-node attention term drops from $O(N^2D)$ to $O(|U|^2D)$.","core_discovery":"The paper's central claim is that the brain regions driving neurodegenerative pathology can be found cheaply enough to be used as a filter rather than as a post-hoc explanation. BrainMAP defines candidate subsystems from the AAL atlas, scores each one by random-forest accuracy on region-pooled features, and keeps the union of subsystems that beat the whole-brain baseline; the resulting subgraph is the only graph the model sees. On the retained nodes, cross-node attention aligns each fMRI parcel with DTI tracts, a per-node gate blends the modalities, and agreement and disparity terms are concatenated with the gated mixture into a 5D-per-node embedding that is distilled by SVD before a lightweight GCN classifies it. With this pipeline the paper reports 82.3% ACC / 91.5% AUC on ADNI and 86.2% ACC / 93.8% AUC on PPMI, and credits the subgraph filter with reducing memory by over 45% and runtime by over 25% relative to its own no-filter variant, while the fusion module is what preserves accuracy.","pith_inferences":["Beyond the paper, the localization mechanism can be tested independently of the fusion design: if randomly chosen equal-sized ROI subsets match BrainMAP's accuracy, then the atlas scoring, not the subgraph content, is the source of the gain.","Beyond the paper, the same atlas-scoring procedure transfers to other brain atlases, other modalities, or regression targets such as cognitive scores, since its only requirement is a labeled signal per candidate ROI.","Beyond the paper, the reported per-epoch times imply the remaining bottleneck is the $O(|U|^2D)$ cross-node attention, so scaling to larger or smaller graphs would need sparse attention or node sampling, a follow-up the paper does not address."],"forward_implications":["Multimodal brain-graph diagnosis can run orders of magnitude faster than whole-brain multimodal baselines, with ADNI per-epoch time at 0.0243 s and PPMI at 0.0023 s.","The same atlas-scoring procedure doubles as a localization output: on ADNI it highlights subcortical, limbic, olfactory, and auditory regions, consistent with early Alzheimer's involvement.","Removing the subgraph filter leaves accuracy essentially unchanged on ADNI but more than doubles memory use; removing the fusion module drops accuracy by about 10 points, so the two modules separate the efficiency and accuracy contributions.","Because the fusion module computes per-node agreement, disparity, and gating terms, the model itself produces a per-region confidence signal, which supports clinical interpretation beyond a single diagnosis."],"supporting_citations":[{"why":"Supplies the AAL atlas ROI definitions and the subsystem masks (visual, limbic, DMN, etc.) that AGSF scores and unions.","marker":"[10]"},{"why":"Provides the ADNI fMRI/DTI cohort and labels used for the Alzheimer's experiments.","marker":"[11]"},{"why":"Provides the PPMI fMRI/DTI cohort and labels used for the Parkinson's experiments.","marker":"[12]"},{"why":"Supplies the SVD-based feature-distillation step that removes top and bottom loading components before the GCN.","marker":"[19]"},{"why":"Supplies the functional-connectivity extraction step via Pearson correlation on AAL-parcellated fMRI.","marker":"[20]"},{"why":"Supplies the DTI preprocessing and structural-connectivity extraction with fractional anisotropy features.","marker":"[21]"}],"fun_headline_variants":["BrainMAP pinpoints disease regions with 50% less compute","Atlas filter finds disease subgraphs, cutting compute in half","Multimodal graph model targets brain disease, slashes compute","Fusing fMRI and DTI, BrainMAP localizes disease cheaply"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that using the disease labels to select which brain regions to keep does not bias the later five-fold accuracy evaluation of the classifier, because the selection step is described outside the cross-validation loop.","fun_headline_variants_meta":{"raw":{"variants":["BrainMAP pinpoints disease regions with 50% less compute","Atlas filter finds disease subgraphs, cutting compute in half","Multimodal graph model targets brain disease, slashes compute","Fusing fMRI and DTI, BrainMAP localizes disease cheaply"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000655,"raw_usage":{"total_tokens":3019,"prompt_tokens":982,"completion_tokens":2037,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":1964}},"tokens_in":598,"tokens_out":2037,"duration_ms":14979,"temperature":1.0,"reasoning_tokens":1964,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:21:54.249960+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the whole pipeline with atlas-mask selection nested inside each cross-validation fold, so masks are chosen only from training subjects and then evaluated on the held-out fold; if ADNI ACC/AUC falls materially below 82.3%/91.5% or PPMI below 86.2%/93.8%, the headline numbers depend on label information used at selection time rather than on the subgraph mechanism itself.","supporting_citations":[{"cited_title":"Automated anatom- ical labeling of activations in spm using a macroscopic anatomical parcellation of the mni mri single-subject brain,","cited_arxiv_id":null,"evidence_quote":"Supplies the AAL atlas ROI definitions and the subsystem masks (visual, limbic, DMN, etc.) that AGSF scores and unions."},{"cited_title":"Alzheimer’s disease neuroimaging initiative (adni) clinical characterization,","cited_arxiv_id":null,"evidence_quote":"Provides the ADNI fMRI/DTI cohort and labels used for the Alzheimer's experiments."},{"cited_title":"The parkin- son progression marker initiative (ppmi),","cited_arxiv_id":null,"evidence_quote":"Provides the PPMI fMRI/DTI cohort and labels used for the Parkinson's experiments."},{"cited_title":"Adaptive brain network augmentation based on group-aware graph learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the SVD-based feature-distillation step that removes top and bottom loading components before the GCN."},{"cited_title":"Gretna: a graph theoretical network analysis toolbox for imaging connectomics,","cited_arxiv_id":null,"evidence_quote":"Supplies the functional-connectivity extraction step via Pearson correlation on AAL-parcellated fMRI."},{"cited_title":"Panda: a pipeline toolbox for analyzing brain diffusion images,","cited_arxiv_id":null,"evidence_quote":"Supplies the DTI preprocessing and structural-connectivity extraction with fractional anisotropy features."}],"review_version":1}