{"id":"7da55a20-43bf-4684-90c7-43eb43ccdc89","arxiv_id":"2508.06565","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A vision-language model that treats brain subnetworks as image tokens and aligns them with words in clinical reports reaches 92.94% MCI accuracy on ADNI, but label leakage and uncontrolled baselines undermine the claim.","lead":"Researchers built a model that pairs brain wiring diagrams with patient text reports to spot mild cognitive impairment, a possible early Alzheimer's stage. The paper claims very high accuracy and interpretable links, but the text inputs contain test-like scores and the comparison lacks fair control, so the result is not established.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The text modality is built from label-defining variables (MMSE, CDR, biomarkers), so the 92.94% accuracy and connectome-text pairs may be label leakage, not cross-modal learning.","rationale":"The paper's architecture is coherent and the brain-subnetwork-as-token idea is worth exploring, but the experimental setup as described invalidates the central claims. Section 3.1 explicitly builds the text from variables that are part of the MCI definition, so the high accuracy and the interpretability findings may simply reflect the model reading the diagnosis out of the narrative. This is not a disagreement with consensus or a stylistic complaint; it is a correctness risk at the core of the paper. The reader's weakest assumption identified this same leakage problem, and I agree with that assessment. A rerun with redacted text is a concrete, feasible check that would settle whether the multimodal result is real. Table 1's lack of same-split comparisons and missing error bars are secondary issues that would remain after leakage is addressed. Because the reader already rejected the paper, my recommendation leaves the verdict unchanged. I raise no ad hominem concerns; the issue is in the experimental design, not the authors' intent.","tokens_in":800,"tokens_out":819,"duration_ms":54940,"concrete_test":"On the same 418 subjects and the same 80/20 split, rerun the full pipeline with two text conditions: (1) the original narrative as in the paper; (2) a redacted narrative with all outcome-related numeric fields removed (MMSE, CDR, APOE-4, tau, Abeta), keeping only age, sex, education, and any free-text notes. Report text-only and multimodal accuracy for both conditions with confidence intervals. If redaction drops text-only accuracy from 86.96% toward the imaging-only 73.77% and multimodal accuracy no longer exceeds the redacted text-only baseline, the reported 92.94% result is attributable to label leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.1 states that clinical narratives are generated from demographic information, biomarker measurements (APOE-4), cognitive assessments (MMSE, CDR), and doctors' notes. Because MCI status in ADNI is operationally defined in part by MMSE and CDR scores, the text input can encode the label before imaging is used. The paper's own Table 2 shows text-only reaches 86.96% accuracy, close to the 92.94% multimodal result, so the multimodal gain could be a small increment on a label-leaking text feature rather than evidence of meaningful fusion. The claimed 'clinically meaningful connectome-text pairs' in Figs. 3-4 are also suspect: tokens such as phosphorylated_tau, aβ42, MMSE, and CDR are copied from the same structured fields that define the label, so the cross-modal attention may simply be retrieving diagnostic variables from the narrative. This is the load-bearing premise: the central claims of state-of-the-art prediction and clinically meaningful cross-modal associations require the text and imaging to be at least partially independent evidence. That premise fails by the paper's own description of how the reports were constructed.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a multimodal framework for MCI diagnosis that aligns DTI-derived structural connectomes with clinical report text. Brain subnetworks (rows of the SC matrix) are treated as tokens and encoded by a ViT; clinical reports are encoded by BERT; cross-modal attention is applied at both the subject level and the connectome level. On ADNI data (301 NC, 117 MCI), the method reports 92.94% accuracy, with text-only and imaging-only ablations at 86.96% and 73.77%, respectively. The paper also presents visualizations of top brain-subnetwork/text-token pairs and claims state-of-the-art performance and clinically meaningful multimodal associations.","tokens_in":8545,"tokens_out":5003,"duration_ms":60216,"significance":"If the reported results were valid, the paper would make a useful contribution: the idea of treating brain subnetworks as tokens for connectome-text alignment is a sensible inductive bias, the method is clearly described, and the code is stated to be public. However, the central evidence is undermined by a likely label-leakage problem. Section 3.1 states that clinical narratives are generated from demographic information, biomarker measurements (APOE-4), cognitive assessments (MMSE, CDR), and doctors' notes. Since MCI status in ADNI is operationally defined in part by MMSE and CDR scores, the text modality can encode the outcome before imaging is used. The 86.96% text-only accuracy in Table 2 is close to the 92.94% multimodal accuracy, suggesting the multimodal gain may be a small increment on a label-leaking text feature rather than meaningful cross-modal fusion. The interpretability findings in Figs. 3-4 are also largely circular because tokens such as phosphorylated_tau, aβ42, hippocampus, and entorhinal_cortex are copied from the same structured fields that define the label. These concerns affect the paper's two central claims, so the work currently does not support its stated","major_comments":[{"comment":"Label leakage: the text modality is assembled from MMSE, CDR, APOE-4, and CSF biomarker values, which are the same variables used to define the MCI/NC label in ADNI. The text-only C-only row reaches 86.96% accuracy, only 6 percentage points below the multimodal result. Under these conditions, the 92.94% accuracy and the claimed benefit of cross-modal alignment may simply reflect the model reading the diagnosis from structured fields inserted into the report. Please rerun the experiments with reports redacted to remove all label-proximate variables (or at least show that text-only performance on redacted doctors' notes is near chance), and report whether the multimodal advantage persists.","section":"Section 3.1, Table 2"},{"comment":"The 'clinically meaningful connectome-text pairs' are largely circular. Tokens such as phosphorylated_tau, aβ42, hippocampus, entorhinal_cortex, MMSE, and CDR are field names or structured values that the authors explicitly wrote into the clinical narratives. Recovering associations between these tokens and known AD-related subnetworks is expected by construction and does not provide independent evidence that the model learned medical knowledge. Please add control analyses: for example, permute token-to-subnetwork assignments, remove all biomarker/cognitive phrases, or compare against a bag-of-words model trained on the same narratives. Without such controls, the interpretability claims in the abstract and conclusion are unsupported.","section":"Section 3.2-3.3, Figs. 3-4"},{"comment":"The state-of-the-art claim is not supported by the comparison in Table 1. The cited studies use different datasets, cohort sizes, modalities, and evaluation splits; no statistical significance tests or confidence intervals are reported. The 92.94% accuracy on 84 test subjects (20% of 418) has a wide confidence interval, so the comparison is not meaningful. Please include same-data baselines under identical train/test splits, repeated cross-validation, and confidence intervals or significance tests.","section":"Section 3.4, Table 1"},{"comment":"The experimental protocol is not sufficiently robust for the strength of the claims. A single 80/20 split with 418 subjects yields a test set of roughly 84 subjects; the difference between 90.59% and 92.94% in the alignment ablation is within sampling noise. There is no mention of repeated splits, seeds, class-balancing details, or external validation. Please report mean and standard deviation over multiple runs or cross-validation folds, and perform a paired significance test for the ablation comparisons.","section":"Section 3.1, Table 2"}],"minor_comments":[{"comment":"Reference numbering is inconsistent: reference [16] is listed as both 'Zhang et al. (2025) rsfMRI' and 'Song et al. (2024) DTI+record'. The row in category (C) labeled 'Song et al. (2024) [16]' appears to be a citation error.","section":"Table 1"},{"comment":"The symbols ℒ!\" and ℒ#\" in the caption are garbled; the text uses different notation for connectome-level and subject-level losses. Please standardize the notation.","section":"Table 2 caption"},{"comment":"Several loss equations have malformed parentheses, e.g., L_ct2i and L_ci2t contain unmatched parentheses and an unusual sum over softmax terms. The formulas should be rewritten in standard notation.","section":"Section 2.2"},{"comment":"The figure is overcrowded and several labels are difficult to read, especially in the clinical report encoder and the cross-attention blocks. Please enlarge fonts and separate the alignment schematic from the architecture diagram.","section":"Figure 2"},{"comment":"The statement 'Code is publicly available' has no URL. If the code is available, please include a link; if not, remove the sentence.","section":"Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The core architectural idea is worth considering, but the current evidence is invalidated by the construction of the text input. The authors should be asked to demonstrate that their results are not an artifact of label leakage, e.g., by evaluating on reports that exclude MMSE/CDR/biomarker values and by adding permutation controls for the interpretability analysis. If those experiments do not preserve the reported advantage, the paper should not be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read of arXiv:2508.06565. The core idea—treating each brain subnetwork as a token and aligning at both subject and connectome levels with clinical text—is genuinely new relative to the cited work, and the method is straightforward to follow. The authors earn credit for a clean formulation and for reporting a text-only ablation, which is where the trouble shows.\n\nThe soft spot is not in the math but in the data construction. Section 3.1 says the clinical narratives are assembled from demographic info, APOE-4, MMSE, CDR, and doctors' notes. MCI status in ADNI is operationally defined in part by MMSE and CDR, so the text input already encodes the label. That explains why text-only hits 86.96% accuracy, and the fusion's 92.94% is a small increment on a leaky feature. The interpretability results are also circular: the top tokens are the very fields (phosphorylated_tau, aβ42, MMSE) that were concatenated into the input, so recovering their association with known AD regions is not a discovery. Table 1 compares across different cohorts and splits, with no error bars, so the 'state-of-the-art' claim is unsupported.\n\nThe fix is clear: strip outcome-related fields from the text, add same-subject/split baselines, and release the code (the link is missing despite the claim). If the method works with text that doesn't explicitly contain MMSE/CDR, it would be a real result. As written, the central claims do not survive.\n\nThat said, the architecture itself is thoughtfully designed and may be worth exploring. I'd send this to review, with the expectation of major revision, because the flaw is likely fixable and the subnetwork-token idea deserves a fair test. For my own work, I won't cite it yet.","headline":"The architecture is a plausible novelty, but the evaluation leaks the label through the text, so the headline numbers are not credible as written.","tokens_in":9089,"tokens_out":2281,"would_cite":false,"duration_ms":24698,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Treating brain subnetworks as tokens aligned with clinical report words yields 92.94% accuracy in distinguishing mild cognitive impairment from controls and recovers tau-related connectome-text associations.","keywords":["medical vision-language model","multimodal learning","mild cognitive impairment","brain connectome","DTI","cross-modal alignment","Alzheimer's disease","interpretability"],"falsifier":"Retrain the model on the same reports with all cognitive-scale values (MMSE, CDR) and outcome-defining biomarker fields removed, keeping only the doctors' narrative wording. If accuracy falls to roughly the imaging-only level (73.77%) or the text-only advantage disappears, the cross-modal claim is not supported; if it stays near 92.94%, the alignment is genuinely using independent clinical semantics.","tokens_in":8086,"feed_emoji":"🧠","tokens_out":7667,"duration_ms":83285,"temperature":0.7,"pith_summary":"This paper claims that combining a DTI brain connectome with a generated clinical narrative can diagnose mild cognitive impairment more accurately than imaging-only models, while also producing interpretable links between brain networks and clinical vocabulary. Its central move is to treat each brain region's connectivity profile as a token, like a word or image patch, and align those tokens with BERT word tokens from the clinical text in a shared latent space at two levels: whole-subject and fine-grained connectome-level. The authors report 92.94% accuracy on MCI versus normal controls, with ablations showing that removing either modality or either alignment level lowers performance, and they present recovered pairs such as phosphorylated_tau with cingulate subnetworks as evidence that the alignments are clinically meaningful. If correct, this gives clinicians a way to turn routine reports and a single diffusion scan into both a prediction and a map of which text observations go with which brain systems.","feed_headline":"Brain subnetworks as tokens hit 92.94% MCI accuracy","feed_subtitle":"Aligning connectome tokens with words in clinical reports tops imaging-only models and surfaces tau-related brain-text links.","key_machinery":"The load-bearing object is the brain sub-network token: the row of the structural-connectivity matrix that records one brain region's fiber connectivity to every other region, making network-level rather than voxel-level information the unit of alignment. A vanilla ViT encoder treats these tokens like image patches; a BERT encoder treats clinical-report words as tokens; and bidirectional cross-attention between the two token sets produces similarity scores that are trained with weighted softmax contrastive losses at connectome level and InfoNCE-style losses at subject level. The category-level claim is carried by this two-level alignment plus the classification head.","core_discovery":"The discovery claimed is that brain subnetworks can serve as the imaging-side tokens in a vision-language alignment, and that aligning them with clinical-report word tokens yields better MCI classification and clinically interpretable pairs. From the structural connectivity matrix $SC \\in \\mathbb{R}^{N\\times N}$, each row is treated as a sub-network token; a learnable class token aggregates all rows, and the sequence is fed to a vanilla ViT. A BERT encoder produces token- and report-level text representations. Bidirectional cross-attention between sub-network tokens and word tokens yields a connectome-level cosine-similarity matrix, while a subject-level InfoNCE-style loss matches global bra","pith_inferences":["The paper does not isolate whether the clinical narrative's cognitive scores are necessary for the gain; an extension would retrain with reports stripped of MMSE, CDR, and biomarker fields to measure how much of the 92.94% comes from genuinely independent text semantics.","If the connectome-token recipe ports to other modalities, the framework could be used to mine large report repositories for disease-specific network terminology without manual annotation.","The reported tau-cingulate and aβ42-parietal pairs could be treated as priors for targeted ROI analyses, effectively turning the model's attention weights into hypotheses about where early pathology appears before atrophy is visible."],"forward_implications":["If the accuracy claim holds, DTI plus a text report is enough to match or beat multi-imaging fusion, which lowers the acquisition burden for early AD screening.","The recovered connectome-text pairs (tau, aβ42, MMSE, CDR tokens mapped to cingulate and parietal subnetworks) give concrete region-versus-measure hypotheses that clinical studies could test prospectively.","Treating connectome rows as tokens offers a general recipe: the same alignment machinery can be applied to functional connectivity, morphometry, or PET-derived networks with the same report encoder.","Ablation results suggest both alignment levels matter, meaning fine-grained subnetwork-to-word correspondences carry information beyond whole-subject matching."],"supporting_citations":[{"why":"Supplies the standard DTI preprocessing and fiber-tracking pipeline that produces the structural-connectivity input.","marker":"[4]"},{"why":"Provides prior evidence that MCI involves grey-matter and white-matter changes in parietal and posterior cingulate cortex, which the recovered pairs are said to match.","marker":"[5]"},{"why":"Establishes that tau deposition in cingulate and precuneus regions tracks cognitive decline, validating the phosphorylated_tau-to-cingulate association.","marker":"[9]"},{"why":"Best-performing multi-imaging baseline (DTI+rsfMRI, 92.70% accuracy) that the method must beat to claim state-of-the-art.","marker":"[19]"},{"why":"Recent multi-imaging deep-fusion baseline (92.30%) used as comparison in the MCI prediction table.","marker":"[20]"},{"why":"Imaging-plus-record multimodal baseline from a large dementia study, representing the competing imaging-non-imaging approach.","marker":"[22]"},{"why":"Recent imaging-plus-text model that is the closest methodological relative and the strongest text-augmented baseline.","marker":"[23]"}],"fun_headline_variants":["Connectome tokens + clinical words beat imaging-only MCI models","Treating brain subnetworks as tokens improves MCI detection","Aligning connectome tokens with clinical notes achieves 92.94% MCI","Subnetwork tokens align with clinical words for early AD diagnosis","Connectome-text pairs tie brain networks to clinical notes"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that the clinical narratives do not already encode the diagnostic outcome, even though they are generated from MMSE and CDR scores that are partly used to define MCI, so the text and imaging are treated as independent evidence.","fun_headline_variants_meta":{"raw":{"variants":["Connectome tokens + clinical words beat imaging-only MCI models","Treating brain subnetworks as tokens improves MCI detection","Aligning connectome tokens with clinical notes achieves 92.94% MCI","Subnetwork tokens align with clinical words for early AD diagnosis","Connectome-text pairs tie brain networks to clinical notes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.002438,"raw_usage":{"total_tokens":9196,"prompt_tokens":730,"completion_tokens":8466,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":474,"completion_tokens_details":{"reasoning_tokens":8380}},"tokens_in":474,"tokens_out":8466,"duration_ms":61548,"temperature":1.0,"reasoning_tokens":8380,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T23:39:24.054351+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the model on the same reports with all cognitive-scale values (MMSE, CDR) and outcome-defining biomarker fields removed, keeping only the doctors' narrative wording. If accuracy falls to roughly the imaging-only level (73.77%) or the text-only advantage disappears, the cross-modal claim is not supported; if it stays near 92.94%, the alignment is genuinely using independent clinical semantics.","supporting_citations":[{"cited_title":"Predicting brain structural network using functional con-nectivity[J]","cited_arxiv_id":null,"evidence_quote":"Supplies the standard DTI preprocessing and fiber-tracking pipeline that produces the structural-connectivity input."},{"cited_title":"Parietal cortex matters in Alzheimer's disease: an overview of structural, functional and metabolic findings[J]","cited_arxiv_id":null,"evidence_quote":"Provides prior evidence that MCI involves grey-matter and white-matter changes in parietal and posterior cingulate cortex, which the recovered pairs are said to match."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes that tau deposition in cingulate and precuneus regions tracks cognitive decline, validating the phosphorylated_tau-to-cingulate association."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Best-performing multi-imaging baseline (DTI+rsfMRI, 92.70% accuracy) that the method must beat to claim state-of-the-art."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Recent multi-imaging deep-fusion baseline (92.30%) used as comparison in the MCI prediction table."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Imaging-plus-record multimodal baseline from a large dementia study, representing the competing imaging-non-imaging approach."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Recent imaging-plus-text model that is the closest methodological relative and the strongest text-augmented baseline."}],"review_version":1}