{"id":"442a4524-2796-46b8-a093-c10654b85aab","arxiv_id":"2412.06212","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A mask-guided multimodal GNN that fuses brain connectomes with PubMed abstract embeddings is claimed to improve AD classification and interpretability, but the evidence is undermined by inconsistent ablations and missing controls.","lead":"This paper proposes a multimodal graph neural network that augments brain connectome data with text embeddings of 20,000 PubMed abstracts on Alzheimer's disease, using learned masks to highlight important brain regions and knowledge items and to guide fine-tuning. It reports accuracy gains over plain GNNs on two AD cohorts, but the experimental evidence has serious gaps, including an ablation table that contradicts the stated benefit of more knowledge.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim that AD-specific domain knowledge drives the gains is untested: no control text corpus, no error bars, and the knowledge-size ablation is non-monotonic.","rationale":"The reader's weakest_assumption is the same one I would stress-test: gains are attributed to AD-specific knowledge content, but the experimental design never isolates content from added model capacity. The paper's own Table 2 provides internal evidence that knowledge size does not behave monotonically: in multiple cells, 1% or 10% knowledge beats 100% (e.g., MM-GINE on OASIS DTI ACC 0.8221 vs 0.7521; MM-GAT on OASIS fMRI ACC 0.8221 vs 0.7632). This makes the capacity/regularization confound more plausible and reinforces the need for a content-control experiment. The paper does have useful components: the masked multimodal architecture is clearly described, and the qualitative mask analyses are suggestive, but qualitative observations of distinct male/female patterns are not validated against expert-curated ground truth, so they do not substitute for the missing quantitative control. I found no basis to soften the reader's rejection: the central empirical claim is currently unsupported. My recommendation is UNCHANGED because my independent read reaches the same verdict for essentially the same reason, and the concrete test described above would settle the concern.","tokens_in":12920,"tokens_out":2119,"duration_ms":24213,"concrete_test":"Run the same MM-GNN pipeline on OASIS and ADNI-D with three text corpora matched in size and length distribution: (a) the 20,108 AD PubMed abstracts, (b) 20,108 non-AD PubMed abstracts randomly sampled from other biomedical topics, and (c) token-shuffled versions of the AD abstracts to destroy semantics while preserving token statistics. Use identical architecture, training schedule, and at least 10 random seeds per condition; report mean and standard deviation of ACC, AUC, and F1, and run paired significance tests (e.g., paired t-test or Wilcoxon) comparing (a) with (b) and (c). If condition (a) does not significantly outperform both controls on the same metrics, the claim that AD-specific domain knowledge drives the gains is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's load-bearing assertion in Section 3 is that injecting domain knowledge significantly improves GNN performance. The only comparison is MM-GNN against vanilla GNN (Table 1), but MM-GNN adds a BERT encoder, an MLP, a second fusion GNN, and a mask-based fine-tuning stage. Any of these components could explain the accuracy gains independently of whether the injected text is AD-specific. The knowledge-ablation in Table 2 makes the attribution even less secure: for MM-GINE on OASIS (DTI), ACC at 1% knowledge (0.8221) exceeds ACC at 10% (0.8184) and at 100% (0.7521); similar inversions appear elsewhere. That pattern is inconsistent with a simple 'more relevant knowledge helps' mechanism and points instead to noise, capacity, or regularization effects. No standard deviations or repeated-seed statistics are reported, so we cannot distinguish real improvements from variance. Because the central claim is specifically about AD-related knowledge, not about having an extra neural branch, the absence of a content-matched control is the decisive gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a self-guided multimodal graph neural network (MM-GNN) for Alzheimer's disease (AD) classification that fuses brain connectome graphs with uncurated PubMed abstracts. A backbone GNN encodes the graph, a pretrained language model (BERT-Large) followed by an MLP encodes knowledge items, and a fusion GNN combines the two modalities to produce predictions. The method then learns data and knowledge masks via a Gumbel-Softmax parameterization, using them as explanations and to guide edge-sampling-based graph augmentation for fine-tuning. Experiments on OASIS and ADNI-D datasets with DTI and fMRI modalities compare MM-GNN and a fine-tuned variant (MM-GNN-F) against vanilla GCN, GINE, and GAT baselines, reporting accuracy, AUC, and F1. The paper also presents qualitative saliency maps and an ablation on the size of the knowledge set. The central claim is that injecting AD-specific domain knowledge significantly improves GNN performance and interpretability.","tokens_in":13163,"tokens_out":3465,"duration_ms":35221,"significance":"If the central claim were established, the framework would be a meaningful step toward replacing human-expert-guided knowledge injection with automatic retrieval from raw, uncurated literature, with potential value for brain-connectome analysis and explainable medical decision support. The method is clearly described, the multimodal architecture is reasonable, and the idea of using learned masks for both explanation and augmentation is coherent. However, the empirical support for the central claim is currently inadequate: the comparisons conflate knowledge content with additional model capacity, no uncertainty quantification is provided, and the knowledge-size ablation is non-monotonic in a way that contradicts the paper's own interpretation. The paper does not report code or data release, which limits reproducibility.","major_comments":[{"comment":"The central claim that 'the performance can be improved significantly when the domain knowledge is injected into the inference of GNN' (Section 3, Main Results) is not supported by the experimental design. MM-GNN differs from the vanilla GNN baselines not only by the injected knowledge but also by the addition of a BERT-Large encoder, an MLP, a fusion GNN, and a mask-guided fine-tuning stage. Without a control that replaces the AD-specific PubMed abstracts with non-AD or randomly chosen text, or that removes the text modality while keeping the extra capacity, the observed accuracy gains cannot be attributed to AD-specific domain knowledge. The current comparisons only demonstrate that a more complex model outperforms a simpler one.","section":"Section 3, Table 1 and Main Results"},{"comment":"Table 2 contradicts the paper's conclusion that increasing the size of domain knowledge to a large scale provides a noticeable boost. For MM-GINE on OASIS (DTI), ACC at 100% knowledge is 0.7521, while at 10% it is 0.8184 and at 1% it is 0.8221. For MM-GINE on OASIS (fMRI), ACC decreases monotonically from 0.8221 (1%) to 0.7644 (10%) to 0.7558 (100%). The text states that 'the benefits from a larger set of domain knowledge become obvious when increasing the size of domain knowledge to more than 20,000 (100%)', but these rows show the opposite trend. This non-monotonic pattern suggests that the differences may be due to noise, regularization effects, or capacity, rather than a knowledge-driven mechanism, and it directly undermines the load-bearing assertion that AD-specific knowledge content is what drives the improvements.","section":"Section 3, Table 2 (Ablation Study on the Size of Domain Knowledge)"},{"comment":"No standard deviations, confidence intervals, repeated runs, or significance tests are reported anywhere in the paper. Every metric is a single point estimate. Several reported improvements are small (for example, MM-GCN-F versus MM-GCN on OASIS fMRI ACC: 0.7436 versus 0.7387), and given the absence of variance information, the reader cannot determine whether any of the differences are statistically meaningful. This is a load-bearing issue for the paper's central empirical claim, and it applies to both the main results and the knowledge-size ablation.","section":"All experimental tables (Tables 1 and 2)"}],"minor_comments":[{"comment":"The abstract uses 'interoperability' where 'interpretability' is clearly intended; the same wording appears in Section 1 ('enhance performance and interoperability'). Please correct this typo.","section":"Abstract and Section 1"},{"comment":"There is a typo: 'multimodel GNN' should be 'multimodal GNN'.","section":"Section 3, Datasets and Settings"},{"comment":"The caption states that color is used only to distinguish ROIs, but the figure is described as a saliency map; please clarify how the saliency values are encoded visually, since the current caption makes the figure's message ambiguous.","section":"Figure 3 caption"},{"comment":"The observations about the knowledge-wise masks are purely qualitative, with no quantitative metric such as overlap with known AD-related literature or downstream task performance of the selected knowledge. Adding such a metric would strengthen the interpretability claims.","section":"Section 3, Qualitative Analysis on Knowledge-wise Masks"},{"comment":"The loss and the fine-tuning procedure depend on hyperparameters λ1 through λ4, the temperature τ, and the threshold T, but no sensitivity analysis or discussion of their selection is provided. Reporting the chosen values and a small robustness study would improve reproducibility.","section":"Section 2, Eq. (1)"}],"recommendation":"reject","confidential_remarks":"The paper's central claim—that AD-specific domain knowledge, rather than the added neural components, drives the reported gains—is not supported by the current experiments. The missing control with non-AD text and the absence of any error bars or significance tests would already be sufficient for a major revision, but Table 2's non-monotonic knowledge-size ablation actively contradicts the stated conclusion. In my view, the empirical evidence as presented cannot justify the claimed contribution, and the required fixes (control experiments, repeated-seed statistics, and a corrected, honest analysis of the ablation) go beyond local revision. The paper may be better positioned as a workshop paper or as a preliminary study once those experiments are performed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a coherent framework and the application is timely, but the headline claim—that injecting AD-specific literature into a GNN improves classification—is not actually tested. The experiments only compare MM-GNN against vanilla GNN; they never swap in non-AD text, so any of the added components (BERT branch, MLP, fusion GNN, mask fine-tuning) could explain the gains. That is the load-bearing gap, and the stress-test note is right to flag it.\n\nWhat is new: the specific combination—a fusion graph over BERT-encoded PubMed abstracts, Gumbel-Softmax masks for graph/knowledge importance, and mask-guided edge sampling for fine-tuning—is a fresh configuration for brain connectome classification. The method section is clear and detailed. The qualitative mask analysis, while not validated against known biomarkers, at least shows the mechanism produces structured outputs. The assembled 20k-abstract PubMed corpus for AD is a useful resource on its own.\n\nSoft spots, in order of severity. First, no control corpus. Without a text branch fed with matched but non-AD abstracts, the central claim is untestable. Second, Table 2 undermines the knowledge-size narrative: for MM-GINE and MM-GAT on several settings, 1% knowledge beats 100% (e.g., OASIS DTI ACC 0.8221 vs 0.7521 for MM-GINE). The text tries to spin this as \"benefits become obvious at 100%,\" but the numbers do not show that. Third, no error bars, confidence intervals, or significance tests, so we cannot tell which differences are real. Also absent: any comparison to GNN baselines with matched parameter counts, or to existing knowledge-infusion methods.\n\nThe novelty score of 5 is about right. The idea is worth exploring, but the empirical support is not there yet. I would send it to a serious referee because the method is clearly specified and the central question is addressable with a few added experiments. A revision that adds a non-AD text control, repeated-seed statistics, and an honest reading of the ablation would put the claim to a real test. As it stands, this is a promising work-in-progress, not a demonstrated result.\n\nRecommendation: engage with it, but require the missing controls before accepting anything.","headline":"Interesting framework, but the central claim that AD-specific knowledge drives the gains is untested and partially contradicted by the paper's own ablation.","tokens_in":13695,"tokens_out":2538,"would_cite":false,"duration_ms":24425,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that injecting uncurated PubMed abstracts into a multimodal graph neural network improves Alzheimer's disease classification and yields interpretable brain-region masks.","keywords":["graph neural networks","multimodal learning","Alzheimer's disease","domain knowledge injection","brain connectome","model interpretability","retrieval-augmented generation","graph augmentation"],"falsifier":"Run the same multimodal GNN with a control knowledge set of, say, 20,000 abstracts from unrelated biomedical fields or shuffled AD abstracts; if classification accuracy and the learned masks stay essentially unchanged, the central claim that AD domain knowledge drives the improvement would be refuted.","tokens_in":12753,"feed_emoji":"🧠","tokens_out":5281,"duration_ms":51641,"temperature":0.7,"pith_summary":"The paper claims that Alzheimer's disease classification with graph neural networks can be improved by feeding the model uncurated domain knowledge, namely titles and abstracts of about 20,000 PubMed papers, without any human expert curating the knowledge. The proposed model fuses the brain-connectome graph with language embeddings of those abstracts, learns two masks that mark which brain edges and which knowledge items matter for each prediction, and then fine-tunes itself by augmenting the graph according to the masks. On the OASIS and ADNI-D datasets, the multimodal GNN outperforms the plain GNN baselines, with relative gains of roughly 10–24% in ACC, AUC, and F1 on ADNI-D. The motivation is that expert-guided design of GNNs is slow and scarce, so a self-guided pipeline that reads literature directly could scale to other disorders.","feed_headline":"Raw PubMed text lifts GNN accuracy on Alzheimer's diagnosis","feed_subtitle":"A graph AI reads 20,000 abstracts automatically and learns which brain regions matter, no expert needed.","key_machinery":"The load-bearing object is a pair of learnable masks, one on the graph edges ($M_d$, parameterized by $\\alpha$) and one on the fusion-graph edges between the brain node and knowledge nodes ($M_k$, parameterized by $\\beta$), sampled with Gumbel-Softmax so the whole pipeline stays differentiable. The fusion graph is the bridge: a backbone GNN $f_B$ provides the graph embedding $E_d^G$, a pretrained language model followed by an MLP provides knowledge embeddings $E_K^i$, and the fusion GNN $f_F$ classifies from the graph that connects them. After pretraining, the masks are optimized with a loss that balances prediction consistency, label fit, sparsity, and discreteness, and the learned mask values then drive an edge-sampling augmentation to fine-tune the model.","core_discovery":"In the paper's own terms, the central discovery is that domain knowledge can be represented as natural language and injected into a GNN through a fusion graph: a backbone GNN embedding of the brain graph is connected to language-model embeddings of knowledge items, and a fusion GNN classifies from this combined graph. The model learns two real-valued masks, one over graph edges and one over knowledge edges, that act as explanations for both graph-wise and knowledge-wise importance, and it uses those masks to guide a graph-augmentation fine-tuning step. The authors show that this procedure improves ACC, AUC, and F1 over vanilla GNNs on DTI and fMRI versions of both OASIS and ADNI-D, and that the learned masks produce salient-ROI maps that differ by sex and by GNN backbone.","pith_inferences":["Editorial extension: the paper does not compare against a multimodal GNN with a non-AD or scrambled text corpus, so the attribution of gains to AD-specific knowledge content remains untested; a control-text experiment would clarify whether the improvement is content-driven or capacity-driven.","Editorial extension: the knowledge-mask distributions in the paper show that most abstracts are downweighted, suggesting the model is performing a form of implicit retrieval; a direct comparison with sparse retrieval or retrieval-augmented generation over the same corpus could isolate the value of the learned soft retrieval mechanism.","Editorial extension: the sex-specific mask differences are observational; linking them to external neuroimaging or genetic biomarkers of AD would be a natural next test, as would checking whether the masks generalize across cohorts.","Editorial extension: the knowledge-size ablation hints at a critical mass of knowledge needed before gains appear; testing intermediate sizes between 10% and 100% could reveal where the transition occurs."],"forward_implications":["If the central claim holds, raw literature can serve as a knowledge source for GNN-based diagnosis, reducing the need for teams of human experts to manually tailor models.","The learned graph masks could provide automatic, dataset-specific explanations of which brain regions drive AD versus MCI classification, and the reported sex-specific differences offer a testable hypothesis about sex-linked AD biomarkers.","The ablation showing that the full 20,000-record knowledge set helps more than 200- or 2,000-record subsets suggests that scaling uncurated literature input may further improve performance.","The same self-guided multimodal recipe could transfer to other graph-based medical prediction tasks where a body of literature is available."],"supporting_citations":[{"why":"Supplies the GCN backbone architecture that the multimodal GNN extends and that serves as a vanilla baseline.","marker":"[24]"},{"why":"Supplies the GAT backbone architecture used in the experiments and as another vanilla baseline.","marker":"[44]"},{"why":"Supplies the GINE backbone architecture used in the experiments and as another vanilla baseline.","marker":"[63]"},{"why":"Provides the pretrained BERT-Large language model used to encode the PubMed abstracts as knowledge embeddings.","marker":"[9]"},{"why":"Introduces the Gumbel-Softmax reparameterization that makes the mask-based edge sampling differentiable.","marker":"[20]"},{"why":"Provides the OASIS brain imaging dataset used for evaluation.","marker":"[26]"},{"why":"Provides the ADNI-D brain imaging dataset used for evaluation.","marker":"[31]"},{"why":"Inspires the retrieval-augmented design by showing how retrieved knowledge can be integrated into a model's inference.","marker":"[27]"}],"fun_headline_variants":["Self-guided graph AI reads PubMed abstracts to sharpen Alzheimer's diagnosis","No expert needed: GNN learns from abstracts to boost Alzheimer's diagnosis","Knowledge-infused GNN mines medical papers, improves Alzheimer's predictions","Graph AI self-infuses PubMed knowledge to enhance Alzheimer's accuracy","Alzheimer's graph AI learns from raw abstracts, no domain expert required"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The improvement over plain graph neural networks comes from the Alzheimer's-specific content of the injected text, not just from the extra model capacity of the new text-processing branch.","fun_headline_variants_meta":{"raw":{"variants":["Self-guided graph AI reads PubMed abstracts to sharpen Alzheimer's diagnosis","No expert needed: GNN learns from abstracts to boost Alzheimer's diagnosis","Knowledge-infused GNN mines medical papers, improves Alzheimer's predictions","Graph AI self-infuses PubMed knowledge to enhance Alzheimer's accuracy","Alzheimer's graph AI learns from raw abstracts, no domain expert required"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000243,"raw_usage":{"total_tokens":1527,"prompt_tokens":944,"completion_tokens":583,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":492}},"tokens_in":560,"tokens_out":583,"duration_ms":6098,"temperature":1.0,"reasoning_tokens":492,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:53:30.523612+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same multimodal GNN with a control knowledge set of, say, 20,000 abstracts from unrelated biomedical fields or shuffled AD abstracts; if classification accuracy and the learned masks stay essentially unchanged, the central claim that AD domain knowledge drives the improvement would be refuted.","supporting_citations":[{"cited_title":"MedRxiv pp","cited_arxiv_id":null,"evidence_quote":"Provides the OASIS brain imaging dataset used for evaluation."},{"cited_title":"Biological psychiatry 89(8), 757–765 (2021)","cited_arxiv_id":null,"evidence_quote":"Provides the ADNI-D brain imaging dataset used for evaluation."},{"cited_title":"Advances in Neural Information Processing Systems 33, 9459–9474 (2020)","cited_arxiv_id":null,"evidence_quote":"Inspires the retrieval-augmented design by showing how retrieved knowledge can be integrated into a model's inference."}],"review_version":1}