Pith. sign in

REVIEW 4 major objections 4 minor 66 references

Agentic LLM loop beats direct prompting on connectome classification.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 05:49 UTC pith:RXT5KNUE

load-bearing objection Novel and honest framework, but GFM test-set leakage and pass@k make the headline results untrustworthy. the 4 major comments →

arxiv 2607.22082 v1 pith:RXT5KNUE submitted 2026-07-24 cs.MA

When Language Models Meet NeuroGraphs: Exploring Enhanced Agentic LLM Framework Towards Brain Network Analysis

Jiaxing Li , Rui Dong , Muyao Tang , Youyong Kong This is my paper
classification cs.MA
keywords brain network analysisagentic LLMconnectome classificationretrieval-augmented generationreflection mechanisminterpretable reasoningrs-fMRIgraph foundation model
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

BrainAgent is an inference-time framework that turns connectome classification into an agentic loop: the LLM first reads a functional brain network as sorted edge triplets plus computed topology descriptors, then queries neuroscience knowledge and similar labeled cases, then reasons and finally reflects on its own verdict before outputting prediction, rationale, and confidence. The paper's central claim is that this training-free workflow consistently improves accuracy and precision across four public rs-fMRI datasets (ASD, ADHD, gender, MDD) and across six different LLM backbones, compared to direct prompting, Chain-of-Thought, and Reflection baselines. The authors also argue that the framework reduces the over-prediction bias that general LLMs show in medical classification, and yields more structured, verifiable explanations at region, subgraph, and graph levels. A sympathetic reader would care because it suggests that small open-source models, not just frontier systems, can be turned into usable, explainable tools for brain-network analysis without any fine-tuning.

Core claim

The paper's central discovery, stated in its own terms, is that an LLM can act as a reliable predictor and analyst for brain networks if it is embedded in a brain-specific agentic loop rather than asked to reason directly over serialized graph text. BrainAgent converts each connectome into compact multi-level descriptors, augments the LLM's context with retrieved neuroscience knowledge (HARK) and similar subject cases (CARD), and appends a reflection stage that verifies the initial prediction against the graph evidence and retrieved knowledge. Across four rs-fMRI benchmarks—autism, ADHD, gender classification, depression—and six backbone LLMs, this workflow consistently improves accuracy and

What carries the argument

The central mechanism is the BrainAgent loop: a Think–Report–Action–Observation cycle in which the LLM receives a serialized connectome enhanced with analytical descriptors (region, subgraph, graph-level), routes calls to graph-analysis functions, retrieves domain knowledge and similar cases, and then reflects on its output. The case-retrieval component (CARD) blends graph embeddings from a pretrained brain-graph encoder with text embeddings of LLM-generated case reports; the knowledge component (HARK) does two-stage anatomical/semantic retrieval. These modules work together to ground the LLM's reasoning in subject-specific evidence and to catch unsupported or overconfident claims.

Load-bearing premise

The case-retrieval encoder is pretrained on all 4,760 graphs in the four datasets (Appendix G, Table 5), while the paper only excludes the 20% test split from the retrieval database itself; if those test graphs were also used in GFM pretraining, the reported CARD contribution is contaminated by test information and the central claim of training-free generalization weakens.

What would settle it

Retrain the GFM graph encoder on only the training splits (80% of each dataset), then rerun BrainAgent with CARD on the untouched test splits. If the accuracy/precision gains over direct prompting shrink or vanish, the case-retrieval improvement is an artifact of test-set leakage; if they hold, the claim survives.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • No LLM fine-tuning or task-specific classifier training is required: the gains come entirely from inference-time tool use, retrieval, and reflection.
  • Open-source and small backbones benefit nearly as much as closed-source frontier models, pointing toward local, privacy-preserving deployment.
  • The reflection stage corrects concrete numerical and evidential errors (e.g., a density miscalculation in the case study), so the final rationale is more trustworthy.
  • Precision improves while recall may drop, meaning the framework trades a degenerate all-positive policy for a more balanced decision boundary—a desirable property in screening.
  • The structured two-path retrieval and multi-level explanation format can be reused as a template for other graph-structured medical data.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same agentic pattern—topology-aware textualization, domain-knowledge retrieval, case retrieval, reflection—should transfer to other scientific graph-classification tasks (e.g., molecular or protein-function prediction) where a knowledge base and labeled case reports exist.
  • CARD's dual-modality similarity (graph embedding + textual report) is a reusable recipe; the fixed 0.5 weighting of the two modalities is arbitrary and dataset-specific tuning could improve retrieval.
  • Because pass@1 accuracy remains moderate, a natural extension consistent with the paper's own outlook is to add calibration or a small supervised head on top of the agent's output to convert its precision gains into higher single-answer accuracy.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes BrainAgent, an agentic LLM framework for binary brain-network (connectome) classification. It first converts each rs-fMRI graph into multi-level textual descriptions via graph-analysis tools, then runs a Think–Report–Action–Observation loop with two retrieval modules — HARK for neuroscience knowledge and CARD for similar cases — followed by a reflection stage that produces a structured prediction and rationale. Experiments on ABIDE, ADHD, HCP, and Rest-meta-MDD with several closed- and open-source LLMs claim consistent improvements over direct prompting, CoT, and Reflection baselines, plus ablations and interpretability analyses. The paper is detailed, with appendices covering dataset construction, prompts, retrieval details, and case studies.

Significance. If the empirical claims held, BrainAgent would be a useful contribution: a modular, inference-time framework that makes LLM-based connectome classification more accurate, less positively biased, and more interpretable. The paper ships substantial engineering detail — prompts, retrieval pipelines, a graph foundation model, case examples, and a code link — which is a strength. However, the central empirical claim is currently not established. The GFM used by CARD appears to have been pretrained on all 4,760 graphs, including the test splits, and the headline pass@3 metric is inflated by construction. These are load-bearing issues: the paper's own ablation attributes large gains to CARD, and the most striking numbers in Tables 2–3 are the @3 columns. The framework itself may be salvageable, but the reported experiments do not currently support the abstract's 'consistently improves' claim.

major comments (4)
  1. [Appendix G, Table 5; Section 3.1; Appendix H.1] The GFM graph encoder is pretrained on 4,760 graphs (Table 5), which equals the total number of samples in Table 1 (618+938+1,039+2,165). Section 3.1 states that 20% of each dataset is held out as the test set and the remaining samples are used for 'retrieval database construction and model-related preparation,' but no statement says GFM pretraining excluded test graphs. Appendix H.1 explicitly avoids the test set for report generation, showing leakage awareness, but no such guarantee is given for GFM. Since CARD uses GFM embeddings to retrieve similar cases, and the ablation in Figure 4 shows large drops when CARD is removed, the reported gains may be substantially contaminated by transductive leakage. The authors must either demonstrate that GFM was pretrained only on non-test graphs or retrain and report all results.
  2. [Section 3.3, Eq. (8)] The pass@k metric as defined is not an accuracy measure for binary classification. For k=3, any sample for which at least one of the three outputs matches the label counts as correct; a model that emits both labels across the three runs receives 100% accuracy by construction. The @3 columns in Tables 2–3 therefore conflate genuine improvement with the trivial effect of output diversity in binary tasks. The paper should report pass@1 as the primary metric, or use majority voting / agreement-based aggregation, and should not headline @3 gains.
  3. [Tables 2–3 and Section 3.4] No error bars, confidence intervals, or significance tests are reported for any of the accuracy/precision/recall numbers. LLM decoding is stochastic even at low temperature, and comparisons across methods are based on single point estimates. The claim of 'consistent improvement' needs repeated runs (at least for @1) with paired tests or variance reporting; otherwise the gains, some of which are modest, cannot be distinguished from noise.
  4. [Appendix H.2] The CARD top-k value is selected by testing k in {3,4,5,6} and reporting the best k=3. The text does not state whether this selection was made on a validation split or on the test set. If the test set was used for this hyperparameter choice, the reported numbers are optimistically biased. Please clarify the selection protocol and, if necessary, evaluate on an independent split.
minor comments (4)
  1. [Prompts in Appendix I] Several typos and formatting issues appear in the prompt templates: 'CALUCULATING', 'hallunation', 'precition', and inconsistent JSON quoting. These should be cleaned up, though they do not affect the technical content.
  2. [Tables 2–3] The @3 columns are labeled as Accuracy/Recall/Precision, but under the pass@k criterion a sample is correct if any output is correct. This should be clearly denoted as 'pass@3 accuracy' etc., and the aggregation rule for Recall/Precision should be stated explicitly in the table captions.
  3. [Section K.2] The 'Unsuccessful Records' section shows a prediction from 'Claude', but Claude is not listed among the evaluated backbones in Section 3.2 or Appendix D. Please clarify whether this is an additional model or a typo.
  4. [Section K.1, Finding 3] The discussion says LLMs yield little gain over SLMs, but the open-source models evaluated (Qwen3.5-35B, Qwen3.5-9B, Gemma4-26B) are not typically described as small language models. Clarify what is meant by SLM in this context.

Circularity Check

1 steps flagged

GFM pretraining appears to include the 20% test splits (4,760 = sum of all four datasets), so CARD's reported retrieval gains are partially a fitted-input effect rather than clean held-out prediction.

specific steps
  1. fitted input called prediction [Appendix G, Table 5; Table 1; §3.1 / D.1; Appendix H.1]
    "Number of graphs 4760 ... For each dataset, we randomly select 20% of the samples as the test set and use the remaining samples for retrieval database construction and model-related preparation. ... Since this process does not involve test dataset, there is no risk of data leakage."

    The four datasets contain 618 + 938 + 1,039 + 2,165 = 4,760 samples, exactly the number of graphs Table 5 says were used to pretrain GFM. If the 20% test splits had been excluded, the pretraining count would be 3,808. The paper only rules out test data for CARD report generation, never for GFM pretraining. Hence, for a test query G_i, the graph embedding h^G_i = Readout(GFM(G_i)) is computed by an encoder already pretrained on that exact graph. The CARD similarity α^G is therefore fitted to the test input by construction, and reporting BrainAgent's CARD-dependent accuracy gains as held-out prediction reduces to evaluating on the encoder's training data.

full rationale

The core agentic derivation is not definitionally circular: the understanding module uses deterministic graph statistics, HARK retrieves externally constructed neuroscience knowledge, and the LLM reasoning/reflection steps are not defined in terms of the test labels. No self-citation chain or ansatz is load-bearing. However, the empirical headline is compromised at the CARD/GFM step. The paper's own numbers show GFM pretraining used 4,760 graphs, which is the total of all four datasets including the 20% test splits, while no statement excludes test graphs from GFM; the only leakage-free statement concerns generated reports. Since the ablation attributes large drops to removing CARD, much of the reported improvement may be an artifact of a graph encoder fit on the test graphs themselves. This is a partial, localized circularity in the evaluation, not full equivalence, so I score it 6.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The framework relies on a long chain of domain and methodological assumptions. The most load-bearing are that text serialization plus hand-computed graph statistics preserve enough connectome information for an LLM to reason about it; that LLM-written case reports with ground-truth labels are valid evidence; and that retrieval-based label injection is an acceptable predictor. The pass@k evaluation convention is an ad hoc assumption that materially changes results.

free parameters (5)
  • beta (CARD fusion weight) = 0.5
    Hand-fixed in Eq. (4)/(15) to balance graph-level and text-level retrieval similarity; no sensitivity analysis.
  • CARD top-k = 3
    Appendix H.2 states the authors tested top-k in {3,4,5,6} and found 3 best; this is data-dependent selection without a described held-out validation.
  • HARK top-k = 5
    Set to 5 for fine-grained retrieval; fixed, not justified by analysis.
  • HARK IoU threshold = 0.5
    Coarse retrieval threshold in Appendix F.2; hand-set.
  • pass@k evaluation multiplicity k = 1 and 3; @3 emphasized
    Choice of reporting and emphasizing k=3 is consequential because with binary labels k>=2 allows credit for outputting both labels.
axioms (5)
  • domain assumption Binary thresholded Pearson-correlation connectomes under the AAL 90 parcellation adequately represent brain-network state for diagnosis.
    Appendix B constructs graphs via DPARSF, AAL time series, Pearson correlation, and thresholding; the whole evaluation depends on this representation.
  • domain assumption LLM-generated case reports conditioned on ground-truth labels are reliable analytical evidence.
    Appendix H.1 explicitly feeds labels into report generation, arguing labels make report generation easier; these reports are then used as retrieval evidence in CARD.
  • domain assumption PubMedBERT and GFM embeddings capture brain-network similarity relevant to diagnosis.
    CARD's dual-modality score assumes graph and text embedding similarities are predictive of label-matching cases; no independent validation is provided.
  • ad hoc to paper Pass@k with k>1 is a valid accuracy criterion for binary classification.
    Eq. (8) defines pass@k; using 'correct if any of k outputs is correct' for a two-class problem is non-standard and inflates scores.
  • standard math Standard graph-theoretic metrics (clustering, small-worldness, modularity, efficiency) can be computed by external functions and then interpreted by the LLM.
    The understanding stage relies on standard graph measurements; the LLM is instructed to use them rather than calculate them.

pith-pipeline@v1.3.0-alltime-deepseek · 41497 in / 11573 out tokens · 122885 ms · 2026-08-01T05:49:46.379748+00:00 · methodology

0 comments
read the original abstract

Brain network analysis is crucial for understanding cognition and neurological disorders, yet existing deep learning methods mainly treat connectome analysis as a graph-to-logit classification problem, offering limited explanatory reasoning. Large language models (LLMs) provide a promising interface for knowledge-intensive scientific analysis, but directly applying general-purpose LLMs to brain networks remains challenging due to the structure-language gap, limited neuroscience grounding, and overconfident positive predictions. In this paper, we propose \textbf{BrainAgent}, an agentic LLM framework for knowledge-enhanced brain network analysis. BrainAgent reformulates connectome classification as an iterative process of topology-aware understanding, external retrieval, reasoning, and reflection. Specifically, it first converts raw brain networks into compact multi-level structural descriptions through brain-specific analysis tools, then retrieves relevant neuroscience knowledge and task-specific cases to ground the reasoning process, and finally generates structured predictions with reflective verification. Experiments on four public rs-fMRI datasets show that BrainAgent consistently improves different closed-source and open-source LLM backbones over direct prompting and standard reasoning baselines. Further ablation and interpretability analyses demonstrate the effectiveness of each component and show that BrainAgent produces more comprehensive, multi-level, and verifiable explanations.These results indicate that agentic LLMs provide a practical route toward interpretable and knowledge-grounded brain network analysis.

Figures

Figures reproduced from arXiv: 2607.22082 by Jiaxing Li, Muyao Tang, Rui Dong, Youyong Kong.

Figure 1
Figure 1. Figure 1: Illustration of the motivation and overall framework of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overall framework of BrainAgent: an understanding module that converts raw brain [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Precision ranking of multiple recent LLMs on four public brain network analysis datasets. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Ablation study of BrainAgent on four public datasets. We report accuracy using DeepSeek [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Interpretability comparison between base LLMs and BrainAgent-enhanced LLMs across [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Preprocess of fMRI data and construction for FC dataset. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Process of HARK. HARK is a two-stage, hierarchical augmented retrieval for neural [PITH_FULL_IMAGE:figures/full_fig_p021_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: (Left) Structure of Graph Foundation Model. (Right) Training objective of GFM. [PITH_FULL_IMAGE:figures/full_fig_p022_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Visualization of loss curve on GFM. GFM 𝒢𝒊 Text Encoder similarity 𝜶𝒊,𝒋 𝓖 𝜶𝒊,𝒋 𝓣 𝜶#𝒊,𝒋 𝓓𝟏 𝓓𝟐 𝓓𝟑 …… Brain Graph Dataset ×𝜷 ×(𝟏 − 𝜷) similarity Ranking input graph return 𝒕𝒐𝒑𝒌 Graph Textual cases && reports LLM Analysis 𝒉𝒊 𝓖 𝒉𝒋 𝓖 𝒉𝒋 𝓣 𝒉𝒊 𝓣 [PITH_FULL_IMAGE:figures/full_fig_p024_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Process of CARD. CARD is a case-level augmented retrieval framwork via dual-modality [PITH_FULL_IMAGE:figures/full_fig_p024_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

66 extracted references · 8 linked inside Pith

  1. [1]

    Small language models are the future of agentic ai

    Peter Belcak, Greg Heinrich, Shizhe Diao, Yonggan Fu, Xin Dong, Saurav Muralidharan, Yingyan Celine Lin, and Pavlo Molchanov. Small language models are the future of agentic ai. arXiv preprint arXiv:2506.02153, 2025

    Pith/arXiv arXiv 2025

  2. [2]

    Complex brain networks: graph theoretical analysis of structural and functional systems.Nature Reviews Neuroscience, 10(3):186–198, 2009

    Ed Bullmore and Olaf Sporns. Complex brain networks: graph theoretical analysis of structural and functional systems.Nature Reviews Neuroscience, 10(3):186–198, 2009

    2009

  3. [3]

    The direct consortium and the rest-meta- mdd project: towards neuroimaging biomarkers of major depressive disorder.Psychoradiology, 2(1):32–42, 03 2022

    Xiao Chen, Bin Lu, Hui-Xian Li, Xue-Ying Li, et al. The direct consortium and the rest-meta- mdd project: towards neuroimaging biomarkers of major depressive disorder.Psychoradiology, 2(1):32–42, 03 2022

    2022

  4. [4]

    The adhd-200 consortium: a model to advance the translational potential of neuroimaging in clinical neuroscience.Frontiers in systems neuroscience, 6:62, 2012

    ADHD-200 consortium. The adhd-200 consortium: a model to advance the translational potential of neuroimaging in clinical neuroscience.Frontiers in systems neuroscience, 6:62, 2012

    2012

  5. [5]

    Braingb: A benchmark for brain network analysis with graph neural networks.IEEE Transactions on Medical Imaging, 42(2):493–506, 2023

    Hejie Cui, Wei Dai, Yanqiao Zhu, Xuan Kan, Antonio Aodong Chen Gu, Joshua Lukemire, Liang Zhan, Lifang He, Ying Guo, and Carl Yang. Braingb: A benchmark for brain network analysis with graph neural networks.IEEE Transactions on Medical Imaging, 42(2):493–506, 2023

    2023

  6. [6]

    Interpretable graph neural networks for connectome-based brain disorder analysis

    Hejie Cui, Wei Dai, Yanqiao Zhu, Xiaoxiao Li, Lifang He, and Carl Yang. Interpretable graph neural networks for connectome-based brain disorder analysis. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, pages 375–385. Springer, 2022

    2022

  7. [7]

    Deepseek-v3.2: Pushing the frontier of open large language models, 2025

    DeepSeek-AI, Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, et al. Deepseek-v3.2: Pushing the frontier of open large language models, 2025

    2025

  8. [8]

    The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism.Molecular psychiatry, 19(6):659–667, 2014

    Adriana Di Martino, Chao-Gan Yan, Qingyang Li, Erin Denio, Francisco X Castellanos, Kaat Alaerts, Jeffrey S Anderson, Michal Assaf, Susan Y Bookheimer, Mirella Dapretto, et al. The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism.Molecular psychiatry, 19(6):659–667, 2014

    2014

  9. [9]

    Spreadsheetllm: Encoding spreadsheets for large language models.arXiv preprint arXiv:2407.09025, 2024

    Haoyu Dong, Jianbo Zhao, Yuzhang Tian, Junyu Xiong, Shiyu Xia, Mengyu Zhou, Yun Lin, José Cambronero, Yeye He, Shi Han, et al. Spreadsheetllm: Encoding spreadsheets for large language models.arXiv preprint arXiv:2407.09025, 2024

    Pith/arXiv arXiv 2024

  10. [10]

    Bleg: Llm functions as powerful fmri graph-enhancer for brain network analysis, 2026

    Rui Dong, Zitong Wang, Jiaxing Li, Weihuang Zheng, and Youyong Kong. Bleg: Llm functions as powerful fmri graph-enhancer for brain network analysis, 2026

    2026

  11. [11]

    M3d-bfs: a multi-stage dynamic fusion strategy for sample-adaptive multi-modal brain network analysis, 2026

    Rui Dong, Xiaotong Zhang, Jiaxing Li, Yueying Li, Jiayin Wei, and Youyong Kong. M3d-bfs: a multi-stage dynamic fusion strategy for sample-adaptive multi-modal brain network analysis, 2026

    2026

  12. [12]

    When attention sink emerges in language models: An empirical view.arXiv preprint arXiv:2410.10781, 2024

    Xiangming Gu, Tianyu Pang, Chao Du, Qian Liu, Fengzhuo Zhang, Cunxiao Du, Ye Wang, and Min Lin. When attention sink emerges in language models: An empirical view.arXiv preprint arXiv:2410.10781, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Domain-specific language model pretraining for biomedical natural language processing.ACM Transactions on Computing for Healthcare, 3(1):1–23, 2021

    Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. Domain-specific language model pretraining for biomedical natural language processing.ACM Transactions on Computing for Healthcare, 3(1):1–23, 2021

    2021

  14. [14]

    Inductive representation learning on large graphs

    Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. InAdvances in Neural Information Processing Systems (NeurIPS), pages 1024–1034, 2017. 10

    2017

  15. [15]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000–16009, 2022

    2022

  16. [16]

    G-retriever: Retrieval-augmented generation for textual graph understanding and question answering.Advances in Neural Information Processing Systems, 37:132876–132907, 2024

    Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh V Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson, and Bryan Hooi. G-retriever: Retrieval-augmented generation for textual graph understanding and question answering.Advances in Neural Information Processing Systems, 37:132876–132907, 2024

    2024

  17. [17]

    Graphmae: Self-supervised masked graph autoencoders

    Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang. Graphmae: Self-supervised masked graph autoencoders. InProceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 594–604, 2022

    2022

  18. [18]

    Brainnpt: Pre-training trans- former networks for brain network classification.IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2024

    Jinlong Hu, Yangmin Huang, Nan Wang, and Shoubin Dong. Brainnpt: Pre-training trans- former networks for brain network classification.IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2024

    2024

  19. [19]

    Med-moe: Mixture of domain-specific experts for lightweight medical vision-language models.arXiv preprint arXiv:2404.10237, 2024

    Songtao Jiang, Tuo Zheng, Yan Zhang, Yeying Jin, Li Yuan, and Zuozhu Liu. Med-moe: Mixture of domain-specific experts for lightweight medical vision-language models.arXiv preprint arXiv:2404.10237, 2024

    Pith/arXiv arXiv 2024

  20. [20]

    Ragraph: A general retrieval-augmented graph learning framework

    Xinke Jiang, Rihong Qiu, Yongxin Xu, Wentao Zhang, Yichen Zhu, Ruizhe Zhang, Yuchen Fang, Xu Chu, Junfeng Zhao, and Yasha Wang. Ragraph: A general retrieval-augmented graph learning framework. InAdvances in Neural Information Processing Systems, 2024

    2024

  21. [21]

    Kipf and Max Welling

    Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017

    2017

  22. [22]

    Multi-connectivity representation learning network for major depressive disorder diagnosis.IEEE Transactions on Medical Imaging, 42(10):3012–3024, 2023

    Youyong Kong, Wenhan Wang, Xiaoyun Liu, Shuwen Gao, Zhenghua Hou, Chunming Xie, Zhijun Zhang, and Yonggui Yuan. Multi-connectivity representation learning network for major depressive disorder diagnosis.IEEE Transactions on Medical Imaging, 42(10):3012–3024, 2023

    2023

  23. [23]

    Biobert: a pre-trained biomedical language representation model for biomedical text mining.Bioinformatics, 36(4):1234–1240, 2020

    Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. Biobert: a pre-trained biomedical language representation model for biomedical text mining.Bioinformatics, 36(4):1234–1240, 2020

    2020

  24. [24]

    Retrieval-augmented generation for knowledge-intensive nlp tasks

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks. InAdvances in Neural Information Processing Systems, volume 33, pages 9459–9474, 2020

    2020

  25. [25]

    Llava-med: Training a large language-and-vision assistant for biomedicine in one day.Advances in Neural Information Processing Systems, 36:28541–28564, 2023

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language-and-vision assistant for biomedicine in one day.Advances in Neural Information Processing Systems, 36:28541–28564, 2023

    2023

  26. [26]

    Dvornek, Muhan Zhang, Siyuan Gao, Juntang Zhuang, Dustin Scheinost, Lawrence H

    Xiaoxiao Li, Yuan Zhou, Nicha C. Dvornek, Muhan Zhang, Siyuan Gao, Juntang Zhuang, Dustin Scheinost, Lawrence H. Staib, Pamela Ventola, and James S. Duncan. Braingnn: Interpretable brain graph neural network for fmri analysis.Medical Image Analysis, 74:102233, 2021

    2021

  27. [27]

    Neurobridge: Bridging functional and structural brain networks via neural coupling and consistency-guided dynamic graph learning.Medical Image Analysis, 110:103993, 2026

    Yueying Li, Rui Dong, Xiaoyun Liu, Yonggui Yuan, and Youyong Kong. Neurobridge: Bridging functional and structural brain networks via neural coupling and consistency-guided dynamic graph learning.Medical Image Analysis, 110:103993, 2026

    2026

  28. [28]

    Neurofield-agl: Neurofield-attentive graph learning on functional connectivity for mental disorder diagnosis

    Yueying Li, Jiaxing Li, Yue Zhou, Youyong Kong, and Yonggui Yuan. Neurofield-agl: Neurofield-attentive graph learning on functional connectivity for mental disorder diagnosis. IEEE Sensors Journal, 26(5):7730–7742, 2026

    2026

  29. [29]

    Medchain: Bridging the gap between LLM agents and clinical practice with interactive sequence

    Jie Liu, Wenxuan Wang, Zizhan Ma, Guolin Huang, SU Yihang, Kao-Jung Chang, Haoliang Li, Linlin Shen, Michael Lyu, and Wenting Chen. Medchain: Bridging the gap between LLM agents and clinical practice with interactive sequence. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2025. 11

    2025

  30. [30]

    Sherlock: Towards dynamic knowledge adaptation in llm-enhanced e-commerce risk management, 2025

    Nan Lu, Yurong Hu, Jiaquan Fang, Yan Liu, Rui Dong, Yiming Wang, Rui Lin, and Shaoyi Xu. Sherlock: Towards dynamic knowledge adaptation in llm-enhanced e-commerce risk management, 2025

    2025

  31. [31]

    Biogpt: generative pre-trained transformer for biomedical text generation and mining.Briefings in bioinformatics, 23(6):bbac409, 2022

    Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. Biogpt: generative pre-trained transformer for biomedical text generation and mining.Briefings in bioinformatics, 23(6):bbac409, 2022

    2022

  32. [32]

    Inject- ing new knowledge into large language models via supervised fine-tuning.arXiv preprint arXiv:2404.00213, 2024

    Nick Mecklenburg, Yiyou Lin, Xiaoxiao Li, Daniel Holstein, Leonardo Nunes, Sara Mal- var, Bruno Silva, Ranveer Chandra, Vijay Aski, Pavan Kumar Reddy Yannam, et al. Inject- ing new knowledge into large language models via supervised fine-tuning.arXiv preprint arXiv:2404.00213, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    Mmgpl: Multimodal medical data analysis with graph prompt learning.Medical Image Analysis, 97:103225, 2024

    Liang Peng, Songyue Cai, Zongqian Wu, Huifang Shang, Xiaofeng Zhu, and Xiaoxiao Li. Mmgpl: Multimodal medical data analysis with graph prompt learning.Medical Image Analysis, 97:103225, 2024

    2024

  34. [34]

    Qwen3.5: Towards native multimodal agents, February 2026

    Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026

    2026

  35. [35]

    Capabilities of gemini models in medicine.arXiv preprint arXiv:2404.18416, 2024

    Khaled Saab, Tao Tu, Wei-Hung Weng, Ryutaro Tanno, David Stutz, Ellery Wulczyn, Fan Zhang, Tim Strother, Chunjong Park, Elahe Vedadi, et al. Capabilities of gemini models in medicine.arXiv preprint arXiv:2404.18416, 2024

    Pith/arXiv arXiv 2024

  36. [36]

    Toolformer: Language models can teach themselves to use tools

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettle- moyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. InAdvances in Neural Information Processing Systems, volume 36, 2023

    2023

  37. [37]

    Reflexion: language agents with verbal reinforcement learning

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. InThirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  38. [38]

    Rest: a toolkit for resting-state functional magnetic resonance imaging data processing.PloS one, 6(9):e25031, 2011

    Xiao-Wei Song, Zhang-Ye Dong, Xiang-Yu Long, Su-Fang Li, Xi-Nian Zuo, Chao-Zhe Zhu, Yong He, Chao-Gan Yan, and Yu-Feng Zang. Rest: a toolkit for resting-state functional magnetic resonance imaging data processing.PloS one, 6(9):e25031, 2011

    2011

  39. [39]

    Graphgpt: Graph instruction tuning for large language models

    Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. Graphgpt: Graph instruction tuning for large language models. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 491–500, 2024

    2024

  40. [40]

    Constructing high-order functional connectivity networks with temporal information from fmri data.IEEE Transactions on Medical Imaging, 2024

    Yingzhi Teng, Kai Wu, Jing Liu, Yifan Li, and Xiangyi Teng. Constructing high-order functional connectivity networks with temporal information from fmri data.IEEE Transactions on Medical Imaging, 2024

    2024

  41. [41]

    The wu-minn human connectome project: an overview.Neuroimage, 80:62–79, 2013

    David C Van Essen, Stephen M Smith, Deanna M Barch, Timothy EJ Behrens, Essa Yacoub, Kamil Ugurbil, Wu-Minn HCP Consortium, et al. The wu-minn human connectome project: an overview.Neuroimage, 80:62–79, 2013

    2013

  42. [42]

    Graph attention networks

    Petar Veliˇckovi´c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. InInternational Conference on Learning Representations, 2018

    2018

  43. [43]

    Rag4gfm: Bridging knowledge gaps in graph foundation models through graph retrieval augmented generation

    Xingliang Wang, Zemin Liu, Junxiao Han, and Shuiguang Deng. Rag4gfm: Bridging knowledge gaps in graph foundation models through graph retrieval augmented generation. InThe Thirty- ninth Annual Conference on Neural Information Processing Systems, 2025

    2025

  44. [44]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V . Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. InProceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY , USA, 2022. Curran Associates Inc. 12

    2022

  45. [45]

    A brain graph foundation model: Pre-training and prompt-tuning across broad atlases and disorders

    Xinxu Wei, kanhao zhao, Yong Jiao, Lifang He, and Yu Zhang. A brain graph foundation model: Pre-training and prompt-tuning across broad atlases and disorders. InThe Fourteenth International Conference on Learning Representations, 2026

    2026

  46. [46]

    Bloomberggpt: A large language model for finance, 2023

    Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. Bloomberggpt: A large language model for finance, 2023

    2023

  47. [47]

    Contrastive graph pooling for explainable classification of brain networks.IEEE Transactions on Medical Imaging, 2024

    Jiaxing Xu, Qingtian Bian, Xinhang Li, Aihu Zhang, Yiping Ke, Miao Qiao, Wei Zhang, Wei Khang Jeremy Sim, and Balázs Gulyás. Contrastive graph pooling for explainable classification of brain networks.IEEE Transactions on Medical Imaging, 2024

    2024

  48. [48]

    BrainPrompt: Multi-Level Brain Prompt Enhancement for Neurological Condition Identification

    Jiaxing Xu, Kai He, Yue Tang, Wei Li, Mengcheng Lan, Xia Dong, Yiping Ke, and Mengling Feng. BrainPrompt: Multi-Level Brain Prompt Enhancement for Neurological Condition Identification . Inproceedings of Medical Image Computing and Computer Assisted Intervention – MICCAI 2025, volume LNCS 15971, pages 172 – 182. Springer Nature Switzerland, September 2025

    2025

  49. [49]

    Qwen3 technical report, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, et al. Qwen3 technical report, 2025

    2025

  50. [50]

    Griffiths, Yuan Cao, and Karthik Narasimhan

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems, volume 36, 2023

    2023

  51. [51]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. InThe eleventh international conference on learning representations, 2023

    2023

  52. [52]

    RH- brainFS: Regional heterogeneous multimodal brain networks fusion strategy

    Hongting Ye, Yalu Zheng, Yueying Li, Ke Zhang, Youyong Kong, and Yonggui Yuan. RH- brainFS: Regional heterogeneous multimodal brain networks fusion strategy. InThirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  53. [53]

    Recgpt technical report.arXiv preprint arXiv:2507.22879, 2025

    Chao Yi, Dian Chen, Gaoyang Guo, Jiakai Tang, Jian Wu, Jing Yu, Mao Zhang, Sunhao Dai, Wen Chen, Wenjun Yang, et al. Recgpt technical report.arXiv preprint arXiv:2507.22879, 2025

    Pith/arXiv arXiv 2025

  54. [54]

    Understanding

    Wenhao Zheng, Liaoyaqi Wang, Dongshen Peng, Hongxia Xu, Yun Li, Hongtu Zhu, Tianfan Fu, and Huaxiu Yao. Multimodal clinical trial outcome prediction with large language models. arXiv preprint arXiv:2402.06512, 2024. 13 A Related Works A.1 GNN-based Brain network analysis. Graph neural networks (GNNs) have become a widely used backbone for brain network an...

    Pith/arXiv arXiv 2024

  55. [55]

    The prompt consists of basic data information and statistical results from defined analysis function calls

    Understanding: Prompt template for Understanding part is shown in Prompt I. The prompt consists of basic data information and statistical results from defined analysis function calls. 25 Output is required as strict json format with region-level, subgraph-level and graph-level analysis

  56. [56]

    BrainAgent is required to perform reasoning before action

    Think and Report: Prompt template for Think part is shown in Prompt I. BrainAgent is required to perform reasoning before action. Meanwhile, it should also maintain and update a fixed-size report which stores useful history information

  57. [57]

    LLM is prompted to select corresponding function call from available function tools

    Action: Prompt template for Action part is shown in Prompt I. LLM is prompted to select corresponding function call from available function tools. Meanwhile, concrete queries are required as parameters for different function call. retrieve_knowledge stands for HARK module andretrieve_casestands for CARD module

  58. [58]

    We prompt LLM to make prediction and give corresponding reasons

    Analysis: Prompt template for Analysis part is shown in Prompt I. We prompt LLM to make prediction and give corresponding reasons

  59. [59]

    Analysis

    reflection: Prompt template for Reflection part is shown in Prompt I. Reflection is made after final analysis for correctness and refinement. LLM is asked to reflect on different generation contents, including LLM analysis, prediction and reasons in “Analysis” part. 26 Prompt A: Prompt for Understanding # Role Definition You are an expert in neuroscience ...

  60. [60]

    Analyze: Your analysis

  61. [61]

    ‘json {{

    Action 1: Summarization of action 1 and observation. ... ”’ - action: Your available action must be within these tools below: {tools} # Output Requirements Your output must strictly follow json format below.DO NOT MODIFY ITS STRUCTURE: “‘json {{ "think": "your thinking process, within 100 words", "report": "your report, within 300 words", }} “‘ Now your s...

  62. [62]

    If the analysis from graph data is correct

  63. [63]

    If the reason is valid based on the data and corresponding analysis

  64. [64]

    ‘json {{

    If the final prediction is correct. - Confidence from LLM generation is also a factor indicating llm’s own certainty for its prediction. - To avoid hallucinations, you need to combine your judgement with official neuro-knowledge. You can also refer to external knowledge, tools and resources to improve your reflection. - After reflection, you should also f...

    arXiv

  65. [65]

    Attention Sink

    LLM fails to invoke different tools across multiple rounds and only calls those placed at the forefront (in our case, LLM only calls retrieve_knowledge function). This can be regarded as "Attention Sink" phenomenon [12], where LLM tends to focus more on tokens at the forefront. In BrainAgent, we solve this issue by popping certain function from the functi...

  66. [66]

    action":

    Parameters of the function calls returned by LLM are inconsistent with the requirements specified in the prompt. For example, for function retrieve_knowledge, we define its parameter query: str and regions_q: List[int] indicating query content and brain regions respectively. However, for LLM generated json which is shown in K.2, its arguments becomeregion...

    2000