Pith. sign in

REVIEW 3 major objections 5 minor 49 references

SeekBrain claims that anchoring an LLM-driven multi-agent system in a repertoire of analysis recipes distilled from published code-paper pairs lets it outperform general-purpose agents on end-to-end neuroscience data analysis.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 08:48 UTC pith:7NE7X3Q2

load-bearing objection SeekBrain is a useful engineering contribution with a genuinely clever recipe-crystallization idea, but its headline benchmark is inflated by overlap between the recipe library and the BrainArena tasks; the case studies are promising but need code and data release. the 3 major comments →

arxiv 2607.29347 v1 pith:7NE7X3Q2 submitted 2026-07-31 cs.MA cs.AI

SeekBrain: An Autonomous Multi-Agent System for Accelerating Neuroscience Discovery

Jiamin Wu , Peishan Xiang , Jingyang Chen , Yuqing Zhu , Yuxi Li , Ling Luo , Qihao Zheng , Jialiang Zu
show 21 more authors
Yongchao Wu Mindong Liu Haitao Wu Chaofan Hu Yijie Sun Yuqi Hang Yu Zhu Shuo Li Yue Fan Shiyang Feng Wanghan Xu Tianlei Zhang Jie Zhang Wenlong Zhang Bo Zhang Kai Wang Lei Bai Mianxin Liu Wanli Ouyang Jiulin Du Chunfeng Song
This is my paper
classification cs.MA cs.AI
keywords multi-agent systemneuroscience data analysisanalysis recipe repertoireLLM agentBrainArena benchmarkzebrafish neural representationsmouse decision-makingprocedural memory
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper is trying to establish that the bottleneck in automating neuroscience is not reasoning but knowledge representation: analytical expertise is scattered across publications and codebases, so generic agents hallucinate methods. SeekBrain answers by auto-crystallizing a library of analysis recipes from code-paper pairs and using that repertoire as procedural memory to anchor hierarchical planning and execution. On a new expert-annotated benchmark, BrainArena, SeekBrain scores 75.8 versus 64.1 and 58.1 for two leading general-purpose coding agents, and removing the repertoire drops the score to 63.2. The paper also reports two real-world successes: structured, distributed brain-wide neural representations of larval zebrafish behavior, and a shared axis of regional decoding strength in a mouse decision-making dataset. A sympathetic reader would care because the recipe mechanism offers a concrete path for turning fragmented methodological knowledge into reproducible, increasingly autonomous discovery.

Core claim

The central claim is that the bottleneck is a knowledge-representation problem, not a reasoning problem: analytical expertise is scattered across publications and codebases, and general-purpose agents are prone to methodological hallucinations without it. SeekBrain's answer is the Neuroscience Analysis Repertoire: a set of analysis recipes, each linking a scientific aim to required data, processing and modeling steps, visualization, and code, auto-crystallized from code-paper pairs. Recipes are retrieved and adapted by a Formulation Agent, checked by a Validation Agent, and improved by distilling validated researcher traces back into the library. The supporting evidence is the BrainArena ben

What carries the argument

The load-bearing object is the Neuroscience Analysis Repertoire, a library of 68 analysis recipes in which each recipe crystallizes a published study's analysis (scientific rationale, required inputs, processing and modeling steps, visualization, and code) into a reusable procedure. A Crystallization Agent builds recipes from code-paper pairs; a Formulation Agent retrieves and adapts them into a methodological schema; a Validation Agent rejects outputs that deviate from the recipe's expectations; and a Distillation Agent folds validated researcher traces back into the library. The repertoire is what differentiates domain-grounded execution from generic coding, quantified by the ablation that

Load-bearing premise

The head-to-head benchmark result assumes that the BrainArena tasks are not recycling the same published analyses that the repertoire was crystallized from; if the recipe library overlaps heavily with the task ground truth, the reported advantage reflects recall of near-answer procedures rather than autonomous scientific reasoning.

What would settle it

Build a set of neuroscience analysis tasks from studies published after the repertoire was fixed (or explicitly withheld from recipe construction) and run SeekBrain on them without allowing new recipe crystallization. If its advantage over general-purpose coding agents disappears or shrinks sharply, the benchmark result is inflated by task-recipe overlap; if the advantage persists, the repertoire is genuinely enabling transferable analysis skill.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the repertoire is the source of reliability, then general-purpose coding agents will continue to trail on fragmented scientific data until they embed comparable procedural knowledge.
  • The largest reported gains are on hard tasks and on interpretive validity, implying that procedural domain memory improves biologically grounded interpretation, not just figure reproduction.
  • Because validated user traces are distilled back into the repertoire, the system's capability is designed to accumulate with use, making it a growing communal asset rather than a frozen tool.
  • The two case studies indicate the same engine can operate in researcher-led mode (multi-stage zebrafish analysis) and agent-led mode (proposing and testing a mouse-decoding hypothesis).
  • BrainArena itself offers a reusable expert-rubric benchmark with 32 tasks and over 700 deduction subitems for measuring end-to-end neuroscience analysis agents.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the repertoire is crystallized from published studies and the BrainArena tasks are figure-reproduction tasks from published studies, part of the 75.8-versus-63.2 recipe advantage may reflect retrieval of near-ground-truth procedures rather than autonomous scientific reasoning; a holdout benchmark built from studies excluded from recipe construction would separate the two.
  • The recipe-crystallization mechanism looks transferable to other disciplines whose methodological knowledge is scattered across papers and codebases (for example ecology, genomics, or clinical data), provided the same leakage risk is controlled.
  • The framework predicts that agent performance should scale with repertoire coverage and diversity; with only 68 recipes and 32 tasks, this scaling curve can be measured directly by adding recipes and watching benchmark scores.
  • We read the mouse shared-axis result primarily as a demonstration that the agent can propose and test a hypothesis end-to-end; the paper does not rule out alternative global confounds, so it is best treated as a workflow demonstration rather than an established neural principle.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. SeekBrain is an LLM-based multi-agent system for neuroscience data analysis. It builds a 'Neuroscience Analysis Repertoire' by crystallizing recipes from paper-code pairs, then uses hierarchical planning and an execution engine (formulation, coding, validation, interpretation) to perform end-to-end analyses. The authors introduce BrainArena, a 32-task benchmark derived from 10 published studies, and report that SeekBrain achieves a mean score of 75.8 versus 64.1 for Claude Code and 58.1 for Codex, with a repertoire ablation dropping SeekBrain to 63.2. Two case studies demonstrate researcher-guided analyses of larval zebrafish data and agent-led re-analysis of the IBL mouse decision-making dataset.

Significance. If the benchmark comparison were clean, this paper would be a useful practical contribution: it provides a concrete architecture for encoding dispersed methodological knowledge into reusable recipes and applies it to diverse multimodal neuroscience tasks. The manuscript includes open-source code, a project page, detailed recipe tables, and a fully specified benchmark, which are real reproducibility strengths. However, the central head-to-head claim is currently compromised by overlap between the recipe repertoire and the benchmark source studies, and the LLM judge belongs to the same model family as the evaluated systems. The case studies demonstrate that the system can execute complex analyses, but they do not by themselves establish autonomous scientific discovery. The contribution is therefore promising but not yet convincingly supported by the reported quantitative evidence.

major comments (3)
  1. [§4.4, §4.5, Supp. Tables 1–2] The BrainArena head-to-head claim is load-bearing and is weakened by task/recipe overlap. Supp. Table 2 shows tasks are drawn from 10 source studies, including Findling et al. [29], Marquez-Legorreta [46], Atanas et al. [39], and Légaré et al. [45]. Supp. Table 1 lists recipes such as `multi-mouse-spike_behavior-glm-decoding_metrics`, which decodes prior probability from ORBvl inter-trial activity — the exact Findling analysis that underlies the 'Neural prior decoding' task. Similar overlaps exist for C. elegans recipes, zebrafish gene-expression recipes, and structural/functional connectivity recipes. Because the Formulation Agent retrieves and adapts recipes (§4.2), the agent receives reference methodology for a large fraction of benchmark tasks. The ablation (75.8→63.2, Fig. 2i) confirms that the repertoire drives much of the gain. The authors must quantify task-level overlap, re-eval
  2. [§4.5, Evaluation] All benchmark scores are assigned by a Claude Opus 4.6 LLM judge, while SeekBrain and Claude Code use Claude Opus 4.7 as their core reasoning model. The judge and evaluated systems share large portions of pre-training and instruction-tuning data, creating a risk of systematic preference for Claude-style outputs. No human-expert agreement analysis is reported. Please provide human re-scoring on a random task subset or use a judge from a different model family as a robustness check; without this, the reported statistical differences (Friedman/Dunn, Fig. 2e) are not yet established as scientifically meaningful.
  3. [§2.2, Fig. 2i] The recipe ablation is informative but does not control for the recipe's contribution relative to baselines. Removing the repertoire from SeekBrain and comparing to Claude Code/Codex conflates two effects: the value of the recipe content and the value of SeekBrain's orchestration around it. An additional condition in which Claude Code and Codex receive the same retrieved recipe as a context document would identify whether the margin comes from the recipe itself or from the agent's planning/execution. Without this condition, the claim that SeekBrain performs 'domain-grounded planning and execution' better than baselines is not fully supported.
minor comments (5)
  1. [§2.3] Typo: 'representation representation' appears in the paragraph describing the CEBRA latent manifold.
  2. [Supp. Table 4 vs Supp. Table 5] The 'Cross-modal decay correlation' row reports 59 (with repertoire) and 55 (without) in Supp. Table 4, but Supp. Table 5 states the ablated score increased from 49 to 60. Please unify these numbers.
  3. [Figure 2] The caption lists panels e–i multiple times and does not clearly map each panel letter to the described content. Please revise the caption to match the panel layout.
  4. [§2.2, Benchmark results] The text says the Research Planning Engine is ablated in this experiment, but this is stated only later in Methods (§4.5). State this in the main Results paragraph so readers understand the benchmark evaluates the execution engine plus repertoire, not the full hierarchical planning system.
  5. [Author affiliations] The corresponding author email in the header contains garbled characters ('envel⌢opesongchunfeng@pjlab.org.cn'). Please correct.

Circularity Check

2 steps flagged

BrainArena head-to-head advantage is partly circular: the recipe repertoire is crystallized from the same 10 source studies that define BrainArena's rubrics, so high scores partly reflect retrieving the reference method; the two case studies are not circular.

specific steps
  1. fitted input called prediction [§4.4 recipe crystallization; §4.5 BrainArena construction; Supplementary Tables 1–3]
    "To construct the repertoire autonomously, a Crystallization Agent extracts and synthesizes methodological knowledge from paired inputs comprising open-access publications and their corresponding source-code repositories. ... BrainArena built on source studies with publicly available neuroscience data and reproducible figure-level analyses. The benchmark contained 32 tasks from 10 source studies ..., with each task corresponding to a key figure in the original publication."

    The repertoire's source papers are the same studies that define BrainArena's ground-truth rubrics. Concretely, the recipe `multi-mouse-spike_behavior-glm-decoding_metrics` (Supp. Table 1) 'decodes trial-by-trial prior probability from binned ORBvl inter-trial population activity using cross-validated linear models benchmarked against a Bayesian observer' — the exact analysis scored in the 'Neural prior decoding' task drawn from Findling et al. [29] (Supp. Tables 2–3). When the Formulation Agent retrieves and adapts that recipe, it is handed the reference method behind the task's rubric. The stated leakage mitigation (one bioRxiv task released after the model cutoff) does not separate the repertoire from the benchmark's source material. The high BrainArena score therefore partly measures re

  2. other [§2.2 Benchmark results; Fig. 2i; Supplementary Table 4]
    "To further validate the role of the analysis recipes, we removed them from the workflow in an ablation study. This ablation led to issues in key analysis steps (Supplementary Table 4) and reduced the mean score from 75.8 to 63.2 (Wilcoxon signed-rank test, P <0.0001; Fig. 2i)."

    This ablation is used to attribute SeekBrain's advantage to domain-grounded recipes, but the same numbers show that the repertoire — built from the same 10 papers that define the BrainArena tasks — supplies most of the head-to-head margin. With recipes SeekBrain scores 75.8 vs. 64.1 (Claude Code) and 58.1 (Codex); without recipes it drops to 63.2, below Claude Code. Thus the claimed superiority on the benchmark is substantially a retrieval advantage over the benchmark's own source procedures, not evidence of autonomous scientific reasoning. The advantage over Codex persists, which is why the circularity is partial rather than total.

full rationale

The two real-world case studies are not circular: they apply established public tools (Mearns-style bout clustering, CEBRA, Procrustes alignment, PCA/permutation on published decoder maps) to new questions and report statistics whose validity depends on the data and stated assumptions, not on the benchmark. The circularity is confined to the BrainArena evaluation claim. By construction, the Neuroscience Analysis Repertoire is extracted from paper-code pairs (Supp. Table 1) and BrainArena tasks are figure-reproduction rubrics from the same 10 source studies (Supp. Table 2); the Findling prior-decoding recipe is a one-to-one match to a benchmark task, and similar overlaps exist for C. elegans, zebrafish FC/structure, and other tasks. Since the Formulation Agent's stated workflow is to 'retriev(e) a task-relevant analysis recipe' as a methodological reference, the agent receives the reference method for overlapping tasks. The ablation gap (75.8→63.2) confirms that the benchmark gain is largely carried by the repertoire. The paper's only stated leakage mitigation is one bioRxiv task published after the Claude model's cutoff, which does not separate the repertoire from the benchmark's source material. I therefore assign a partial-circularity score of 6: the central benchmark claim is partly self-referential at the evaluation level, while the scientific case studies retain independent content.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

Central claims rest primarily on benchmark design assumptions and data-analysis choices rather than new physical axioms. The only invented artifact is the software repertoire, which is an engineering component with no independent falsifiable handle. No new physical entities are postulated.

free parameters (4)
  • Bout cluster count K = 3
    Selected by silhouette score across tested K values; defines the three bout types used to supervise CEBRA and compute per-cluster neural contrast maps in Section 2.3.
  • CEBRA latent dimension = 8
    Hand-fixed 8D embedding; affects 3NN decoding accuracy (0.949 +/- 0.034) and the Procrustes centroid alignment in Section 2.3.
  • Supervoxel budget = 3000 per larva
    Hand-fixed number of region-aware k-means supervoxels in Section 2.3; determines the spatial resolution of all brain-wide significant-SV maps.
  • Inclusion threshold for Procrustes analysis = clusters with at least 3 bouts; 13 larvae
    Sample-inclusion rule in Section 2.3 that determines the cross-fish centroid-alignment sample; could bias the shared-structure claim.
axioms (5)
  • domain assumption Recipes crystallized from published code-paper pairs provide valid, generalizable guidance for unseen neuroscience analyses.
    The entire repertoire design (Section 4.4) and recipe-ablation results rest on this; if recipes merely memorize benchmark source studies, the BrainArena gains are inflated.
  • domain assumption Claude Opus 4.6 rubric-based judging is a valid proxy for expert human scoring of figures and conclusions.
    Evaluation section (4.5) uses only an LLM judge; no human-expert calibration or inter-judge agreement is reported.
  • domain assumption Region-aware k-means supervoxels (3000 per larva) preserve behavior-relevant neural geometry in larval zebrafish data.
    All zebrafish spatial statistics in Section 2.3 are computed on supervoxel activity traces; a poor grouping would change which regions appear significant.
  • standard math PCA, column-shuffle permutation, and OLS residualization are appropriate statistical models for the five published decoder maps.
    Used in Section 2.4; the methods are standard, but the shared-axis claim assumes cross-region comparability of the decoder maps.
  • domain assumption The 201-region complete-case subset is representative of the 279 Beryl-labeled regions in the Findling et al. table.
    Section 2.4 retains only regions with finite values for all five decoders; no missing-data sensitivity analysis is provided.

pith-pipeline@v1.3.0-daily-deepseek · 27747 in / 15633 out tokens · 154536 ms · 2026-08-03T08:48:58.746582+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of SeekBrain: An Autonomous Multi-Agent System for Accelerating Neuroscience Discovery." pith.science (2026). https://pith.science/paper/7NE7X3Q2

@misc{pith2026260729347,
  author       = {Pith},
  title        = {Pith review of: SeekBrain: An Autonomous Multi-Agent System for Accelerating Neuroscience Discovery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7NE7X3Q2}},
  note         = {Machine review of arXiv:2607.29347}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Modern neuroscience relies on integrating multi-scale, multimodal datasets to uncover the neural principles underlying intelligence. However, analytical challenges posed by highly heterogeneous data and fragmented workflows increasingly constrain discoveries. Here we introduce SeekBrain, an autonomous multi-agent framework designed to accelerate neuroscience discovery through domain-grounded hierarchical planning and cross-modal data analysis. SeekBrain dynamically constructs a repertoire of analysis recipes extracted from code-paper pairs. By coupling this codified expertise with agentic planning and execution engines, the framework scalably generates hypotheses and analytical pipelines on demand. Systematic evaluation on the expert-annotated BrainArena benchmark demonstrates that SeekBrain substantially outperforms state-of-the-art agent baselines across various analysis tasks. Crucially, when deployed in real-world research, SeekBrain integrated behavioral, neural, and anatomical data to reveal structured, distributed neural representations of larval zebrafish behavior and a shared axis of regional decoding strength across the brain in a mouse decision-making task. These results establish SeekBrain as a scalable and practical tool for accelerating data-driven discoveries in neuroscience.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 3 linked inside Pith

  1. [1]

    A multimodal cell census and atlas of the mammalian primary motor cortex

    BRAIN Initiative Cell Census Network (BICCN). “A multimodal cell census and atlas of the mammalian primary motor cortex”. In:Nature598.7879 (2021), pp. 86–102

    2021

  2. [2]

    Functional neuroimaging as a catalyst for integrated neuroscience

    Emily S. Finn, Russell A. Poldrack, and James M. Shine. “Functional neuroimaging as a catalyst for integrated neuroscience”. In:Nature623.7986 (2023), pp. 263–273

    2023

  3. [3]

    A high-resolution transcriptomic and spatial atlas of cell types in the whole mouse brain

    Zizhen Yao, Cindy T. J. van Velthoven, Michael Kunst, Meng Zhang, Delissa McMillen, et al. “A high-resolution transcriptomic and spatial atlas of cell types in the whole mouse brain”. In: Nature624.7991 (2023), pp. 317–332

    2023

  4. [4]

    Functional connectomics spanning multiple areas of mouse visual cortex

    The MICrONS Consortium. “Functional connectomics spanning multiple areas of mouse visual cortex”. In:Nature640.8058 (2025), pp. 435–447

    2025

  5. [5]

    Neu- ronal wiring diagram of an adult brain

    Sven Dorkenwald, Arie Matsliah, Amy R. Sterling, Philipp Schlegel, Szi-chieh Yu, et al. “Neu- ronal wiring diagram of an adult brain”. In:Nature634.8032 (2024), pp. 124–138

    2024

  6. [6]

    A brain-wide map of neural activity during complex behaviour

    International Brain Laboratory, Dora Angelaki, Brandon Benson, Julius Benson, Daniel Birman, et al. “A brain-wide map of neural activity during complex behaviour”. In:Nature645.8079 (2025), pp. 177–191

    2025

  7. [7]

    Integrating multimodal data to understand cortical circuit architecture and function

    Anton Arkhipov, Nuno da Costa, Saskia de Vries, Trygve Bakken, Corbett Bennett, et al. “Integrating multimodal data to understand cortical circuit architecture and function”. In: Nature Neuroscience28.4 (2025), pp. 717–730

    2025

  8. [8]

    Joint modelling of brain and behaviour dynamics with artificial intelligence

    Mackenzie Weygandt Mathis and Alexander Mathis. “Joint modelling of brain and behaviour dynamics with artificial intelligence”. In:Nature Reviews Neuroscience27.2 (2026), pp. 87–100

    2026

  9. [9]

    A multi-agent system for automating scientific discovery

    Ali Essam Ghareeb, Benjamin Chang, Ludovico Mitchener, Angela Yiu, Caralyn J. Szostkiewicz, et al. “A multi-agent system for automating scientific discovery”. In:Nature655.8122 (2026), pp. 497–505

    2026

  10. [10]

    Towards end-to- end automation of AI research

    Chris Lu, Cong Lu, Robert Tjarko Lange, Yutaro Yamada, Shengran Hu, et al. “Towards end-to- end automation of AI research”. In:Nature651.8107 (2026), pp. 914–919

    2026

  11. [11]

    Accelerating scientific discovery with Co-Scientist

    Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Petar Sirkovic, et al. “Accelerating scientific discovery with Co-Scientist”. In:Nature655.8122 (2026), pp. 487–496

    2026

  12. [12]

    Cellvoyager: Ai compbio agent generates new insights by autonomously analyzing biological data

    Samuel Alber, Bowen Chen, Eric Sun, Alina Isakova, Aaron J. Wilk, et al. “Cellvoyager: Ai compbio agent generates new insights by autonomously analyzing biological data”. In:Nature Methods23.4 (2026), pp. 749–759

    2026

  13. [13]

    An AI system to help scientists write expert-level empirical software

    Eser Aygün, Anastasiya Belyaeva, Gheorghe Comanici, Marc Coram, Hao Cui, et al. “An AI system to help scientists write expert-level empirical software”. In:Nature654.8120 (2026), pp. 909–916

    2026

  14. [14]

    InternAgent-1.5: A Unified Agentic Framework for Long-Horizon Autonomous Scientific Discovery

    Shiyang Feng, Runmin Ma, Xiangchao Yan, Yue Fan, Yusong Hu, et al. “InternAgent-1.5: A Unified Agentic Framework for Long-Horizon Autonomous Scientific Discovery”. In:arXiv preprint arXiv:2602.08990(2026)

    arXiv 2026

  15. [15]

    InternA- gent: When Agent Becomes the Scientist–Building Closed-Loop System from Hypothesis to Verification

    InternAgent Team, Bo Zhang, Shiyang Feng, Xiangchao Yan, Jiakang Yuan, et al. “InternA- gent: When Agent Becomes the Scientist–Building Closed-Loop System from Hypothesis to Verification”. In:arXiv preprint arXiv:2505.16938(2025)

    Pith/arXiv arXiv 2025

  16. [16]

    Lei Bai, Zongsheng Cao, Yang Chen, Zhiyao Cui, Shangheng Du, et al.Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent. 2026. arXiv: 2606.30616[cs.CL].url:https://arxiv.org/abs/2606.30616. 17

    Pith/arXiv arXiv 2026

  17. [17]

    Predicting new research directions in materials science using large language models and concept graphs

    Thomas Marwitz, Alexander Colsmann, Ben Breitung, Christoph Brabec, Christoph Kirchlech- ner, et al. “Predicting new research directions in materials science using large language models and concept graphs”. In:Nature Machine Intelligence8.4 (2026), pp. 535–544

    2026

  18. [18]

    Large language models surpass human experts in predicting neuroscience results

    Xiaoliang Luo, Akilles Rechardt, Guangzhi Sun, Kevin K. Nejad, Felipe Yáñez, et al. “Large language models surpass human experts in predicting neuroscience results”. In:Nature Human Behaviour9.2 (2025), pp. 305–315.issn: 2397-3374

    2025

  19. [19]

    SCANPY: large-scale single-cell gene expression data analysis

    F Alexander Wolf, Philipp Angerer, and Fabian J Theis. “SCANPY: large-scale single-cell gene expression data analysis”. In:Genome Biology19.1 (2018), p. 15

    2018

  20. [20]

    Autonomous biomedical research with an artificial intelligence agent

    Kexin Huang, Serena Zhang, Hanchen Wang, Yuanhao Qu, Yingzhou Lu, et al. “Autonomous biomedical research with an artificial intelligence agent”. In:Science(2026), eadz4351.doi: 10.1126/science.adz4351

    doi:10.1126/science.adz4351 2026

  21. [21]

    SciToolAgent: a knowledge- graph-driven scientific agent for multitool integration

    Keyan Ding, Jian Yu, Jiaxin Huang, Yuxin Yang, Qi Zhang, et al. “SciToolAgent: a knowledge- graph-driven scientific agent for multitool integration”. In:Nature Computational Science5 (2025), pp. 962–972

    2025

  22. [22]

    GeneAgent: self- verification language agent for gene-set analysis using domain databases

    Zhizheng Wang, Qiao Jin, Chih-Hsuan Wei, Shubo Tian, Po-Ting Lai, et al. “GeneAgent: self- verification language agent for gene-set analysis using domain databases”. In:Nature Methods 22 (2025), pp. 1677–1685

    2025

  23. [23]

    Risks of AI scientists: prioritizing safeguarding over autonomy

    Xiangru Tang, Qiao Jin, Kunlun Zhu, Tongxin Yuan, Yichi Zhang, et al. “Risks of AI scientists: prioritizing safeguarding over autonomy”. In:Nature Communications16 (2025), p. 8317

    2025

  24. [24]

    Artificial intelligence and illusions of understanding in scientific research

    Lisa Messeri and M. J. Crockett. “Artificial intelligence and illusions of understanding in scientific research”. In:Nature627 (2024), pp. 49–58

    2024

  25. [25]

    Anthropic.Claude Opus 4.7 System Card. Tech. rep. Accessed: 2026-06-29. Anthropic, 2026. url:https://www.anthropic.com/claude-opus-4-7-system-card

    2026

  26. [26]

    OpenAI.GPT -5.5 System Card. Tech. rep. Accessed: 2026-06-29. OpenAI, 2026.url:https: //openai.com/index/gpt-5-5-system-card/

    2026

  27. [27]

    Deconstructing Hunting Behavior Reveals a Tightly Coupled Stimulus-Response Loop

    Duncan S. Mearns, Joseph C. Donovan, António M. Fernandes, Julia L. Semmelhack, and Herwig Baier. “Deconstructing Hunting Behavior Reveals a Tightly Coupled Stimulus-Response Loop”. In:Current Biology30.1 (2020), 54–69.e9

    2020

  28. [28]

    Learnable latent embed- dings for joint behavioural and neural analysis

    Steffen Schneider, Jin Hwa Lee, and Mackenzie Weygandt Mathis. “Learnable latent embed- dings for joint behavioural and neural analysis”. In:Nature617.7960 (2023), pp. 360–368

    2023

  29. [29]

    Brain-wide representations of prior information in mouse decision-making

    Charles Findling, Félix Hubert, International Brain Laboratory, Luigi Acerbi, Brandon Benson, et al. “Brain-wide representations of prior information in mouse decision-making”. In:Nature 645.8079 (2025), pp. 192–200

    2025

  30. [30]

    A benchmark of expert-level academic questions to assess AI capabilities

    Center for AI Safety, Scale AI, and HLE Contributors Consortium. “A benchmark of expert-level academic questions to assess AI capabilities”. In:Nature649.8099 (2026), pp. 1139–1146

    2026

  31. [31]

    SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

    Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, et al. “SWE-bench: Can Language Models Resolve Real-World GitHub Issues?” In:arXiv preprint arXiv:2310.06770 (2023)

    Pith/arXiv arXiv 2023

  32. [32]

    MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering

    Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, et al. “MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering”. In:International Conference on Learning Representations. 2025

    2025

  33. [33]

    Spontaneous behaviors drive multidimensional, brainwide activity

    Carsen Stringer, Marius Pachitariu, Nicholas Steinmetz, Charu Bai Reddy, Matteo Carandini, et al. “Spontaneous behaviors drive multidimensional, brainwide activity”. In:Science364.6437 (2019), eaav7893. 18

    2019

  34. [34]

    Single-trial neural dynamics are dominated by richly varied movements

    Simon Musall, Matthew T. Kaufman, Ashley L. Juavinett, Steven Gluf, and Anne K. Church- land. “Single-trial neural dynamics are dominated by richly varied movements”. In:Nature Neuroscience22.10 (2019), pp. 1677–1686

    2019

  35. [35]

    Distributed coding of choice, action and engagement across the mouse brain

    NicholasA.Steinmetz,PeterZatka-Haas,MatteoCarandini,andKennethD.Harris.“Distributed coding of choice, action and engagement across the mouse brain”. In:Nature576.7786 (2019), pp. 266–273

    2019

  36. [36]

    The Neurodata Without Borders ecosystem for neurophysiological data science

    Oliver Rübel, Andrew Tritt, Ryan Ly, et al. “The Neurodata Without Borders ecosystem for neurophysiological data science”. In:eLife11 (2022), e78362

    2022

  37. [37]

    Google.Gemini 3.1 Pro Model Card. Tech. rep. Accessed: 2026-06-29. Anthropic, 2026.url: https://storage.googleapis.com/deepmind-media/Model-Cards/Gemini-3- 1-Pro-Model-Card.pdf

    2026

  38. [38]

    ReAct:SynergizingReasoning and Acting in Language Models

    ShunyuYao,JeffreyZhao,DianYu,NanDu,IzhakShafran,etal.“ReAct:SynergizingReasoning and Acting in Language Models”. In:International Conference on Learning Representations. 2023

    2023

  39. [39]

    Brain-wide representations of behavior spanning multiple timescales and states in C. elegans

    Adam A. Atanas, Jungsoo Kim, Ziyu Wang, et al. “Brain-wide representations of behavior spanning multiple timescales and states in C. elegans”. In:Cell186.19 (2023), 4134–4151.e31

    2023

  40. [40]

    Plastic landmark anchoring in zebrafish compass neurons

    Ryosuke Tanaka and Ruben Portugues. “Plastic landmark anchoring in zebrafish compass neurons”. In:Nature650.8102 (2026), pp. 673–680

    2026

  41. [41]

    Multimodal cue integration and learning in a neural representation of head direction

    Melanie A. Basnak, Anna Kutschireiter, Tatsuo S. Okubo, Albert Chen, Pavel Gorelik, et al. “Multimodal cue integration and learning in a neural representation of head direction”. In: Nature Neuroscience28.8 (2025), pp. 1729–1740

    2025

  42. [42]

    Continuous cell-type diversification in mouse visual cortex development

    Yuan Gao, Cindy T. J. van Velthoven, Changkyu Lee, et al. “Continuous cell-type diversification in mouse visual cortex development”. In:Nature647.8088 (2025), pp. 127–142

    2025

  43. [43]

    The dynamics and geometry of choice in the premotor cortex

    Mikhail Genkin, Krishna V. Shenoy, Chandramouli Chandrasekaran, and Tatiana A. Engel. “The dynamics and geometry of choice in the premotor cortex”. In:Nature645.8079 (2025), pp. 168–176

    2025

  44. [44]

    Cholinergicfeedbackformodality- and context-specific modulation of sensory representations

    BinYu,YuxuanYue,ChiRen,RuiYun,ByungkookLim,etal.“Cholinergicfeedbackformodality- and context-specific modulation of sensory representations”. In:Science388.6753 (2025), pp. 1324–1329

    2025

  45. [45]

    Structural and genetic determinants of zebrafish functional brain networks

    AntoineLégaré,MadoLemieux,VincentBoily,SandrinePoulin,ArthurLégaré,etal.“Structural and genetic determinants of zebrafish functional brain networks”. In:Science Advances11.28 (2025), eadv7576

    2025

  46. [46]

    Whole-Brain Co-Mapping of Gene Expression and Neuronal Activity at Cellular Resolution in Behaving Zebrafish

    Emmanuel Marquez-Legorreta, Greg M. Fleishman, Luuk W. Hesselink, et al. “Whole-Brain Co-Mapping of Gene Expression and Neuronal Activity at Cellular Resolution in Behaving Zebrafish”. In:bioRxiv(2026)

    2026

  47. [47]

    Anthropic.Claude Opus 4.6 System Card. Tech. rep. Accessed: 2026-06-29. Anthropic, 2026. url:https://www.anthropic.com/claude-opus-4-6-system-card. 19 A. Supplementary Information A.1. Supplementary Notes A.1.1. Implementation details of SeekBrain This Supplementary Note provides the implementation details of analysis execution engine. For each task, Seek...

    2026

  48. [48]

    Rubric(JSON): Ground-truth conclusion, details scoring criteria, point allocations (max_point), and deduction rules (deduction_rule)

  49. [49]

    reasoning

    Model’s reproduced code(Python): The executable analysis and visualization code used to generate the reproduced figure 5.Model’s submission(Text): The conclusion or methods summary generated by the model Grading Rules: • Themaximumtotalscoreis {max_score}. Assigneachrubricitemascorefrom0uptoitsspecified max_point. •Award partial credit if the submission p...

    2025