Formalizes interface-constrained semi-Markov decision processes and proves a finite-sample bound for neural IC-Q that decomposes into neural approximation error, interface gap, and mixing-time residual, with experiments showing parity to centralized oracles.
super hub Canonical reference
Towards an AI co-scientist
Canonical reference. 78% of citing Pith papers cite this work as background.
154
Pith papers citing it
2
external citations · Crossref
Background
78% of classified citations
abstract
Scientific discovery relies on scientists generating novel hypotheses that undergo rigorous experimental validation. To augment this process, we introduce an AI co-scientist, a multi-agent system built on Gemini 2.0. The AI co-scientist is intended to help uncover new, original knowledge and to formulate demonstrably novel research hypotheses and proposals, building upon prior evidence and aligned to scientist-provided research objectives and guidance. The system's design incorporates a generate, debate, and evolve approach to hypothesis generation, inspired by the scientific method and accelerated by scaling test-time compute. Key contributions include: (1) a multi-agent architecture with an asynchronous task execution framework for flexible compute scaling; (2) a tournament evolution process for self-improving hypotheses generation. Automated evaluations show continued benefits of test-time compute, improving hypothesis quality. While general purpose, we focus development and validation in three biomedical areas: drug repurposing, novel target discovery, and explaining mechanisms of bacterial evolution and anti-microbial resistance. For drug repurposing, the system proposes candidates with promising validation findings, including candidates for acute myeloid leukemia that show tumor inhibition in vitro at clinically applicable concentrations. For novel target discovery, the AI co-scientist proposed new epigenetic targets for liver fibrosis, validated by anti-fibrotic activity and liver cell regeneration in human hepatic organoids. Finally, the AI co-scientist recapitulated unpublished experimental results via a parallel in silico discovery of a novel gene transfer mechanism in bacterial evolution. These results, detailed in separate, co-timed reports, demonstrate the potential to augment biomedical and scientific discovery and usher an era of AI empowered scientists.
hub tools
citation-role summary
background 25
baseline 1
other 1
citation-polarity summary
claims ledger
- abstract Scientific discovery relies on scientists generating novel hypotheses that undergo rigorous experimental validation. To augment this process, we introduce an AI co-scientist, a multi-agent system built on Gemini 2.0. The AI co-scientist is intended to help uncover new, original knowledge and to formulate demonstrably novel research hypotheses and proposals, building upon prior evidence and aligned to scientist-provided research objectives and guidance. The system's design incorporates a generate, debate, and evolve approach to hypothesis generation, inspired by the scientific method and accele
authors
co-cited works
representative citing papers
AutoResearchBench is a new benchmark showing top AI agents achieve under 10% success on complex scientific literature discovery tasks that demand deep comprehension and open-ended search.
FermiLink is a unified AI agent framework that automates multidomain scientific simulations via separated package knowledge bases and a four-layer progressive disclosure mechanism, reproducing 56% of target figures in benchmarks and generating research-grade results on unpublished problems.
The SDE benchmark shows LLMs lag on scientific discovery tasks relative to general science tests, with diminishing scaling returns and shared weaknesses across models.
The authors create the first large-scale dataset and taxonomy of failure modes in multi-agent LLM systems to explain their limited performance gains.
An evaluation benchmark shows that AI models produce measurably better research hypotheses when allowed to search the literature themselves, and that the gain grows with model capability.
TruthInsightBench evaluates AI coding agents on open-ended scientific discovery across 40 blind tasks and finds a performance plateau where execution is competent but the discriminating acts of science (controls, robustness, falsifiability, generalization) are largely absent.
Code-level autonomous research loops exhibit 'algorithmic mode collapse'—semantic diversity drops while surface diversity remains high—causing a widening gap between in-loop and held-out performance.
Confine the LLM to compiling plan edits over a deterministic, versioned dataflow executor that never calls the model, so research guarantees hold by construction.
A token-based authorization system with simulation and human gates enables safe LLM-written ARTIQ control code execution on trapped-ion platforms while blocking unauthorized hardware access.
Coding agents rediscovering a particle observable in Geant4 simulations reach correct answers via incorrect mechanisms in 7 of 28 episodes, requiring separate measurement of mechanism fidelity and honesty.
PhySciBench benchmark shows current AI models achieve at most 33.5% accuracy on physical science tasks; DelveAgent framework improves accuracy by up to 7.5 points and cuts costs to one-third.
SciAgentArena is a new interactive benchmark for AI agents on scientific tasks that finds agents handle clear data-analysis workflows but struggle with novel insights, self-directed exploration, and open-ended questions.
InquiTree shows LLM agents suffer from degrading critical capabilities during extended scientific interactions and perform worse on papers published after their training cutoffs.
CollabSim is a new CSCW-grounded simulation framework that enables controlled multi-agent experiments to measure collaborative competence in LLM agents.
Introduces the Matter to Mechanism benchmark of 2,645 structured instances and a composite metric suite for evaluating AI co-scientists on problem-to-hypothesis reasoning in battery materials research.
Frontier LLMs reason well about past science but are near chance at judging whether specific advances will happen, systematically late on when, and overconfident — even with extra pre-cutoff knowledge.
Hybrid human-AI networks in 5x5 grids reached lower final polarization than human-only networks after eight rounds of opinion revision on polarizing topics.
Current AI agents fail to reliably invent transferable ML methods on a controlled 140-task benchmark, and more test-time compute or context does not remove the scientific-judgment bottleneck.
An interactive AI workbench for mathematicians achieves 48% on FrontierMath Tier 4 and helped solve open problems in early tests.
AI CFD Scientist autonomously discovers a Spalart-Allmaras runtime correction reducing lower-wall Cf RMSE by 7.89% on the periodic hill at Reh=5600 while using a vision-language gate to detect 14 of 16 silent failures missed by solver checks.
An LLM agent autonomously identifies and experimentally validates a previously unreported optical bilinear interaction on a physical platform.
RosettaSearch applies LLM-driven multi-objective search at inference time to improve backbone-conditioned protein sequences, recovering designs with 18-68% better structural fidelity and 2.5x higher success rates than single-pass models like LigandMPNN.
Kosmos is an AI scientist that maintains coherence over hundreds of agent steps via a shared world model, executes thousands of code lines and reads thousands of papers per run, and produces traceable reports with 79.4% statement accuracy according to independent reviewers.
citing papers explorer
-
Learning to Hand Off: Provably Convergent Workflow Learning under Interface Constraints
Formalizes interface-constrained semi-Markov decision processes and proves a finite-sample bound for neural IC-Q that decomposes into neural approximation error, interface gap, and mixing-time residual, with experiments showing parity to centralized oracles.
-
AutoResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery
AutoResearchBench is a new benchmark showing top AI agents achieve under 10% success on complex scientific literature discovery tasks that demand deep comprehension and open-ended search.
-
FermiLink: A Unified Agent Framework for Multidomain Autonomous Scientific Simulations
FermiLink is a unified AI agent framework that automates multidomain scientific simulations via separated package knowledge bases and a four-layer progressive disclosure mechanism, reproducing 56% of target figures in benchmarks and generating research-grade results on unpublished problems.
-
Evaluating Large Language Models in Scientific Discovery
The SDE benchmark shows LLMs lag on scientific discovery tasks relative to general science tests, with diminishing scaling returns and shared weaknesses across models.
-
Why Do Multi-Agent LLM Systems Fail?
The authors create the first large-scale dataset and taxonomy of failure modes in multi-agent LLM systems to explain their limited performance gains.
-
AgentIdeaBench: Benchmarking Scientific Ideation in the Agent Era
An evaluation benchmark shows that AI models produce measurably better research hypotheses when allowed to search the literature themselves, and that the gain grows with model capability.
-
TruthInsightBench: An Evidence-Grounded Benchmark for Automated Evaluation of Open-Ended Scientific Discovery Agents
TruthInsightBench evaluates AI coding agents on open-ended scientific discovery across 40 blind tasks and finds a performance plateau where execution is competent but the discriminating acts of science (controls, robustness, falsifiability, generalization) are largely absent.
-
Beneath the Diff: Diagnosing and Mitigating Algorithmic Mode Collapse in Code-Level Autonomous Research Loops
Code-level autonomous research loops exhibit 'algorithmic mode collapse'—semantic diversity drops while surface diversity remains high—causing a widening gap between in-loop and held-out performance.
-
Confining Nondeterminism: AI-Driven Research Systems as DBMSs for Reliable, Non-Wasteful, Transparent, and Collaborative Research [Vision]
Confine the LLM to compiling plan edits over a deterministic, versioned dataflow executor that never calls the model, so research guarantees hold by construction.
-
A hardware-safety-gated system for LLM-written native ARTIQ control code on a trapped-ion platform
A token-based authorization system with simulation and human gates enables safe LLM-written ARTIQ control code execution on trapped-ion platforms while blocking unauthorized hardware access.
-
Position: Correct Answer, Wrong Mechanism -- When AI Scientists Defend General Claims Their Own Data Contradicts
Coding agents rediscovering a particle observable in Geant4 simulations reach correct answers via incorrect mechanisms in 7 of 28 episodes, requiring separate measurement of mechanism fidelity and honesty.
-
Deep Research in Physical Sciences: A Multi-Agent Framework and Comprehensive Benchmark
PhySciBench benchmark shows current AI models achieve at most 33.5% accuracy on physical science tasks; DelveAgent framework improves accuracy by up to 7.5 points and cuts costs to one-third.
-
Benchmarking AI Agents for Addressing Scientific Challenges Across Scales
SciAgentArena is a new interactive benchmark for AI agents on scientific tasks that finds agents handle clear data-analysis workflows but struggle with novel insights, self-directed exploration, and open-ended questions.
-
InquiTree: Evaluating AI Agents in the Scientific Inquiry Loop with Paper-Derived Research Trees
InquiTree shows LLM agents suffer from degrading critical capabilities during extended scientific interactions and perform worse on papers published after their training cutoffs.
-
CollabSim: A CSCW-Grounded Methodology for Investigating Collaborative Competence of LLM Agents through Controlled Multi-Agent Experiments
CollabSim is a new CSCW-grounded simulation framework that enables controlled multi-agent experiments to measure collaborative competence in LLM agents.
-
Matter to Mechanism: A Benchmark for AI Co-Scientists in Materials and Battery Research
Introduces the Matter to Mechanism benchmark of 2,645 structured instances and a composite metric suite for evaluating AI co-scientists on problem-to-hypothesis reasoning in battery materials research.
-
Scientific reasoning does not reliably translate into scientific forecasting in frontier AI
Frontier LLMs reason well about past science but are near chance at judging whether specific advances will happen, systematically late on when, and overconfident — even with extra pre-cutoff knowledge.
-
An Experimental Method to Study Opinion Diffusion in Human-AI Hybrid Societies
Hybrid human-AI networks in 5x5 grids reached lower final polarization than human-only networks after eight rounds of opinion revision on polarizing topics.
-
MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI
Current AI agents fail to reliably invent transferable ML methods on a controlled 140-task benchmark, and more test-time compute or context does not remove the scientific-judgment bottleneck.
-
AI co-mathematician: Accelerating mathematicians with agentic AI
An interactive AI workbench for mathematicians achieves 48% on FrontierMath Tier 4 and helped solve open problems in early tests.
-
AI CFD Scientist: Toward Open-Ended Computational Fluid Dynamics Discovery with Physics-Aware AI Agents
AI CFD Scientist autonomously discovers a Spalart-Allmaras runtime correction reducing lower-wall Cf RMSE by 7.89% on the periodic hill at Reh=5600 while using a vision-language gate to detect 14 of 16 silent failures missed by solver checks.
-
End-to-end autonomous scientific discovery on a real optical platform
An LLM agent autonomously identifies and experimentally validates a previously unreported optical bilinear interaction on a physical platform.
-
RosettaSearch: Multi-Objective Inference-Time Search for Protein Sequence Design
RosettaSearch applies LLM-driven multi-objective search at inference time to improve backbone-conditioned protein sequences, recovering designs with 18-68% better structural fidelity and 2.5x higher success rates than single-pass models like LigandMPNN.
-
Kosmos: An AI Scientist for Autonomous Discovery
Kosmos is an AI scientist that maintains coherence over hundreds of agent steps via a shared world model, executes thousands of code lines and reads thousands of papers per run, and produces traceable reports with 79.4% statement accuracy according to independent reviewers.
-
AlphaEvolve: A coding agent for scientific and algorithmic discovery
AlphaEvolve is an LLM-orchestrated evolutionary coding agent that discovered a 4x4 complex matrix multiplication algorithm using 48 scalar multiplications, the first improvement over Strassen's algorithm in 56 years, plus optimizations for Google data centers and hardware.
-
Code Researcher: Deep Research Agent for Large Systems Code and Commit History
Code Researcher retrieves global context via multi-step reasoning on code semantics, patterns, and commit history to fix Linux kernel crashes, reaching 48% crash-resolution rate versus 31% for baselines.
-
When AI Co-Scientists Fail: SPOT-a Benchmark for Automated Verification of Scientific Research
SPOT shows that state-of-the-art AI models detect fewer than one in five known errors in full scientific papers, with precision below 7%.
-
Assessing the Chemical Intelligence of Large Language Models
Reasoning LLMs score 50-57% on ChemIQ, a new 816-question short-answer organic chemistry benchmark, while non-reasoning models score 3-7%, with first-time success in SMILES-to-IUPAC naming and NMR structure elucidation.
-
Human-LLM Compound System for Scientific Ideation through Facet Recombination and Novelty Evaluation
Scideator enables facet-based scientific ideation through LLM-driven extraction, human-guided recombination, analogous retrieval, and facet-grounded novelty verification, showing significantly higher creativity support than a baseline LLM in a user study with CS researchers.
-
AutoKD: Autonomous Knowledge Discovery
A six-agent LLM loop with a persistent insight graph produced cumulative empirical findings on Amazon, WVS, and SciSciNet data that blinded human and AI raters preferred over matched published gold claims in 65.9% of non-tie overall comparisons, with no literature-derived query left fully uncovered.
-
Hakken: Predicting future discoveries to fill the gaps in today's knowledge
A knowledge-prediction system generated 1.5 million aging-related hypotheses and produced two wet-lab-supported new interactions, TP53-BAMBI and RAF1-TNF.
-
TREMORS: An Agentic Assistant for Multi-Datacenter Seismic Data Acquisition
An LLM-driven agent parses natural language seismology requests into a structured schema and runs them through a deterministic multi-datacenter data retrieval graph.
-
Navigating Epistemic Monocultures in AI-Driven Science: A Simulation Study
A simulation finds uniform AI assistance helps scientific communities only for modular problems with moderate use, while personalized AI helps more broadly but requires institutional adaptation.
-
Improving the matrix multiplication exponent with modern optimization and AlphaEvolve
Using gradient-based optimization with JAX and the AlphaEvolve coding agent at recursion level 4, the authors improve the upper bound on the matrix multiplication exponent from 2.371339 to 2.371177.
-
HERMES: a multi-agent framework for structured knowledge extraction from ultra-long documents in geoscience
HERMES, a multi-agent LLM framework with evidence tracing, extracted 32,277 fossil taxa and 451,878 attributes from the 55-volume Treatise on Invertebrate Paleontology and transferred to paleomagnetism and geochemistry.
-
ARAC: Benchmarking Auto-Research's Alignment and Completeness on End-to-End Researchs
ARAC-Bench introduces a three-stage researcher-mimicking benchmark that scores autonomous research systems on proposal, experiment, and synthesis quality; evaluated systems reach at most 67.9 out of 100, and scores correlate with PhD rankings at 0.81 on average.
-
Beyond Solution-Centric Search: Adaptive Inquiry and Knowledge Revision for Autonomous ML Engineering
Iris, an inquiry-revision agent that maintains revisable task knowledge while probing unknowns, achieves 64.9% any-medal on MLE-Bench in 12 hours, the best reported among compared systems.
-
OrchestrXR: A Multi-Agent System for Idea-to-Prototype XR Study Authoring
OrchestrXR uses multi-agent orchestration with structured schemas to generate Unity XR study prototypes from ideas, supported by a user study with 12 researchers indicating effective support and intent preservation.
-
Autonomous Scientific Discovery via Iterative Meta-Reflection
DiscoPER uses code generation, statistical validation, and second-order meta-reflection on accumulated discoveries to recover 8 of 9 known ecological patterns on a new benchmark at 72.7% support rate.
-
Testing Frontier Large Language Models' Physics Literacy in Parallel Physical Worlds
Introduces an auditable four-stage diagnostic for LLM physics reasoning in novel frameworks and applies it to three parallel worlds, yielding pass rates of 6/15, 6/15, and 0/15 on frontier models with noted qualitative-quantitative asymmetry.
-
AutoTrainess: Teaching Language Models to Improve Language Models Autonomously
AutoTrainess exposes training operations via agent-computer interfaces and outperforms CLI-only baselines on PostTrainBench with scores of 26.94 vs 23.21 for GPT-5.4 and similar gains on other models.
-
One Reflection Is Not Enough: Self-Correcting Autonomous Research via Multi-Hypothesis Failure Attribution
SAGE with MHFA improves failure recovery in autonomous research agents, raising metrics-bearing outputs from 42% to 92% on a 12-topic benchmark versus single-reflection baselines.
-
Agentic-Ideation: Sample Efficient Agentic Trajectories Synthesis for Scientific Ideation Agents
Agentic-Ideation uses oracle-guided multi-agent synthesis to generate efficient training trajectories for scientific ideation agents, reporting 11.91% quality gains and over 10x sample efficiency versus workflow baselines.
-
Evidence-Informed LLM Beliefs for Continual Scientific Discovery
Evidence-informed belief updates make Bayesian surprise non-stationary in LLM hypothesis search, with embedding-based RAG identifying 37.5% spurious static surprisals and modified search (filtering plus diversity) yielding 30.62% higher accumulated non-stationary surprisal across five domains.
-
Socratic agents for autonomous scientific discovery in high-dimensional physical systems
AHOIS is a Socratic multi-agent AI that autonomously discovers and validates a random-interference encoding strategy for multimode fiber optics, achieving 76.97% MNIST and 83.17% Fashion-MNIST accuracy with 16x16 measurements of effective rank 56.9.
-
Narration-of-Thought: Inference-Time Scaffolding for Defeasible Ethical Reasoning in Large Language Models
Narration-of-Thought, an inference-time five-section narrative scaffold, reduces stakeholder collapse to under 1% and uncertainty suppression by 28–72 points across four frontier models with no training.
-
Thinking Like a Scientist? A Structural Study of LLM-Generated Research Methods
LLMs given only research questions from 1000 arXiv CS papers recommend a narrower set of methods than the original papers, with effective model-entity diversity dropping from 1232 to 59-96 and stronger agreement among LLMs than with papers.
-
Fantastic Scientific Agents and How to Build Them: AgentBuild for Rietveld Refinement
AgentBuild builds scientific agents from version-controlled contracts for Rietveld refinement, progressing through a signal-to-noise ladder to handle 4-hour scans while exposing workflow limits.
-
Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents
Exploratory interview study with 17 developers identifies four forms of emergent oversight work for software agents and documents situated challenges and heuristics.
-
AgentJet: A Distributed Swarm Training Framework for Agentic Reinforcement Learning
A client-server swarm framework lets arbitrary agent code run on detached machines while shared servers train it, and context-timeline merging cuts actor-update time by 6.25x on AppWorld.