The SDE benchmark shows LLMs lag on scientific discovery tasks relative to general science tests, with diminishing scaling returns and shared weaknesses across models.
hub Canonical reference
Autonomous chemical research with large language models
Canonical reference. 88% of citing Pith papers cite this work as background.
33
Pith papers citing it
734
external citations · Crossref
Background
88% of classified citations
hub tools
citation-role summary
background 7
dataset 1
citation-polarity summary
representative citing papers
LAB-Bench provides over 2,400 multiple-choice questions to measure LLM performance on real biology research tasks like literature recall, figure reading, database access, and sequence manipulation, with initial results compared against human expert biologists.
LADeQ is an LLM-driven workflow that autonomously discovers and implements approximation algorithms for CCSD and CISD calculations, delivering speedups while respecting user-specified error tolerances.
PhySciBench benchmark shows current AI models achieve at most 33.5% accuracy on physical science tasks; DelveAgent framework improves accuracy by up to 7.5 points and cuts costs to one-third.
TianJi-Environ is a WRF-Chem-based multi-agent AI framework for autonomous validation of atmospheric chemistry mechanisms through executable experiments and evidence assessment.
Starling, a multi-agent LLM system, extracts ~6.3 million nuanced structured records from PubMed across six tasks with reported error rates of 0.6-7.7%, lower than several curated databases.
AI CFD Scientist autonomously discovers a Spalart-Allmaras runtime correction reducing lower-wall Cf RMSE by 7.89% on the periodic hill at Reh=5600 while using a vision-language gate to detect 14 of 16 silent failures missed by solver checks.
Introduces QCalEval benchmark showing best zero-shot VLM score of 72.3 on quantum calibration plots, with fine-tuning and in-context learning effects varying by model type.
AI coding agents evolve simple ground-state protocols into improved versions for VQE, DMRG, and AFQMC on spin models and molecules by using executable energy scores under fixed compute budgets.
CACM improves language-based drug discovery agents by 36.4% via protocol auditing, a grounded diagnostician, and compressed static/dynamic/corrective memory channels that localize failures and bias corrections.
MatClaw shows a code-first LLM agent autonomously generating and executing workflows for ML force field training, Curie temperature prediction, and parameter search on CuInP2S6, succeeding on code but requiring interventions for tacit domain knowledge.
Domain-specialized small language models enable deterministic atomic-resolution scanning probe microscopy control with 99.3% command accuracy, lower computational cost, and better domain performance than larger general models.
Frozen embeddings from a pretrained heterogeneous graph transformer over a 6.9M-node metabolic-engineering knowledge graph predict fermentation titers at R²=0.41, outperforming tabular baselines (R²=0.24).
Introduces an auditable four-stage diagnostic for LLM physics reasoning in novel frameworks and applies it to three parallel worlds, yielding pass rates of 6/15, 6/15, and 0/15 on frontier models with noted qualitative-quantitative asymmetry.
AHOIS is a Socratic multi-agent AI that autonomously discovers and validates a random-interference encoding strategy for multimode fiber optics, achieving 76.97% MNIST and 83.17% Fashion-MNIST accuracy with 16x16 measurements of effective rank 56.9.
Cordon is a transactional runtime system that binds tool intents to reversible state, staged effects, and audit metadata to validate composed agent workflows before commit.
SIGA is a coding-agent adapter using retrieval, procedural memory, and validation gates that raises success rate on GEOS from 0.720 to 0.789 while cutting variance 16x and matching expert quality in minutes instead of hours.
An LLM-based self-evolving agent discovers a traveling-wave controller with body-frame guidance and yaw feedback that generalizes to unseen targets for an underactuated fluid swimmer.
LAP is a new protocol extending A2A and MCP with four physical-world primitives for agent-to-instrument interaction in autonomous laboratories.
ColPackAgent integrates a custom colpack Python package wrapping HOOMD-blue with MCP tools and an agent skill to enable reliable autonomous workflows for colloidal packing simulations across interactive, prompt-driven, and autoresearch modes.
LEAPBench shows trajectory scoring changes best-model rankings on 53% of tasks, LLMs do not beat Bayesian optimization, and domain-aware prompting underperforms domain-agnostic on biology tasks aligned with published literature.
ToolRosella converts 122 scientific GitHub repositories into 1,580 agent-callable tools at 61.5% success and 4.4× human speed, supporting 84% downstream task success.
ATG maintains explicit DAGs of subtasks to enable dependency tracking, parallel execution, and localized repair in LLM agents, outperforming baselines on three benchmarks with 7B-8B models.
An agentic HPC skill automates NEB microkinetics, recovers from common failures, and benchmarks ~12 universal MLIPs against DFT for CO2 sublimation on graphite.
citing papers explorer
-
Evaluating Large Language Models in Scientific Discovery
The SDE benchmark shows LLMs lag on scientific discovery tasks relative to general science tests, with diminishing scaling returns and shared weaknesses across models.
-
LAB-Bench: Measuring Capabilities of Language Models for Biology Research
LAB-Bench provides over 2,400 multiple-choice questions to measure LLM performance on real biology research tasks like literature recall, figure reading, database access, and sequence manipulation, with initial results compared against human expert biologists.
-
LLM-Guided Test-Time Discovery of Quantum-Chemical Approximation Algorithms
LADeQ is an LLM-driven workflow that autonomously discovers and implements approximation algorithms for CCSD and CISD calculations, delivering speedups while respecting user-specified error tolerances.
-
Deep Research in Physical Sciences: A Multi-Agent Framework and Comprehensive Benchmark
PhySciBench benchmark shows current AI models achieve at most 33.5% accuracy on physical science tasks; DelveAgent framework improves accuracy by up to 7.5 points and cuts costs to one-third.
-
TianJi-Environ: An Autonomous AI Scientist for Atmospheric Environmental Research
TianJi-Environ is a WRF-Chem-based multi-agent AI framework for autonomous validation of atmospheric chemistry mechanisms through executable experiments and evidence assessment.
-
Self-Driving Datasets: From 20 Million Papers to Nuanced Biomedical Knowledge at Scale
Starling, a multi-agent LLM system, extracts ~6.3 million nuanced structured records from PubMed across six tasks with reported error rates of 0.6-7.7%, lower than several curated databases.
-
AI CFD Scientist: Toward Open-Ended Computational Fluid Dynamics Discovery with Physics-Aware AI Agents
AI CFD Scientist autonomously discovers a Spalart-Allmaras runtime correction reducing lower-wall Cf RMSE by 7.89% on the periodic hill at Reh=5600 while using a vision-language gate to detect 14 of 16 silent failures missed by solver checks.
-
QCalEval: Benchmarking Vision-Language Models for Quantum Calibration Plot Understanding
Introduces QCalEval benchmark showing best zero-shot VLM score of 72.3 on quantum calibration plots, with fine-tuning and in-context learning effects varying by model type.
-
Optimizing ground state preparation protocols with autoresearch
AI coding agents evolve simple ground-state protocols into improved versions for VQE, DMRG, and AFQMC on spin models and molecules by using executable energy scores under fixed compute budgets.
-
Constraint-Aware Corrective Memory for Language-Based Drug Discovery Agents
CACM improves language-based drug discovery agents by 36.4% via protocol auditing, a grounded diagnostician, and compressed static/dynamic/corrective memory channels that localize failures and bias corrections.
-
MatClaw: An Autonomous Code-First LLM Agent for End-to-End Materials Exploration
MatClaw shows a code-first LLM agent autonomously generating and executing workflows for ML force field training, Curie temperature prediction, and parameter search on CuInP2S6, succeeding on code but requiring interventions for tacit domain knowledge.
-
Integrating Domain-Specialized Language Models with AI Measurement Tools for Deterministic Atomic-Resolution Experimentation
Domain-specialized small language models enable deterministic atomic-resolution scanning probe microscopy control with 99.3% command accuracy, lower computational cost, and better domain performance than larger general models.
-
Canopy: A Heterograph Foundation Model for Metabolic Engineering
Frozen embeddings from a pretrained heterogeneous graph transformer over a 6.9M-node metabolic-engineering knowledge graph predict fermentation titers at R²=0.41, outperforming tabular baselines (R²=0.24).
-
Testing Frontier Large Language Models' Physics Literacy in Parallel Physical Worlds
Introduces an auditable four-stage diagnostic for LLM physics reasoning in novel frameworks and applies it to three parallel worlds, yielding pass rates of 6/15, 6/15, and 0/15 on frontier models with noted qualitative-quantitative asymmetry.
-
Socratic agents for autonomous scientific discovery in high-dimensional physical systems
AHOIS is a Socratic multi-agent AI that autonomously discovers and validates a random-interference encoding strategy for multimode fiber optics, achieving 76.97% MNIST and 83.17% Fashion-MNIST accuracy with 16x16 measurements of effective rank 56.9.
-
Cordon: Semantic Transactions for Tool-Using LLM Agents
Cordon is a transactional runtime system that binds tool intents to reversible state, staged effects, and audit metadata to validate composed agent workflows before commit.
-
Auto-Configuring Scientific Simulators with Lightweight Coding-Agent Adapters
SIGA is a coding-agent adapter using retrieval, procedural memory, and validation gates that raises success rate on GEOS from 0.720 to 0.789 while cutting variance 16x and matching expert quality in minutes instead of hours.
-
Self-Evolving Scientific Agent Discovers Generalizable Physically-Reasoned Fluid Control
An LLM-based self-evolving agent discovers a traveling-wave controller with body-frame guidance and yaw feedback that generalizes to unseen targets for an underactuated fluid swimmer.
-
LAP: An Agent-to-Instrument Protocol for Autonomous Science
LAP is a new protocol extending A2A and MCP with four physical-world primitives for agent-to-instrument interaction in autonomous laboratories.
-
ColPackAgent: Agent-Skill-Guided Hard-Particle Monte Carlo Workflows for Colloidal Packing
ColPackAgent integrates a custom colpack Python package wrapping HOOMD-blue with MCP tools and an agent skill to enable reliable autonomous workflows for colloidal packing simulations across interactive, prompt-driven, and autoresearch modes.
-
LEAP: Trajectory-Level Evaluation of LLMs in Iterative Scientific Design
LEAPBench shows trajectory scoring changes best-model rankings on 53% of tasks, LLMs do not beat Bayesian optimization, and domain-aware prompting underperforms domain-agnostic on biology tasks aligned with published literature.
-
ToolRosella: Translating Code Repositories into Standardized Tools for Scientific Agents
ToolRosella converts 122 scientific GitHub repositories into 1,580 agent-callable tools at 61.5% success and 4.4× human speed, supporting 84% downstream task success.
-
Atomic Task Graph: A Unified Framework for Agentic Planning and Execution
ATG maintains explicit DAGs of subtasks to enable dependency tracking, parallel execution, and localized repair in LLM agents, outperforming baselines on three benchmarks with 7B-8B models.
-
Toward Exascale AI for Science: A Scalable AI Skill for Autonomous Microkinetics Discovery
An agentic HPC skill automates NEB microkinetics, recovers from common failures, and benchmarks ~12 universal MLIPs against DFT for CO2 sublimation on graphite.
-
Scientific discovery as meta-optimization: a combinatorial optimization case study
Introduces consensus objective aggregation for meta-optimization of scientific discovery and reports improved scaling and speedup for 3-SAT algorithm discovery using digital MemComputing machines.
-
Agentic Publication Protocol: An Attempt to Modernize Scientific Publication
Introduces the Agentic Publication Protocol (APP) as a repository-based standard for publishing papers together with reproducibility artifacts and agent instructions.
-
CMIP-Forge: An Agentic System that Retrieves, Computes, and Self-Reviews Climate Science
CMIP-Forge presents a retrieval-augmented agentic system with automated guardrails and adversarial self-review for autonomous execution of climate research tasks on CMIP6 literature and ESGF data.
-
My Chemical Harness: Evolutionary Molecular Design over Synthetic Pathways with Large Language Model Agents
My Chemical Harness performs evolutionary molecular design by searching over validated synthetic routes with LLMs restricted to high-level preferences, outperforming baselines on an sEH proxy task across multiple metrics.
-
Iteris: Agentic Research Loops for Computational Mathematics
Iteris, an agentic research system, produced evidence and drafts for two open computational math problems that were verified after human correction.
-
Sibyl-AutoResearch: Autonomous Research Needs Self-Evolving Trial-and-Error Harnesses, Not Paper Generators
Sibyl-AutoResearch introduces self-evolving trial-and-error harnesses with auditable conversion units that link trial signals to updated research behaviors and harness repairs in autonomous systems.
-
Bolek: A Multimodal Language Model for Molecular Reasoning
Bolek injects Morgan fingerprint embeddings into an instruction-tuned text model, then fine-tunes on molecular alignment and synthetic chain-of-thought tasks to improve performance and grounding on 15 TDC binary classification endpoints while generalizing to unseen tasks.
-
From Perception to Autonomous Computational Modeling: A Multi-Agent Approach
A multi-agent LLM framework autonomously completes the full computational mechanics pipeline from a photograph to a code-compliant engineering report on a steel L-bracket example.
-
Evolving Roles of LLMs in Scientific Innovation: Assistant, Collaborator, Scientist, and Evaluator
The paper proposes a four-role framework for LLMs in scientific innovation and reviews methods, benchmarks, and limitations across Assistant, Collaborator, Scientist, and Evaluator roles.