REVIEW 13 cited by
SciAgent: Tool-augmented Language Models for Scientific Reasoning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Scientific reasoning poses an excessive challenge for even the most advanced Large Language Models (LLMs). To make this task more practical and solvable for LLMs, we introduce a new task setting named tool-augmented scientific reasoning. This setting supplements LLMs with scalable toolsets, and shifts the focus from pursuing an omniscient problem solver to a proficient tool-user. To facilitate the research of such setting, we construct a tool-augmented training corpus named MathFunc which encompasses over 30,000 samples and roughly 6,000 tools. Building on MathFunc, we develop SciAgent to retrieve, understand and, if necessary, use tools for scientific problem solving. Additionally, we craft a benchmark, SciToolBench, spanning five scientific domains to evaluate LLMs' abilities with tool assistance. Extensive experiments on SciToolBench confirm the effectiveness of SciAgent. Notably, SciAgent-Mistral-7B surpasses other LLMs with the same size by more than 13% in absolute accuracy. Furthermore, SciAgent-DeepMath-7B shows much superior performance than ChatGPT.
Forward citations
Cited by 13 Pith papers
-
Investigating the (De)Composition Capabilities of Large Language Models in Natural-to-Formal Language Conversion
LLMs show measurable deficiencies in both decomposition and composition during natural-to-formal conversion, with decomposition errors dominating, under the new DEDC evaluation framework.
-
RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
AI agents beat human ML experts on short (2-hour) research-engineering tasks, but human experts outperform agents given 8+ hour budgets, measured on seven new open-source RE-Bench environments.
-
NNStar: An end-to-end AI agent for nuclear matter and neutron star physics
NNStar packages the RMF-to-neutron-star pipeline as a portable LLM-agent skill, validated on TM1/NL3/FSU-δ6.7 and demonstrated by an autonomous σ6-extended TM1 fit.
-
SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents
SciAgent-8B, fine-tuned on trajectories synthesized from a tool dependency graph, outperforms Qwen3-VL-235B-Instruct on SciAgentBench, a new 259-task benchmark for multi-step scientific tool-use.
-
UGPhysics: A Comprehensive Benchmark for Undergraduate Physics Reasoning with Large Language Models
UGPhysics is a new bilingual benchmark of 5,520 undergraduate physics problems; the strongest tested LLM, OpenAI o1-mini, reaches only 49.8% accuracy.
-
Bel Esprit: Multi-Agent Framework for Building AI Model Pipelines
A multi-agent LLM framework builds AI model pipelines from ambiguous natural-language queries, improving exact-match accuracy from 15.7% to 25.2% on a 441-example benchmark.
-
Learning to Solve Domain-Specific Calculation Problems with Knowledge-Intensive Programs Generator
A pipeline that generates executable programs from domain knowledge documents and uses them with extracted variables to solve domain-specific calculation problems, improving accuracy over baselines in legal and medical QA.
-
An Auditable Agent Platform For Automated Molecular Optimisation
A hierarchical multi-agent LLM platform with recorded provenance improved average predicted binding affinity for AKT1 by 31%, while single-agent runs favored drug-likeness.
-
CheMatAgent: Enhancing LLMs for Chemistry and Materials Science through Tree-Search Based Tool Learning
CheMatAgent uses hierarchical Monte Carlo tree search with separate policy and execution models, plus trained reward models, to improve tool selection and parameter filling on a new chemistry benchmark, ChemToolBench.
-
Improving Physics Reasoning in Large Language Models Using Mixture of Refinement Agents
MoRA uses GPT-4o to detect miscomprehension, wrong-concept, and computational errors in open-source LLM solutions, then routes specialized agents to fix them, improving multiple-choice physics accuracy by up to 16 per...
-
Toward Efficient Agents: Memory, Tool learning, and Planning
A survey that organizes efficiency techniques for LLM agents into memory, tool learning, and planning, and consolidates benchmarks and metrics for measuring cost-performance trade-offs.
-
Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry
A community report describing 34 hackathon-built LLM applications for materials science and chemistry, with reflections on the event format and preliminary project results.
-
Scientific Hypothesis Generation and Validation: Methods, Datasets, and Future Directions
A survey of LLM-based hypothesis generation and validation whose taxonomy is useful in outline but whose citations and tool descriptions are unreliable.
Discussion (0). Continue with ORCID to comment.