REVIEW 11 cited by
LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Scientific equation discovery is a fundamental task in the history of scientific progress, enabling the derivation of laws governing natural phenomena. Recently, Large Language Models (LLMs) have gained interest for this task due to their potential to leverage embedded scientific knowledge for hypothesis generation. However, evaluating the true discovery capabilities of these methods remains challenging, as existing benchmarks often rely on common equations that are susceptible to memorization by LLMs, leading to inflated performance metrics that do not reflect discovery. In this paper, we introduce LLM-SRBench, a comprehensive benchmark with 239 challenging problems across four scientific domains specifically designed to evaluate LLM-based scientific equation discovery methods while preventing trivial memorization. Our benchmark comprises two main categories: LSR-Transform, which transforms common physical models into less common mathematical representations to test reasoning beyond memorized forms, and LSR-Synth, which introduces synthetic, discovery-driven problems requiring data-driven reasoning. Through extensive evaluation of several state-of-the-art methods, using both open and closed LLMs, we find that the best-performing system so far achieves only 31.5% symbolic accuracy. These findings highlight the challenges of scientific equation discovery, positioning LLM-SRBench as a valuable resource for future research.
Forward citations
Cited by 11 Pith papers
-
LLM-Based Scientific Equation Discovery via Physics-Informed Token-Regularized Policy Optimization
PiT-PO adaptively fine-tunes an LLM during symbolic regression search using physics-validity and token-level redundancy constraints, reporting state-of-the-art benchmark results and a periodic-hill turbulence closure.
-
MOT-SR: Multi-Objective Tool-Augmented Scientific Equation Discovery with Large Language Models
MOT-SR combines tool-augmented data analysis with multi-objective Pareto selection to discover symbolic equations, outperforming LLM-based and classical SR baselines on benchmarks and an EMRI orbital-correction task.
-
Reference-Free Evaluation of Reasoning in Open-Ended Question Answering
An NLI-hypergraph audit with deterministic AND–OR search labels LLM reasoning segments as supported, unsupported, or orphaned, improving balanced F1 over LLM-as-judge on a new 40-case clinical benchmark.
-
Automatic Ordinary Differential Equations Discovery For Biological Systems Using Large Language Model Powered Agentic System
MEDA uses LLM agents to formalize biological constraints and guide symbolic regression, recovering correct ODE structures for canonical and extrapolated biological models, with structural recovery driven mainly by lit...
-
GAE: Graph-Augmented Evolution for Scientific Discovery via Reinforcement Optimization
GAE couples a relational GNN program encoder, a Discrete SAC mutation-type controller, and online GRPO LLM fine-tuning to beat static LLM evolution baselines on nonlinear-oscillator symbolic regression, especially out...
-
When Good Equations Get Bad Scores: Improving Symbolic Regression Through Better Parameter Optimization
SAGE-Fit improves symbolic regression evaluation by exploiting structural and semantic priors to enhance parameter optimization in non-convex inner-loop fitting.
-
FEM-Bench: A Structured Scientific Reasoning Benchmark for Evaluating Code-Generating LLMs
The paper introduces FEM-Bench, a 33-task computational mechanics benchmark, and shows that state-of-the-art LLMs complete at most 30/33 tasks with multiple attempts and fail entirely on geometric-stiffness-related tasks.
-
HypoSpace: A Diagnostic Benchmark for Set-Valued Hypothesis Generation under Underdetermination and Sublinear Coverage Bounds
A benchmark with exactly enumerated valid hypothesis sets shows LLMs maintain high validity but lose uniqueness and coverage as the admissible solution space grows.
-
From Canonical to Complex: Benchmarking LLM Capabilities in Undergraduate Thermodynamics
On a new 50-item thermodynamics benchmark, the best LLM scored 82%, below the authors' 95% tutoring-safety threshold, with diagram-based questions near chance.
-
Ascending the Infinite Ladder: Benchmarking Spatial Deformation Reasoning in Vision-Language Models
Vision-language models can handle some 2D shape puzzles but nearly all fail at multi-step 3D spatial deformation reasoning.
-
DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experience
DrSR improves LLM-based symbolic regression by adding data-aware structural insights and a reflective idea library, beating prior methods on six benchmark tasks.
Discussion (0). Continue with ORCID to comment.