LLM-driven evolutionary program synthesis discovers Generalized Superfast Encodings with exact distance 5 (and 6 on one instance) for molecular Hamiltonians, the first beyond distance 3.
hub Canonical reference
ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution
Canonical reference. 89% of citing Pith papers cite this work as background.
45
Pith papers citing it
Background
89% of classified citations
abstract
We introduce ShinkaEvolve: a new open-source framework leveraging large language models (LLMs) to advance scientific discovery with state-of-the-art performance and unprecedented efficiency. Recent advances in scaling inference time compute of LLMs have enabled significant progress in generalized scientific discovery. These approaches rely on evolutionary agentic harnesses that leverage LLMs as mutation operators to generate candidate solutions. However, current code evolution methods suffer from critical limitations: they are sample inefficient, requiring thousands of samples to identify effective solutions, and remain closed-source, hindering broad adoption and extension. ShinkaEvolve addresses these limitations, introducing three key innovations: a parent sampling technique balancing exploration and exploitation, code novelty rejection-sampling for efficient search space exploration, and a bandit-based LLM ensemble selection strategy. We evaluate ShinkaEvolve across diverse tasks, demonstrating consistent improvements in sample efficiency and solution quality. ShinkaEvolve discovers a new state-of-the-art circle packing solution using only 150 samples, designs high-performing agentic harnesses for AIME mathematical reasoning tasks, identifies improvements to ALE-Bench competitive programming solutions, and discovers novel mixture-of-expert load balancing loss functions that illuminate the space of optimization strategies. Our results demonstrate that ShinkaEvolve achieves broad applicability with exceptional sample efficiency. By providing open-source accessibility and cost-efficiency, this work democratizes open-ended discovery across diverse computational problems.
hub tools
citation-role summary
background 8
baseline 1
citation-polarity summary
representative citing papers
EinsteinArena is a platform for AI agents to collectively discover new mathematical results through open interaction, achieving 12 new state-of-the-art outcomes including raising the 11-dimensional kissing number lower bound from 593 to 604.
FunctionEvolve recovers 107 exact symbolic forms out of 129 synthetic tasks (82.9% SA@50) by using expression-tree structure for evolutionary search, parent selection, mutation, and coefficient scoring with LLMs.
GAMBLe decomposes ADRS into four parameters and an effective landscape, with experiments on 760+ runs across NP-hard problems showing no universal best generator or mechanism and potential gains of 13-67% from component choice.
Jitskit is an iterative LLM-based synthesis pipeline that generates key-value stores matching spec cards for YCSB workloads, resources, and properties, outperforming SOTA baselines on all 18 tested cases by up to 4.6x.
SMCEvolve applies Sequential Monte Carlo sampling to LLM program search with adaptive resampling, mutation mixtures, and convergence control, delivering finite-sample complexity bounds and benchmark gains over prior systems.
Current AI agents fail to reliably invent transferable ML methods on a controlled 140-task benchmark, and more test-time compute or context does not remove the scientific-judgment bottleneck.
CoupleEvo finds that sequential and iterative strategies for evolving LLM-based heuristics yield more stable and higher-quality solutions than an integrated strategy on coupled optimization problems.
DeInfer reduces parallel inference communication cost for decomposed LLMs by up to 78% by moving collective operations into the low-rank latent space and redesigning KV-cache reconstruction for static graph compatibility.
An LLM-powered agentic framework autonomously designs competitive and sometimes superior explainable algorithms for wireless PHY and MAC layer tasks.
k-server-bench formulates potential-function discovery for the k-server conjecture as a code-based inequality-satisfaction task; current agents fully solve the resolved k=3 case and reduce violations on the open k=4 case.
TTT-Discover applies test-time RL to set new state-of-the-art results on math inequalities, GPU kernels, algorithm contests, and single-cell denoising using an open model and public code.
Evolution Fine-Tuning trains LLMs on 156K trajectories spanning 371 tasks to achieve 10.22% average improvement on 22 held-out optimization tasks and match SOTA on select circle-packing problems when combined with test-time RL.
ARTS improves automated scientific discovery by using reasoning LMs with test-time training to separate hypothesis merit from execution quality in tree search, achieving 15.3% relative gains on 22 MLGym and MLEBench tasks.
Code evolution produces link prediction algorithms with average AUC of 0.915 versus 0.783 for human-designed methods across 580 networks, with better scalability.
REVES augments LLM post-training by decoupling revision and verification signals from successful multi-step trajectories, reporting +6.5 point gains on LiveCodeBench over RL baselines.
A parallel-tempering evolutionary framework for LLM hypothesis search improves both quality and diversity of candidates in molecular, equation, and algorithm discovery under fixed validation budgets.
EvoDrive presents an LLM-based agentic evolution framework that generates diverse safety-critical autonomous driving scenarios by maintaining a Pareto archive of attack-realism trade-offs using simulator feedback.
BenchEvolver evolves coding problem solutions to generate harder, valid tasks, producing LiveCodeBench-Plus where frontier models score 27.5-62.6% and enabling RL gains on held-out tests.
LLMs can forecast GPU kernel performance accurately enough to serve as selective surrogates, allowing kernel searches to consider more candidates and recover faster kernels under fixed GPU evaluation budgets.
Bidirectional Evolutionary Search augments autoregressive expansion with evolutionary recombination operators and dense backward subgoal feedback to produce better candidates than standard best-of-N or tree search for language model self-improvement.
A universal LLM optimizer for text artifacts achieves SOTA results on six tasks including tripling ARC-AGI accuracy and cutting cloud costs by 40% via cross-task transfer and side information.
DrugSAGE accumulates cross-task memory of skills, statistical evidence, and recurring errors to let LLM agents achieve top-ranked performance on molecular property prediction tasks with reduced or zero test-time search.
ToolMol integrates evolutionary algorithms with agentic LLMs and precise RDKit tools to optimize multi-objective drug properties, yielding ligands with over 10% better predicted binding affinity and 35% gains in absolute binding free energy on three protein targets.
citing papers explorer
-
Evolving Quantum Error-Correcting Encodings for Molecular Simulation
LLM-driven evolutionary program synthesis discovers Generalized Superfast Encodings with exact distance 5 (and 6 on one instance) for molecular Hamiltonians, the first beyond distance 3.
-
Harnessing the Collective Intelligence of AI Agents in the Wild for New Discoveries
EinsteinArena is a platform for AI agents to collectively discover new mathematical results through open interaction, achieving 12 new state-of-the-art outcomes including raising the 11-dimensional kissing number lower bound from 593 to 604.
-
FunctionEvolve: Structure-Guided Symbolic Regression with LLMs
FunctionEvolve recovers 107 exact symbolic forms out of 129 synthetic tasks (82.9% SA@50) by using expression-tree structure for evolutionary search, parent selection, mutation, and coefficient scoring with LLMs.
-
Don't Gamble, GAMBLe: An Analytical Framework for AI-Driven Research Systems
GAMBLe decomposes ADRS into four parameters and an effective landscape, with experiments on 760+ runs across NP-hard problems showing no universal best generator or mechanism and potential gains of 13-67% from component choice.
-
The Time is Here for Just-in-Time Systems: Challenges and Opportunities
Jitskit is an iterative LLM-based synthesis pipeline that generates key-value stores matching spec cards for YCSB workloads, resources, and properties, outperforming SOTA baselines on all 18 tested cases by up to 4.6x.
-
SMCEvolve: Principled Scientific Discovery via Sequential Monte Carlo Evolution
SMCEvolve applies Sequential Monte Carlo sampling to LLM program search with adaptive resampling, mutation mixtures, and convergence control, delivering finite-sample complexity bounds and benchmark gains over prior systems.
-
MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI
Current AI agents fail to reliably invent transferable ML methods on a controlled 140-task benchmark, and more test-time compute or context does not remove the scientific-judgment bottleneck.
-
CoupleEvo: Evolving Heuristics for Coupled Optimization Problems Using Large Language Models
CoupleEvo finds that sequential and iterative strategies for evolving LLM-based heuristics yield more stable and higher-quality solutions than an integrated strategy on coupled optimization problems.
-
Co-evolving Agent Architectures and Interpretable Reasoning for Automated Optimization
DeInfer reduces parallel inference communication cost for decomposed LLMs by up to 78% by moving collective operations into the low-rank latent space and redesigning KV-cache reconstruction for static graph compatibility.
-
The AI Telco Engineer: Toward Autonomous Discovery of Wireless Communications Algorithms
An LLM-powered agentic framework autonomously designs competitive and sometimes superior explainable algorithms for wireless PHY and MAC layer tasks.
-
$k$-server-bench: Automating Potential Discovery for the $k$-Server Conjecture
k-server-bench formulates potential-function discovery for the k-server conjecture as a code-based inequality-satisfaction task; current agents fully solve the resolved k=3 case and reduce violations on the open k=4 case.
-
Learning to Discover at Test Time
TTT-Discover applies test-time RL to set new state-of-the-art results on math inequalities, GPU kernels, algorithm contests, and single-cell denoising using an open model and public code.
-
Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks
Evolution Fine-Tuning trains LLMs on 156K trajectories spanning 371 tasks to achieve 10.22% average improvement on 22 held-out optimization tasks and match SOTA on select circle-packing problems when combined with test-time RL.
-
Learning the ARTS of Search for Automated Discovery
ARTS improves automated scientific discovery by using reasoning LMs with test-time training to separate hypothesis merit from execution quality in tree search, achieving 15.3% relative gains on 22 MLGym and MLEBench tasks.
-
Code evolution for link prediction in complex networks
Code evolution produces link prediction algorithms with average AUC of 0.915 versus 0.783 for human-designed methods across 580 networks, with better scalability.
-
REVES: REvision and VErification--Augmented Training for Test-Time Scaling
REVES augments LLM post-training by decoupling revision and verification signals from successful multi-step trajectories, reporting +6.5 point gains on LiveCodeBench over RL baselines.
-
Towards Diverse Scientific Hypothesis Search with Large Language Models
A parallel-tempering evolutionary framework for LLM hypothesis search improves both quality and diversity of candidates in molecular, equation, and algorithm discovery under fixed validation budgets.
-
EvoDrive: Pareto Evolution for Safety-Critical Autonomous Driving via Self-Improving LLM Agents
EvoDrive presents an LLM-based agentic evolution framework that generates diverse safety-critical autonomous driving scenarios by maintaining a Pareto archive of attack-realism trade-offs using simulator feedback.
-
BenchEvolver: Frontier Task Synthesis via Solution-Centric Evolution
BenchEvolver evolves coding problem solutions to generate harder, valid tasks, producing LiveCodeBench-Plus where frontier models score 27.5-62.6% and enabling RL gains on held-out tests.
-
GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimization
LLMs can forecast GPU kernel performance accurately enough to serve as selective surrogates, allowing kernel searches to consider more candidates and recover faster kernels under fixed GPU evaluation budgets.
-
Self-Improving Language Models with Bidirectional Evolutionary Search
Bidirectional Evolutionary Search augments autoregressive expansion with evolutionary recombination operators and dense backward subgoal feedback to produce better candidates than standard best-of-N or tree search for language model self-improvement.
-
optimize_anything: A Universal API for Optimizing any Text Parameter
A universal LLM optimizer for text artifacts achieves SOTA results on six tasks including tripling ARC-AGI accuracy and cutting cloud costs by 40% via cross-task transfer and side information.
-
DrugSAGE:Self-evolving Agent Experience for Efficient State-of-the-Art Drug Discovery
DrugSAGE accumulates cross-task memory of skills, statistical evidence, and recurring errors to let LLM agents achieve top-ranked performance on molecular property prediction tasks with reduced or zero test-time search.
-
ToolMol: Evolutionary Agentic Framework for Multi-objective Drug Discovery
ToolMol integrates evolutionary algorithms with agentic LLMs and precise RDKit tools to optimize multi-objective drug properties, yielding ligands with over 10% better predicted binding affinity and 35% gains in absolute binding free energy on three protein targets.
-
FlashEvolve: Accelerating Agent Self-Evolution with Asynchronous Stage Orchestration
FlashEvolve accelerates LLM agent self-evolution via asynchronous stage orchestration and inspectable language-space staleness handling, reporting 3.5-4.9x proposal throughput gains over synchronous baselines on GEPA workloads.
-
Open-Ended Task Discovery via Bayesian Optimization
Generate-Select-Refine is an open-ended Bayesian optimization method that generates tasks and concentrates evaluations on the best one with only logarithmic regret overhead relative to standard single-task optimization.
-
Agentic Architect: An Agentic AI Framework for Architecture Design Exploration and Optimization
An LLM-driven agentic system evolves microarchitectural policies for cache replacement, data prefetching, and branch prediction, producing designs that match or exceed prior state-of-the-art in IPC on standard benchmarks.
-
TurboEvolve: Towards Fast and Robust LLM-Driven Program Evolution
TurboEvolve improves LLM program evolution by running parallel islands with LLM-generated diverse candidates that carry self-assigned weights, an adaptive scheduler, and clustered seed injection to reach stronger solutions at lower evaluation budgets.
-
AI-Driven Research for Databases
Co-evolving LLM-generated solutions with their evaluators enables discovery of novel database algorithms that outperform state-of-the-art baselines, including a query rewrite policy with up to 6.8x lower latency.
-
DeepReviewer 2.0: A Traceable Agentic System for Auditable Scientific Peer Review
An agentic system produces traceable review packages and an un-finetuned 196B model using it covers more major issues than Gemini-3.1-Pro on 134 ICLR 2025 submissions while winning most blind comparisons to human committees.
-
Mathematical exploration and discovery at scale
AlphaEvolve rediscovered best-known solutions for most of 67 tested math problems and found improved solutions in several cases using LLM-guided evolutionary search.
-
Glia: A Human-Inspired AI for Automated Systems Design and Optimization
Glia deploys a multi-agent LLM workflow with reasoning, experimentation, and analysis agents to generate interpretable algorithms for request routing, scheduling, and auto-scaling in distributed GPU clusters, reaching human-expert performance levels.
-
EurekAgent: Agent Environment Engineering is All You Need For Autonomous Scientific Discovery
EurekAgent achieves new state-of-the-art results on mathematics, kernel engineering, and machine learning tasks by engineering agent environments for autonomous scientific discovery, including a 26-circle packing result at under $11 API cost.
-
Learning to Solve and Optimize by Evolving Code
CHECKMATE evolves correct high-performing solvers from formal specs and natural language descriptions, outperforming SOTA on configuration and scheduling problems.
-
Compute Allocation in Evolutionary Search: From Depth-Breadth to Multi-Armed Bandits
BaSE, a multi-armed bandit for LLM call allocation in evolutionary search, raises mean fitness 12.3% over island-protocol baselines across eight model-task pairs.
-
Declarative Data Services: Structured Agentic Discovery for Composing Data Systems
DDS introduces typed contracts at intent, operator DAG, skills, and runtime layers to bound agentic search for data system compositions, achieving convergence on a trading workload where unbounded iteration fails.
-
GEAR: Genetic AutoResearch for Agentic Code Evolution
GEAR applies genetic algorithms to maintain and evolve multiple research states in autonomous code agents, outperforming single-path baselines by continuing to discover improvements over extended runs.
-
PACEvolve++: Improving Test-time Learning for Evolutionary Search Agents
PACEvolve++ uses a phase-adaptive reinforcement learning advisor to decouple hypothesis selection from execution in LLM-driven evolutionary search, delivering faster convergence than prior frameworks on load balancing, recommendation, and protein tasks.
-
FunFuzz: An LLM-Powered Evolutionary Fuzzing Framework
FunFuzz uses parallel LLM islands with candidate migration and adaptive prompting to achieve higher compiler coverage and more unique internal failures than prior LLM fuzzers on GCC and Clang over 24-hour runs.
-
Formally Verified Code Synthesis for Structured Data Translation in a Medical Internet of Things
An LLM and evolutionary search system with formal verification generates code that translates device JSON to FHIR while provably satisfying the target schema, demonstrated on a pulse oximeter integration.
-
Algorithmic algorithm development with LLMs: A Case Study on LLM-Usage for Contraction Order Optimization in Tensor Networks
Case study applies verifier-guided LLM evolutionary agents to contraction-order optimization in tensor networks and concludes that human validation remains essential.
-
AI for Auto-Research: Roadmap & User Guide
AI can generate research artifacts faster than it can verify them, so across all eight lifecycle stages the credible deployment mode is human-governed collaboration rather than full autonomy.
-
Effective Harness Engineering for Algorithm Discovery with Coding Agents
Under fixed token budget on Circle Packing, deeper per-candidate reasoning beats generating more shallow candidates, and capable models produce evaluation hacks at higher rates.
-
AI for Mathematics: Progress, Challenges, and Prospects
AI for math combines task-specific architectures and general foundation models to support research and advance AI reasoning capabilities.
- Evolving Ensemble of Agents