Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Relay, Don't Route: Adaptive Population Handoff for Cost-Efficient LLM-Driven Evolution

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By organizing LLM-evolution budgets around the population handed between a cheap and a strong model, RelayEvolve reports the highest mean score in 11 of 12 benchmark–budget settings.

desk verdict A genuinely new population-handoff idea for cost-aware LLM evolution, but the 11-of-12 headline is underpowered with n=3 and no significance tests. read the letter →

arxiv 2608.05651 v1 pith:GKG63OLN submitted 2026-08-06 cs.CL cs.AIcs.NE

Sichun Luo , Yi Huang , Guanzhi Deng , Haibo Wang , Haochen Luo , Lei Li , Zefa Hu , Junlan Feng
show 1 more author
Qi Liu
This is my paper · ORCID
classification cs.CLcs.AIcs.NE
keywords LLM-drivenevolutionpopulationhandoffrelaygainbudgetallocationquality-diversitybanditschedulingprogramsearchcost-efficientinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RelayEvolve claims that in LLM-driven evolutionary search, the right unit of budget allocation is not the individual model call but the population of candidate programs passed from one model phase to the next. The paper supports this with empirical observations that search progress is front-loaded, early trajectory fitness is a noisy signal, and a cheap model recovers most of the early gains of a strong model at lower cost. On this basis it proposes a training-free scheme: a cheap model explores several trajectories in short blocks, a relay bank measures the marginal improvement of a compact quality-diverse candidate set, and this Relay Gain decides when to start or deepen trajectories and when to hand off to a strong model. The strong model then refines the curated seeds with the remaining budget, and the resulting method reports the highest mean score in 11 of 12 benchmark–budget settings.

What carries the argument

The central object is the relay bank and its Relay Gain. The relay value of a bank $S$ relative to a candidate pool $C$ is $F_C(S)=\lambda Q_r(S)+(1-\lambda)D^q_C(S)$, where $Q_r$ is the average of the top-$r$ normalized qualities and $D^q_C$ is the quality-weighted diversity coverage of the pool by the bank; because both terms are monotone submodular, $F_C$ is too. Relay Gain is the marginal improvement $g_t=F_{C_{t+1}}(S_{t+1})-F_{C_{t+1}}(S_t)$ after a block of cheap-model generations. This single set-level reward drives the Grow–Deepen bandit scheduler, the adaptive stopping rule, and the final greedy curation of the seed population handed to the strong model.

What would settle it

Measure, over many independent runs with the same budget, the correlation between the Relay Gain accumulated during the cheap phase and the additional fitness the strong model gains in the refinement phase; a null or negative correlation would show that Relay Gain is not measuring handoff value. Equivalently, hold the handoff point fixed and replace relay-gain scheduling with random Grow–Deepen choices under the same cheap budget: if performance does not drop, the reward signal is not what carries the method.

Watch

Extended reading notes

Core claim

At the level of the paper's own claims, the discovery is that stateful LLM evolution is better served by optimizing the population handed from a cheap exploration phase to a strong refinement phase than by routing each mutation to a weaker or stronger model. The authors define the relay objective $F_C(S)=\lambda Q_r(S)+(1-\lambda)D^q_C(S)$ over a compact bank, use its marginal increase (Relay Gain) as the reward for a Grow–Deepen bandit scheduler, and trigger handoff when recent relative gains fall below a threshold. The final seed set is chosen by greedy submodular selection on the same objective, then used to initialize a shared strong-model population. The experimental claim is that this population-handoff organization achieves the highest mean score in 11 of 12 settings across four program evolution benchmarks and three budgets.

Load-bearing premise

The load-bearing premise is that Relay Gain, the marginal improvement of a quality-diverse relay bank, faithfully measures how much the strong model will gain from the handed-off population, so that it is a valid reward for scheduling cheap-model blocks and timing the handoff.

Editorial extensions

If this is right

  • If RelayEvolve is right, then cost-aware evolutionary search should allocate budgets to phases and populations, and call-level routing methods that ignore population state will leave performance on the table.
  • Cheap models can front-load exploration cheaply, while strong model calls are best spent refining a compact, diverse set of promising candidates rather than scanning from scratch.
  • A single set-level reward (Relay Gain) can coordinate trajectory allocation, handoff timing, and seed selection, replacing separate heuristics for each decision.
  • Matching or outperforming both All-cheap and All-strong in most settings implies that the optimal model mix is task-dependent and that adaptive handoff is a practical way to find it under a fixed budget.
  • Because no training is required, the framework can be applied on top of existing LLM evolutionary backends and swapped model pairs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Relay Gain is used only as a scheduling reward in the paper; one natural extension would be to test it as a direct predictor of the strong model's downstream improvement, which would turn the scheduler into a forecasting device rather than a heuristic.
  • The same population-handoff objective could be applied with more than two model tiers or with heterogeneous block lengths, treating the handoff point as a continuous resource boundary rather than a single cheap-to-strong switch.
  • Because the handoff is stateful and population-based, RelayEvolve may be particularly compatible with multi-fidelity evaluators: relay gain measures improvement in the candidate bank independently of how each candidate was generated, so evaluation fidelity and model handoff could be scheduled together.
  • A testable extension is to hold the total budget and the final strong-model phase fixed while randomizing the cheap-model schedule; if relay-gain scheduling still wins over random scheduling, the claim that the reward signal carries the value is strengthened.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies cost allocation between cheap and strong LLMs in evolutionary program search. It argues that existing call-level routing ignores the stateful nature of evolution, in which each candidate changes the population inherited by later mutations. The authors propose RelayEvolve, a training-free framework in which a cheap model explores multiple trajectories in short blocks, a Grow–Deepen bandit scheduler allocates blocks using a "Relay Gain" reward defined as the marginal improvement of a quality-diverse candidate bank, and handoff to a strong model occurs when this reward saturates or the cheap-model budget is exhausted. The same submodular relay objective is reused for offline seed curation, and the strong model then refines the selected population with the remaining budget. The paper motivates the design with three empirical findings about front-loaded progress, noisy early trajectories, and cheap-model early competitiveness, then evaluates RelayEvolve on four benchmarks at three budget levels, claiming the highest mean score in 11 of 12 benchmark–budget settings.

Significance. If the empirical claim were robust, this would be a useful contribution: it reframes budget allocation around evolving populations rather than individual model calls, and the unified submodular relay objective is a clean and potentially reusable mechanism. The controlled experimental setup, with all methods sharing the same ShinkaEvolve backend, the same evaluator, and the same cost accounting, is a strength, as is the explicit attempt to coordinate trajectory allocation, stopping, and seed selection under one objective. However, the headline result rests on only three runs per cell with no significance tests, several winning margins are smaller than the reported standard deviations, and the method's many hyperparameters and code are not disclosed. These issues currently prevent the central empirical claim from being accepted as established, although they are fixable within the manuscript's scope.

major comments (4)
  1. [Table 1, Main Results] The headline claim of the highest mean score in 11 of 12 settings is supported by three independent runs per cell, with no significance tests or confidence intervals on the differences. Several reported margins are far smaller than the reported run-to-run standard deviations: in TXN Scheduling at 50% budget, RelayEvolve (3875.09 ± 153.52) is only 1.2 above All-cheap (3873.89 ± 129.43); in Circle Packing (Square) at 50% budget, RelayEvolve (2.4136 ± 0.1228) is only 0.0052 above All-strong (2.4084 ± 0.0237); and in Prism at 75% and 100% budgets, the mean margins over All-cheap are about 0.0044. In addition, the Prism best score 26.2560 is reached by multiple methods, so the 11/12 count partly reflects plateau ties. With n=3, a single seed can flip several of these outcomes. Please report per-seed results, paired or bootstrap significance tests, and confidence intervals on the differences, and increase the number of runs for at least the critical comparisons.
  2. [Method: Adaptive Cheap Model Search and Population Handoff] The method depends on at least ten hyperparameters—h, k, λ, η, c, w, p, ε_F, ε_rel, and the strong-model budget reserve—but no values or selection protocol are reported. The only value mentioned is the "default split of 0.85" in Figure 4(d). It is therefore unclear whether these parameters were fixed before the experiments, chosen on Circle Packing (Square) and then applied to other tasks, or tuned per cell; if any were selected after observing the full benchmark results, the comparison is optimistic. Please provide a full hyperparameter table and a clear tuning procedure, and release code or detailed pseudocode so that the reported numbers can be reproduced.
  3. [Method: Relay Objective and Relay Gain] Relay Gain (Eq. 7) is the marginal improvement of the relay bank under the quality-diversity objective F, and it drives both the bandit scheduler (Eqs. 8–9) and the adaptive stopping rule (Eq. 10). The paper's design claim is that this set-level reward is a faithful proxy for the value of a block's candidates for downstream strong-model refinement. The only direct evidence is the relay-mechanism ablation on Circle Packing (Square) at the 100% budget (Figure 4a), which is itself based on three runs and one task. A concrete validation would be to compute, for each block, the correlation between its relay gain and the eventual contribution of that block's candidates to the final strong-model population. Without such evidence, the scheduler and stopping rule remain heuristic, and the central thesis about population handoff is only partially supported.
  4. [Experiments: Experimental Setup] The budget comparison is not fully documented. The paper states that B = β B_all-strong and caps runs at N = 200 generations, but it does not report, for any method or cell, whether the run terminated by exhausting its monetary budget or by reaching the generation cap, nor how much of the budget was actually spent. This matters because All-cheap is said to coincide across budget levels at less than 0.5B_all-strong; if RelayEvolve or other methods are generation-cap-limited at some budgets, the "same budget" comparison does not hold as straightforwardly as stated. Please report realized costs per method and the termination cause per cell.
minor comments (5)
  1. [Ablation Study] "where the optimization landscape is sufficiency challenging" should read "sufficiently challenging."
  2. [Table 1 caption] The caption says "Best results in each budget–benchmark column are bolded," but the bolded entries in the table are the mean scores, and the "Best" column values are not bolded. Please clarify the bolding convention.
  3. [Figure 2(b)] The label "Cheap/Strong Gain @ Gen20" is not defined in the caption; please state explicitly that it is the ratio of cheap-model to strong-model best-so-far improvement over the first 20 generations.
  4. [Eq. (6)] The deduplication step is not defined; please specify the equivalence criterion used to decide that two program candidates are duplicates.
  5. [Conclusion] The abstract and conclusion state that RelayEvolve "achieves the highest mean score in 11 of 12 settings"; since the means are based on three runs and no significance tests are reported, please qualify the claim as the highest observed mean, or add significance statements.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RelayEvolve's closed-loop relay objective is a design consistency, not a derivation loop; the headline claim is an empirical comparison against external benchmarks and baselines.

full rationale

RelayEvolve's central claim is an empirical performance comparison, not a derivation or a prediction from first principles. The formal chain (Eqs. 1-13) is definitional but not circular: Eq. (7) defines relay gain as the marginal improvement of the relay bank under the monotone submodular objective F_C (Eq. 5); Eq. (8) normalizes it; Eq. (10) uses it as a handoff threshold; Eqs. (11)-(13) re-optimize the same objective for seed curation. That the same objective coordinates scheduling, stopping, and curation is a deliberate design feature, but the reported outcome is task fitness f, which is not equal to the relay objective F_C (a weighted combination of top-r quality and diversity coverage). Maximizing F_C therefore does not by construction determine the headline 11-of-12 result. The evaluation is self-contained against external reference points: tasks are drawn from the SkyDiscover suite (Liu et al. 2026c), the evolutionary backend is ShinkaEvolve (Lange, Imajuku, and Cetin 2026), and baselines (All-cheap, All-strong, Fixed-switch, Random, Bandit, LEVI) share the same budget and backend. Self-citations to SeaEvo, Harness-Aware Self-Evolving, and Cognitive Alpha Mining appear only in the introduction/related work and are not load-bearing for the method's correctness or for the benchmark comparison. The main epistemic risks—three runs per cell without significance tests, the 0.85 strong-model reserve tuned on Circle Packing (Square), and the unvalidated assumption that relay gain proxies downstream strong-model payoff—are correctness/reproducibility concerns, not instances of an output reducing to its inputs by construction. No equation-level or self-citation circularity is present.

Assumptions & free parameters 9 free parameters · 3 assumptions · 0 invented entities

The central claim depends on roughly a dozen unstated hyperparameters (lambda, r, eta, k, h, c, w, thresholds, patience) and a budget split (0.85) that was tuned on one of the four evaluation benchmarks. The method also assumes the quality-diversity objective is monotone submodular and that embedding similarity is a valid proxy for program diversity. No new physical entities are introduced.

free parameters (9)
  • Strong-model budget reserve (budget split) = 0.85
    Selected from ablation on Circle Packing (Square), one of the four evaluation benchmarks, then applied to all tasks; the paper states 'The default split of 0.85 achieves the best performance.'
  • lambda (quality-diversity weight)
    Weight balancing top-r quality and diversity coverage in Eq. (5); value not reported in the paper.
  • r (top-r quality horizon)
    Number of top candidates averaged in Eq. (2); value not reported.
  • eta (code/text embedding weight)
    Weight between code and text embedding cosine similarities in Eq. (3); value not reported.
  • k (relay bank capacity)
    Maximum size of the relay bank and final handoff seed set; value not reported.
  • h (block length)
    Number of generations per block for the cheap model; value not reported.
  • c (UCB exploration constant)
    Exploration coefficient in Eq. (9); value not reported.
  • w (recent reward window)
    Window size for recent-window UCB; value not reported.
  • epsilon_F, epsilon_rel, patience p
    Stabilizing floor, saturation threshold, and patience window for adaptive handoff; values not reported.
assumptions (3)
  • standard math FC is monotone submodular for any fixed reference pool C.
    Both Qr and Dq are asserted to be monotone submodular in the paper, enabling the (1-1/e) greedy guarantee (Nemhauser et al. 1978). This is a standard result for facility-location-like coverage and top-r sums.
  • domain assumption The similarity metric sim(x,x') based on code and text embeddings captures useful program diversity.
    The coverage term Eq. (4) and the relay objective rely on this embedding similarity to measure diversity; no evidence is provided that this corresponds to meaningful search-space coverage.
  • domain assumption The front-loading and cheap-model competitiveness findings hold across tasks, model pairs, and search backends.
    The empirical motivation uses one cheap model, one strong model, and three tasks; the method and its scheduler assume these patterns are general enough to make relay-gain an effective reward.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Relay, Don't Route: Adaptive Population Handoff for Cost-Efficient LLM-Driven Evolution." pith.science (2026). https://pith.science/paper/GKG63OLN

@misc{pith2026260805651,
  author       = {Pith},
  title        = {Pith review of: Relay, Don't Route: Adaptive Population Handoff for Cost-Efficient LLM-Driven Evolution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GKG63OLN}},
  note         = {Machine review of arXiv:2608.05651}
}
read the original abstract

Large language model (LLM)-driven evolution has shown promise for program search and algorithm discovery, but relying on strong models throughout long evolutionary runs is costly. A natural alternative is to combine cheap and strong models under a fixed inference budget. However, existing approaches typically allocate models at the level of individual queries or mutation steps, overlooking that evolutionary search is \textit{stateful}: each generated candidate changes the population from which subsequent mutations are produced. We empirically analyze LLM-driven evolutionary trajectories and find that search progress is strongly front-loaded, early trajectory performance is informative but noisy, and cheap models recover much of the early progress achieved by strong models at lower cost. Motivated by these findings, we propose \textbf{\model}, a training-free framework that shifts budget allocation from individual calls to evolving populations through adaptive \textit{population handoff}. A cheap model explores multiple trajectories in short blocks allocated by a bandit scheduler. Relay Gain, defined as the marginal improvement of a compact, quality-diverse candidate bank constructed for handoff, serves as the scheduler reward and determines when to hand off. The curated candidates initialize a shared strong model population for refinement. Across four benchmarks and three budgets, \model achieves the highest mean score in 11 of 12 settings, outperforming competitive baselines. Our results suggest that in stateful search, budget allocation should be organized around the population, not the individual call.

Figures

Figures reproduced from arXiv: 2608.05651 by the authors.

Figure 1
Figure 1. (a) Fraction of each run’s eventual best-so-far fit [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. (a) Spearman correlation between cheap model [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of RelayEvolve. A cheap LLM explores multiple trajectories in short blocks allocated by a Grow–Deepen bandit, whose reward is the Relay Gain of an online quality-diverse bank. Cheap model evolution stops when Relay Gain saturates or its budget is exhausted. RelayEvolve then reruns greedy submodular curation on the full candidate pool and hands the resulting compact seed set to a shared strong LLM population… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Ablation studies on Circle Packing (Square) at the 100% budget level. From left to right: [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Cost–performance comparison on Circle Packing [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Optimize Cheap, Deploy Strong: Cost-Aware Cross-Tier Transfer for Evolutionary Optimization

    cs.LG 2026-08 conditional novelty 6.0 of 10

    Searching for prompts with a cheap evaluator and a strong reflector matches or beats optimizing directly on the deployment tier, at 5.6 to 14 times lower search cost.

Reference graph

Works this paper leans on

54 extracted references · 28 canonical work pages · cited by 1 Pith paper

  1. [1]

    A.; Tan, S.; Soylu, D.; Ziems, N.; Khare, R.; Opsahl-Ong, K.; Singhvi, A.; Shandilya, H.; Ryan, M

    Agrawal, L. A.; Tan, S.; Soylu, D.; Ziems, N.; Khare, R.; Opsahl-Ong, K.; Singhvi, A.; Shandilya, H.; Ryan, M. J.; Jiang, M.; Potts, C.; Sen, K.; Dimakis, A.; Stoica, I.; Klein, D.; Zaharia, M.; and Khattab, O. 2026. GEPA : Reflective Prompt Evolution Can Outperform Reinforcement Learning. In The Fourteenth International Conference on Learning Representations

    work page 2026

  2. [3]

    Besbes, O.; Gur, Y.; and Zeevi, A. 2014. Stochastic multi-armed-bandit problem with non-stationary rewards. Advances in neural information processing systems, 27

    work page 2014

  3. [5]

    E.; Sen, K.; Zaharia, M.; Dimakis, A.; and Stoica, I

    Cemri, M.; Agrawal, S.; Gupta, A.; Liu, S.; Cheng, A.; Mang, Q.; Naren, A.; Erdogan, L. E.; Sen, K.; Zaharia, M.; Dimakis, A.; and Stoica, I. 2026. AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization. arXiv:2602.20133

    arXiv 2026

  4. [7]

    V.; and Awadallah, A

    Ding, D.; Mallick, A.; Wang, C.; Sim, R.; Mukherjee, S.; R \"u hle, V.; Lakshmanan, L. V.; and Awadallah, A. H. 2024. Hybrid llm: Cost-efficient and quality-aware query routing. In The Twelfth International Conference on Learning Representations

    work page 2024

  5. [8]

    Guo, Q.; Wang, R.; Guo, J.; Li, B.; Song, K.; Tan, X.; Liu, G.; Bian, J.; and Yang, Y. 2024. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. In International Conference on Learning Representations, volume 2024, 34133--34156

    work page 2024

  6. [9]

    T.; Imajuku, Y.; and Cetin, E

    Lange, R. T.; Imajuku, Y.; and Cetin, E. 2026. ShinkaEvolve: Towards Open-Ended and Sample-Efficient Program Evolution. In The Fourteenth International Conference on Learning Representations

    work page 2026

  7. [10]

    Li, L.; Jamieson, K.; DeSalvo, G.; Rostamizadeh, A.; and Talwalkar, A. 2018. Hyperband: A novel bandit-based approach to hyperparameter optimization. Journal of machine learning research, 18(185): 1--52

    work page 2018

  8. [11]

    Liu, F.; Huang, Y.; Luo, S.; Wang, Y.; Yang, Y.; Li, X.; Hu, Z.; Feng, J.; and Liu, Q. 2026 a . Cognitive alpha mining via llm-driven code-based evolution. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 11715--11749

    work page 2026

Show all 54 references
  1. [12]

    Liu, F.; Xialiang, T.; Yuan, M.; Lin, X.; Luo, F.; Wang, Z.; Lu, Z.; and Zhang, Q. 2024. Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model. In International Conference on Machine Learning, 32201--32223. PMLR

    2024

  2. [13]

    Z.; Du, A.; Keutzer, K.; Cheung, A.; Dimakis, A

    Liu, S.; Agarwal, S.; Maheswaran, M.; Cemri, M.; Li, Z.; Mang, Q.; Naren, A.; Boneh, E.; Cheng, A.; Pan, M. Z.; Du, A.; Keutzer, K.; Cheung, A.; Dimakis, A. G.; Sen, K.; Zaharia, M.; and Stoica, I. 2026 b . EvoX: Meta-Evolution for Automated Discovery. arXiv:2602.23413

    2026

  3. [14]

    G.; and Stoica, I

    Liu, S.; Cemri, M.; Agarwal, S.; Krentsel, A.; Naren, A.; Mang, Q.; Li, Z.; Gupta, A.; Maheswaran, M.; Cheng, A.; Pan, M.; Boneh, E.; Ramchandran, K.; Sen, K.; Zaharia, M.; Dimakis, A. G.; and Stoica, I. 2026 c . SkyDiscover: A Flexible, Adaptive Framework for AI-Driven Scient...

    2026

  4. [18]

    L.; Wolsey, L

    Nemhauser, G. L.; Wolsey, L. A.; and Fisher, M. L. 1978. An analysis of approximations for maximizing submodular set functions—I. Mathematical programming, 14(1): 265--294

    1978

  5. [19]

    Z.; Shirobokov, S.; Kozlovskii, B.; Ruiz, F

    Novikov, A.; Vũ, N.; Eisenberger, M.; Dupont, E.; Huang, P.-S.; Wagner, A. Z.; Shirobokov, S.; Kozlovskii, B.; Ruiz, F. J. R.; Mehrabian, A.; Kumar, M. P.; See, A.; Chaudhuri, S.; Holland, G.; Davies, A.; Nowozin, S.; Kohli, P.; and Balog, M. 2025. AlphaEvolve: A coding agent ...

    2025 arXiv

  6. [20]

    E.; Kadous, M

    Ong, I.; Almahairi, A.; Wu, V.; Chiang, W.-L.; Wu, T.; Gonzalez, J. E.; Kadous, M. W.; and Stoica, I. 2025. Route LLM : Learning to Route LLM s from Preference Data. In The Thirteenth International Conference on Learning Representations

    2025

  7. [21]

    OpenRouter . 2026 a . Qwen: Qwen3.5-Flash . https://openrouter.ai/qwen/qwen3.5-flash-20260224

    2026

  8. [22]

    OpenRouter . 2026 b . Qwen: Qwen3.5-Plus 2026-02-15 . https://openrouter.ai/qwen/qwen3.5-plus-20260216

    2026

  9. [23]

    K.; Soros, L

    Pugh, J. K.; Soros, L. B.; and Stanley, K. O. 2016. Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI, 3: 40

    2016

  10. [25]

    P.; Liu, Z.; and Barsoum, E

    Ray, P.; Brahma, P. P.; Liu, Z.; and Barsoum, E. 2026. AdaptEvolve: Improving Efficiency of Evolutionary AI Agents through Adaptive Model Selection. In Findings of the Association for Computational Linguistics: ACL 2026, 40625--40633

    2026

  11. [26]

    P.; Dupont, E.; Ruiz, F

    Romera - Paredes, B.; Barekatain, M.; Novikov, A.; Balog, M.; Kumar, M. P.; Dupont, E.; Ruiz, F. J. R.; Ellenberg, J. S.; Wang, P.; Fawzi, O.; Kohli, P.; and Fawzi, A. 2024. Mathematical discoveries from program search with large language models. Nature, 625(7995): 468--475

    2024

  12. [29]

    Ye, H.; Wang, J.; Cao, Z.; Berto, F.; Hua, C.; Kim, H.; Park, J.; and Song, G. 2024. Reevo: Large language models as hyper-heuristics with reflective evolution. Advances in neural information processing systems, 37: 43571--43608

    2024

  13. [30]

    Mathematical programming , volume=

    An analysis of approximations for maximizing submodular set functions—I , author=. Mathematical programming , volume=. 1978 , publisher=

    1978

  14. [31]

    International Conference on Learning Representations , volume=

    Aflow: Automating agentic workflow generation , author=. International Conference on Learning Representations , volume=

  15. [32]

    and Stoica, Ion , title =

    Liu, Shu and Cemri, Mert and Agarwal, Shubham and Krentsel, Alexander and Naren, Ashwin and Mang, Qiuyang and Li, Zhifei and Gupta, Akshat and Maheswaran, Monishwaran and Cheng, Audrey and Pan, Melissa and Boneh, Ethan and Ramchandran, Kannan and Sen, Koushik and Zaharia, Mate...

    2026

  16. [33]

    Nature , volume=

    Mathematical discoveries from program search with large language models , author =. Nature , volume=. 2024 , publisher=

    2024

  17. [34]

    arXiv preprint arXiv:2305.05176 , year=

    Frugalgpt: How to use large language models while reducing cost and improving performance , author=. arXiv preprint arXiv:2305.05176 , year=

    arXiv

  18. [35]

    Gonzalez and M Waleed Kadous and Ion Stoica , booktitle=

    Isaac Ong and Amjad Almahairi and Vincent Wu and Wei-Lin Chiang and Tianhao Wu and Joseph E. Gonzalez and M Waleed Kadous and Ion Stoica , booktitle=. Route. 2025 , url=

    2025

  19. [36]

    Findings of the Association for Computational Linguistics: ACL 2026 , pages=

    AdaptEvolve: Improving Efficiency of Evolutionary AI Agents through Adaptive Model Selection , author=. Findings of the Association for Computational Linguistics: ACL 2026 , pages=

    2026

  20. [37]

    arXiv preprint arXiv:2605.09764 , year=

    LEVI: Stronger Search Architectures Can Substitute for Larger LLMs in Evolutionary Search , author=. arXiv preprint arXiv:2605.09764 , year=

    arXiv

  21. [38]

    Journal of machine learning research , volume=

    Hyperband: A novel bandit-based approach to hyperparameter optimization , author=. Journal of machine learning research , volume=

  22. [39]

    2025 , eprint=

    AlphaEvolve: A coding agent for scientific and algorithmic discovery , author=. 2025 , eprint=

    2025

  23. [40]

    International Conference on Learning Representations , volume=

    Connecting large language models with evolutionary algorithms yields powerful prompt optimizers , author=. International Conference on Learning Representations , volume=

  24. [41]

    arXiv preprint arXiv:2510.14150 , year=

    Codeevolve: An open source evolutionary coding agent for algorithm discovery and optimization , author=. arXiv preprint arXiv:2510.14150 , year=

    arXiv

  25. [42]

    The Fourteenth International Conference on Learning Representations , year=

    ShinkaEvolve: Towards Open-Ended and Sample-Efficient Program Evolution , author=. The Fourteenth International Conference on Learning Representations , year=

  26. [43]

    2026 , eprint=

    AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization , author=. 2026 , eprint=

    2026

  27. [44]

    2026 , eprint=

    EvoX: Meta-Evolution for Automated Discovery , author=. 2026 , eprint=

    2026

  28. [45]

    2026 , url=

    Lakshya A Agrawal and Shangyin Tan and Dilara Soylu and Noah Ziems and Rishi Khare and Krista Opsahl-Ong and Arnav Singhvi and Herumb Shandilya and Michael J Ryan and Meng Jiang and Christopher Potts and Koushik Sen and Alex Dimakis and Ion Stoica and Dan Klein and Matei Zahar...

    2026

  29. [46]

    International Conference on Machine Learning , pages=

    Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model , author=. International Conference on Machine Learning , pages=. 2024 , organization=

    2024

  30. [47]

    Advances in neural information processing systems , volume=

    Reevo: Large language models as hyper-heuristics with reflective evolution , author=. Advances in neural information processing systems , volume=

  31. [48]

    Machine learning , volume=

    Finite-time analysis of the multiarmed bandit problem , author=. Machine learning , volume=. 2002 , publisher=

    2002

  32. [49]

    International conference on algorithmic learning theory , pages=

    On upper-confidence bound policies for switching bandit problems , author=. International conference on algorithmic learning theory , pages=. 2011 , organization=

    2011

  33. [50]

    Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining , pages=

    Streaming submodular maximization: Massive data summarization on the fly , author=. Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining , pages=

  34. [51]

    arXiv preprint arXiv:1504.04909 , year=

    Illuminating search spaces by mapping elites , author=. arXiv preprint arXiv:1504.04909 , year=

    arXiv

  35. [52]

    2026 , howpublished =

    2026

  36. [53]

    Frontiers in Robotics and AI , volume=

    Quality diversity: A new frontier for evolutionary computation , author=. Frontiers in Robotics and AI , volume=. 2016 , publisher=

    2016

  37. [54]

    Advances in neural information processing systems , volume=

    Stochastic multi-armed-bandit problem with non-stationary rewards , author=. Advances in neural information processing systems , volume=

  38. [55]

    Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Cognitive alpha mining via llm-driven code-based evolution , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  39. [56]

    arXiv preprint arXiv:2605.01120 , year=

    New Bounds for Zarankiewicz Numbers via Reinforced LLM Evolutionary Search , author=. arXiv preprint arXiv:2605.01120 , year=

    arXiv

  40. [57]

    The Twelfth International Conference on Learning Representations , year=

    Hybrid llm: Cost-efficient and quality-aware query routing , author=. The Twelfth International Conference on Learning Representations , year=

  41. [58]

    arXiv preprint arXiv:2601.10657 , year=

    Pacevolve: Enabling long-horizon progress-aware consistent evolution , author=. arXiv preprint arXiv:2601.10657 , year=

  42. [59]

    arXiv preprint arXiv:2604.01658 , year=

    Coral: Towards autonomous multi-agent evolution for open-ended discovery , author=. arXiv preprint arXiv:2604.01658 , year=

    arXiv

  43. [60]

    2026 , url =

    Qwen3.5: Towards Native Multimodal Agents , author =. 2026 , url =

    2026

  44. [61]

    arXiv preprint arXiv:2604.24372 , year=

    SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolution , author=. arXiv preprint arXiv:2604.24372 , year=

    arXiv

  45. [62]

    arXiv preprint arXiv:2508.10925 , year=

    gpt-oss-120b & gpt-oss-20b model card , author=. arXiv preprint arXiv:2508.10925 , year=

    arXiv

  46. [63]

    arXiv preprint arXiv:2607.03935 , year=

    Harness-Aware Self-Evolving: Co-Evolving Model Weights, Harness, and Task Solutions , author=. arXiv preprint arXiv:2607.03935 , year=

    arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.