LLaMEA-BO: A Large Language Model Evolutionary Algorithm for Automatically Generating Bayesian Optimization Algorithms

Elena Raponi; Niki van Stein; Thomas B\"ack; Wenhu Li

REVIEW 4 major objections 5 minor 2 cited by

LLaMEA-BO: A Large Language Model Evolutionary Algorithm for Automatically Generating Bayesian Optimization Algorithms

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper reports that an LLM running a population-based evolution strategy over code prompts can write complete, working Python Bayesian-optimization algorithms that, with no fine-tuning, outperform established BO baselines on 19 of 24…

desk verdict A real framework with an overstated headline: the 19/24 count is not a clean out-of-sample result, so the claims need recalibration before the paper is cited as evidence. read the letter →

arxiv 2505.21034 v1 pith:3LBTLSBV submitted 2025-05-27 cs.LG cs.NE

Wenhu Li , Niki van Stein , Thomas Bäck , Elena Raponi This is my paper

classification cs.LGcs.NE

keywords largelanguagemodelsBayesianoptimizationevolutionstrategyprompt-basedcrossoverautomatedalgorithmdesignBBOBbenchmarkhyperparametertuningtrust-regionsearch

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that designing Bayesian-optimization (BO) algorithms does not have to be a manual, expert-only activity. It presents LLaMEA-BO, a generate–evaluate–improve loop in which a large language model writes full Python implementations of BO pipelines—initial design, surrogate model, acquisition function, and outer loop—and the resulting code is compiled, scored on a small set of continuous benchmark functions, and then refined through prompt-based mutation and crossover. The headline quantitative claim is that the evolved algorithms beat state-of-the-art BO baselines on 19 of the 24 BBOB functions in dimension 5 and keep their edge when moved to higher dimensions and to machine-learning hyperparameter-tuning tasks. If true, this makes LLM-guided evolution a practical route to new BO algorithms rather than a demonstration of code generation.

What carries the argument

The mechanism is a population-based evolution strategy over LLM prompts. Each candidate is a Python class implementing a __call__(self, f, budget) interface with four mandated components—initial sample design, surrogate fitting, acquisition function, and next-point selection—so the LLM's freedom is confined to algorithmic choices rather than boilerplate. Fitness is the Area Over the Convergence Curve averaged over ten BBOB functions in five dimensions with 100 evaluations; crossover asks the LLM to combine two parents' code and then refine the result, mutation asks it to improve one parent, and either elitist (μ+λ) or comma (μ,λ) selection keeps the best programs, breaking ties toward shorter code.

What would settle it

Run LLaMEA-BO's identical search but compute the in-loop fitness on the other fourteen BBOB functions, or on all twenty-four, instead of the chosen ten; if the best evolved algorithms then no longer beat the paper's baseline optimizers on most functions, the performance claim is an artifact of the fitness proxy. An even sharper check is to take the five named generated algorithms and evaluate them on held-out continuous optimization problems that share no function family with BBOB, counting how many still outperform the same baselines.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a closed generate–evaluate–improve cycle, with no extra model training and only a generic BO skeleton in the prompt, produces complete Python implementations of Bayesian-optimization algorithms that are competitive or better than established BO methods. The winning programs include ATRBO and TREvol, both trust-region variants: ATRBO finishes first in aggregate convergence score at dimensions 5 and 10, while TREvol leads at dimensions 20 and 40. The authors attribute this to the combination of a BO-specific code template that fixes the pipeline skeleton, population-based crossover and mutation carried out through two-parent and one-parent prompts, and selection pressure from a small but varied in-loop benchmark.

Load-bearing premise

The load-bearing premise is that a score averaged over ten chosen benchmark functions in five dimensions with only 100 evaluations is a faithful measure of what makes a Bayesian-optimization algorithm good, so that algorithms selected to maximize it will still beat established baselines on the full benchmark, in higher dimensions, and on real tuning tasks.

Editorial extensions

If this is right

Without any fine-tuning, an LLM can serve as an algorithmic co-designer, shifting the bottleneck in BO design from expert knowledge to the search loop and the fitness signal.
The best evolved algorithms are trust-region hybrids, so the automatic search independently rediscovers a design pattern that human specialists had already found valuable.
The generated code is short, plain Python, and benchmark-agnostic, so non-specialists can inspect, adapt, and deploy it in domains where expert tuning time is scarce.
Because the loop is benchmark-agnostic, rerunning it on mixed-integer, constrained, noisy, or multi-objective benchmarks requires only minor modifications, as the paper states in its outlook.
A single 100-generation search costs roughly 20 CPU-hours and 2–20 hours of wall-clock time, making the approach reproducible on modest computing resources.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The strongest unstated risk is selection on a proxy: the in-loop score uses only ten BBOB functions at one dimension with 100 evaluations, so the evolved algorithms are optimized for that slice; their success on the other fourteen functions is evidence against overfitting, but it does not reveal how the search would behave if the proxy changed.
The same loop with a different fitness signal—noisy evaluations, constrained spaces, mixed-integer variables, or a portfolio of LLM backends—would likely evolve a different family of algorithms; the paper itself notes that the trust-region dominance reflects the template and short budget.
A cheap, direct test of the method's generality would be to rerun the search on a disjoint set of benchmark functions and check whether the winning algorithms still beat the same baselines; the current design makes that experiment natural because the loop is benchmark-agnostic.
The comparison on tree-based hyperparameter tasks hints that when an LLM can memorize good defaults, code evolution may matter less than in-context knowledge; the paper flags this as memorization, but the broader implication is that the fitness proxy and task distribution shape which capability the search exploits.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Real talk: this paper has a genuine contribution, but the headline number is not what it appears. LLaMEA-BO is, as far as I can tell, the first LLM-driven framework that generates complete, runnable Bayesian optimization algorithms in code, not just acquisition functions or components. The population-based ES with crossover/mutation, the BO-specific template, and the open-sourced code and raw logs make this a useful artifact. ATRBO, TREvol, etc. are concrete, testable algorithms. The ablation studies and the explicit limitations section (trust-region bias, single LLM, reproducibility caveats) tell me the authors know their own work's edges. That is real and worth engaging.

But the central claim—"outperform SOTA baselines in 19 out of 24 BBOB functions in dimension 5"—does not survive close reading. The in-loop fitness is AOCC on exactly 10 BBOB functions, and those same functions are part of the 24-function final evaluation. So at least part of the reported superiority is selection on the test set, not independent prediction. Only 14 functions are truly held out. On top of that, Appendix D excludes f9 and f19 from the final evaluation because one generated algorithm "cheats" near the origin, but the abstract still says 24. That makes the actual denominator 22, and the decision to exclude was made after seeing results. And there is no per-function significance test anywhere; Table 1 gives aggregate t-tests only. So the 19/24 is a mix of training performance, post-hoc exclusion, and untested individual means. As stated, it's not a clean result.

What holds up? The higher-dimension results (10/20/40) are genuinely out-of-sample because the search only ran in d=5, and the Bayesmark tasks are separate. Those results are competitive but not dominant—LLAMBO beats the generated algorithms on several tasks. So the framework produces competitive algorithms, and there is evidence of transfer, but "state-of-the-art outperformance" is not established.

Who is this for? People working on LLM-based automated algorithm design and, secondarily, BO practitioners looking for new algorithmic building blocks. It deserves a serious referee, not a desk reject. My recommendation: send it to review, but insist the authors recalibrate the claims—report held-out counts, justify the f9/f19 exclusion, add per-function significance or effect sizes, and adjust the abstract. Fix those and this becomes a solid paper.

Referee Report

4 major / 5 minor

Summary. LLaMEA-BO is an evolutionary framework that uses an LLM-driven generate-evaluate-improve loop to produce complete Python implementations of Bayesian optimization (BO) algorithms. Candidate algorithms are scored in the loop by anytime area over the convergence curve (AOCC) on 10 selected BBOB functions in dimension 5 with a budget of 100 evaluations. The best generated algorithms are then benchmarked on all 24 BBOB functions in dimensions 5, 10, 20, and 40, on 25 Bayesmark hyperparameter-tuning tasks and 15 synthetic tasks, and compared against CMA-ES, HEBO, TuRBO1, Vanilla BO, and LLAMBO. The central claim is that the generated algorithms outperform state-of-the-art BO baselines on 19 of 24 BBOB functions in dimension 5 and generalize well to higher dimensions and other tasks.

Significance. If the central claim were fully supported, this would be a meaningful contribution to automated algorithm design: it would demonstrate that LLM-guided evolution can synthesize complete BO algorithms that beat established baselines, and the release of code, logs, and ablations would aid reproducibility. The framework is genuinely novel relative to prior work that uses LLMs for only one BO component. However, the quantitative headline is not cleanly supported by the evidence as presented: the in-loop fitness functions overlap with the final evaluation set, two functions are excluded from the final evaluation after observing results, and the '19/24' count is not backed by per-function significance testing. These issues are load-bearing for the paper's main claim.

major comments (4)

[Section 3.2 and Section 4.2] The in-loop fitness in Section 3.2 is AOCC on exactly 10 BBOB functions (ids 2, 4, 6, 8, 12, 14, 15, 18, 21, 23) in dimension 5, and the final evaluation in Section 4.2 is reported over all 24 BBOB functions. Because the same 10 functions are included in the 24-function evaluation, 10 of the 24 functions are not out-of-sample; the reported superiority on those functions is partly a selection artifact. The paper should report results separately for the 14 held-out functions, or re-run selection on a fully independent benchmark, before claiming a 19/24 out-of-sample result.
[Appendix D] The abstract and Section 4.2 say the generated algorithms are evaluated on the full set of 24 BBOB functions, and the abstract claims outperformance in 19 (out of 24) functions. Appendix D states that f9 and f19 are excluded from the final evaluation of the generated algorithms because a generated algorithm exploits their centered optima. This means the actual evaluation set is 22 functions, not 24, and the exclusion was decided after observing results. The main text must either include f9 and f19 in the evaluation or state the corrected denominator (e.g., 19/22) explicitly.
[Table 1 and Section 5] The claim that the generated algorithms 'outperform state-of-the-art BO baselines in 19 (out of 24) BBOB functions' is not supported by per-function significance testing. Table 1 reports aggregate AOCC values and a paired t-test (alpha = 0.05) only for the best generated algorithm versus the best BO baseline as a whole. A higher mean on an individual function may be within noise, so the count of 19/24 is not established. The authors should provide per-function effect sizes, confidence intervals, or paired tests (e.g., Wilcoxon or bootstrap) and report the number of functions with statistically significant improvements.
[Section 5 and Figure 2] The statement that the generated algorithms 'generalize well to higher dimensions' is based on aggregate AOCC values in Table 1 and violin plots in Figure 2. No significance tests are reported per dimension, and it is not clear whether the higher-dimensional generalization claim holds for all of the top-3 generated algorithms or only for selected ones (ATRBO in low dimensions, TREvol in high dimensions). Please add per-dimension significance tests and clarify which algorithms support the generalization claim.

minor comments (5)

[Section 1] The text contains a garbled symbol '/exclamati⌢n-circle' immediately before the first contribution bullet; it should be removed or replaced with a normal exclamation mark.
[References] The reference list contains two entries for Watanabe (2023a and 2023b) that appear to be the same work; the citation in Appendix A.2.3 uses 2023b while related work uses 2023a, and the duplicates should be merged.
[Table 1 caption] The caption says 'Average Area Over Convergence Curve over all 24 BBOB functions', but Appendix D excludes f9 and f19 from the final evaluation of the generated algorithms; the caption and the appendix should be made consistent.
[Section 4.2] The text says '40 machine learning tasks from the extended Bayesmark suite' without stating the composition; Appendix A.2.2 clarifies 25 public plus 15 synthetic tasks, but the main text should state this explicitly for clarity.
[Figures 1-3] The figure captions do not consistently state whether shaded areas represent standard error or standard deviation; please specify this in every convergence plot.

Circularity Check

1 steps flagged · score 6.0 of 10

The headline '19/24 BBOB functions' is partly an in-sample result: 10 of the 24 functions were used as the evolutionary fitness, so only 14 BBOB functions in d=5 are truly held out; the paper also excludes f9/f19 post hoc, changing the denominator.

fitted input called prediction [Section 3.2 (Evaluation); Section 4.2 (Validation Setup); Section 5 (Results)]
"we measure performance of all generated algorithms with the anytime Area Over the Convergence Curve (AOCC) on a subset of 10 of the 24 noiseless functions of COCO's BBOB suite ... function ids 2,4,6,8,12,14,15,18,21 and 23 ... To validate our proposed framework, we assess the extent to which the best generated algorithms generalize beyond the evaluations conducted during the optimization loop. Figure 2 presents the performance of the best BO algorithms generated by LLaMEA-BO in terms of average AOCC, evaluated on the full BBOB suite."

The in-loop fitness is AOCC on exactly 10 BBOB functions (ids 2,4,6,8,12,14,15,18,21,23) in d=5, and the final 'validation' on the full 24-function BBOB suite in d=5 includes those same 10 functions with the same AOCC measure. The abstract's count '19 (out of 24) BBOB functions in dimension 5' therefore includes 10 functions whose performance was directly optimized during the evolutionary search; those wins are a training artifact, not an independent prediction. Only the remaining 14 BBOB functions in d=5 are truly held out. The paper's framing that this evaluation tests 'generalize beyond the evaluations conducted during the optimization loop' conflates selection performance with out-of-sample generalization.

full rationale

The paper's central numerical claim—19/24 BBOB wins in d=5—is not a clean out-of-sample result because the evolutionary fitness is computed on 10 of those 24 functions (Section 3.2), and the final BBOB validation includes those same functions (Section 4.2/5). This is the fitted-input-called-prediction pattern: the algorithm is selected to maximize AOCC on those 10 landscapes, and the same landscapes are then counted as evidence of superiority. The genuinely out-of-sample portions of the evaluation are the 14 remaining BBOB functions in d=5, all results in d=10/20/40 (since evolution ran only in d=5), and the Bayesmark tasks. The paper also modifies the evaluation set post hoc in Appendix D, excluding f9 and f19 after observing 'TrustRegionAdaptiveTempBOv2' exploit center-biased optima; this makes the stated denominator of 24 inconsistent for the final evaluation. No load-bearing self-citation chain is present: the LLaMEA framework citation [Stein and Bäck, 2025] is prior work by two of the authors, but it is used as a starting point, not to forbid alternatives or to justify the headline result. The core derivation—evolutionary search over LLM-generated BO code—is not definitionally circular, but a headline numerical claim is contaminated by in-sample functions and a post-hoc exclusion, giving a partial circularity score of 6.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The paper does not introduce new physical entities or conserved quantities. Its free parameters are algorithm and framework hyperparameters, most chosen by hand or by ablation on the same benchmark used for evaluation. The key axioms are about benchmark representativeness, the validity of the BO template as an inductive bias, the legitimacy of excluding biased functions, and the transferability of results from a single LLM backend.

free parameters (6)

ATRBO trust-region initial radius r = 2.5
Hand-set in the generated ATRBO algorithm and validated by ablation (Appendix H.2). The algorithm's strong BBOB performance depends on this value.
ATRBO radius decay factor rho = 0.95
Hand-set in ATRBO and ablated; changing it degrades performance on most functions (Appendix H.2).
ATRBO LCB exploration parameter kappa = 2.0
Hand-set in ATRBO and ablated; the ablation shows sensitivity (Appendix H.2).
ATRBO initial sample count n_init = min(10d, B/5)
Chosen in the ATRBO pseudocode (Appendix H.1); affects exploration-exploitation balance.
LLaMEA-BO sampling temperature = 0.5
Selected via ablation on the BBOB subset (Appendix C.2); the framework's performance is somewhat robust but this choice affects diversity.
Crossover rate p_cr = 0.6 or 0.9
Ablated in Appendix C.1.2; the paper reports 0.6 or 0.9 as best, indicating hand-tuning on the benchmark.

assumptions (4)

domain assumption The 10 selected BBOB functions are representative of full BBOB and general BO problems.
Used as in-loop fitness in Section 3.2; if false, algorithms overfit the subset and the 19/24 claim is unsupported.
domain assumption The provided BO template with initial design, surrogate, acquisition, and outer loop is a valid and unbiased inductive bias.
Section 3.1 states the template is shipped to the LLM; this constrains the algorithm space to BO-like structures and may bias toward trust-region variants.
ad hoc to paper Excluding functions with optima near the center (f9, f19) is legitimate rather than post-hoc selection.
Appendix D introduces the exclusion after observing an algorithm exploiting the bias; this can inflate reported generalization.
domain assumption Results from a single LLM backend (gemini-2.0-flash) generalize to other LLMs.
Section 4 and the Limitations state only one backend was used; the framework's performance may depend on the specific model.

how reviews work

0 comments

Cite this review

Pith. "Pith review of LLaMEA-BO: A Large Language Model Evolutionary Algorithm for Automatically Generating Bayesian Optimization Algorithms." pith.science (2026). https://pith.science/paper/3LBTLSBV

@misc{pith2026250521034,
  author       = {Pith},
  title        = {Pith review of: LLaMEA-BO: A Large Language Model Evolutionary Algorithm for Automatically Generating Bayesian Optimization Algorithms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3LBTLSBV}},
  note         = {Machine review of arXiv:2505.21034}
}

read the original abstract

Bayesian optimization (BO) is a powerful class of algorithms for optimizing expensive black-box functions, but designing effective BO algorithms remains a manual, expertise-driven task. Recent advancements in Large Language Models (LLMs) have opened new avenues for automating scientific discovery, including the automatic design of optimization algorithms. While prior work has used LLMs within optimization loops or to generate non-BO algorithms, we tackle a new challenge: Using LLMs to automatically generate full BO algorithm code. Our framework uses an evolution strategy to guide an LLM in generating Python code that preserves the key components of BO algorithms: An initial design, a surrogate model, and an acquisition function. The LLM is prompted to produce multiple candidate algorithms, which are evaluated on the established Black-Box Optimization Benchmarking (BBOB) test suite from the COmparing Continuous Optimizers (COCO) platform. Based on their performance, top candidates are selected, combined, and mutated via controlled prompt variations, enabling iterative refinement. Despite no additional fine-tuning, the LLM-generated algorithms outperform state-of-the-art BO baselines in 19 (out of 24) BBOB functions in dimension 5 and generalize well to higher dimensions, and different tasks (from the Bayesmark framework). This work demonstrates that LLMs can serve as algorithmic co-designers, offering a new paradigm for automating BO development and accelerating the discovery of novel algorithmic combinations. The source code is provided at https://github.com/Ewendawi/LLaMEA-BO.

Figures

Figures reproduced from arXiv: 2505.21034 by the authors.

**Figure 2.** Best algorithm evaluation based on AOCC: Violin plots aggregating over 24 functions, 3 [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Hyperparameter tuning task results. Performance is reported in terms of regret and [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Figures from the paper (22 more)

**Figure 4.** Figure 4: Results from different population sizes and elitism configurations averaged over [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]

**Figure 5.** Figure 5: Results from different crossover rate [0.3, 0.6, 0.9] configurations (using a (4 + 8) ES strategy) averaged over 10 BBOB functions and 4 repetitions per function. C.2 LLM Configuration Figures 6, 7, 8 reveal that sampling temperature 0.5, top-k = 40 and top-p = 0.7 for…

**Figure 6.** Figure 6: Results from different LLM temperature [0.5, 1.0, 1.5] configurations (using a (4 + 8) ES strategy) averaged over 10 BBOB functions and 3 repetitions per function. C.2.2 Top-K 4+8_20 4+8_40 4+8_80 0.165 0.170 0.175 0.180 0.185 0.190 0.195 0.200 AOCC (a) Final AOCC dist…

**Figure 7.** Figure 7: Results from different LLM top-K [20, 40, 80] configurations (using a (4 + 8) ES strategy) averaged over 10 BBOB functions and 3 repetitions per function. C.2.3 Top-P 4+8_0.5 4+8_0.7 4+8_0.95 0.170 0.175 0.180 0.185 0.190 AOCC (a) Final AOCC distributions. 0 5 10 15 20…

**Figure 8.** Figure 8: Results from different LLM top-P [0.5, 0.7, 0.95] configurations (using a (4 + 8) ES strategy) averaged over 10 BBOB functions and 3 repetitions per function. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]

**Figure 9.** Figure 9: Level sets for BBOB function f9 (instance id = 1) in d = 5, showing the optimum near the center of the search domain. Figure from https://coco-platform.org/testsuites/bbob/ functions/f09.html. Figures 9 illustrates how the optimum of f9 lies near the search-space origi…

**Figure 10.** Figure 10: Observed strange behaviour of “TrustRegionAdaptiveTempBOv2” on [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]

**Figure 11.** Figure 11: Loss convergence curves on all BBOB functions in [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]

**Figure 12.** Figure 12: Loss convergence curves on all BBOB functions in [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]

**Figure 13.** Figure 13: Loss convergence curves on all BBOB functions in [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]

**Figure 14.** Figure 14: Loss convergence curves on all BBOB functions in [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]

**Figure 15.** Figure 15: Regret curves on all datasets using Adaboost. [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]

**Figure 16.** Figure 16: Regret curves on all datasets using Regression Trees. [PITH_FULL_IMAGE:figures/full_fig_p022_16.png]

**Figure 17.** Figure 17: Regret curves on all datasets using MLP SGD. [PITH_FULL_IMAGE:figures/full_fig_p023_17.png]

**Figure 18.** Figure 18: Regret curves on all datasets using Random Forest. [PITH_FULL_IMAGE:figures/full_fig_p023_18.png]

**Figure 19.** Figure 19: Regret curves on all datasets using SVM. [PITH_FULL_IMAGE:figures/full_fig_p023_19.png]

**Figure 20.** Figure 20: Average CPU time per algorithm on the Random Forest task averaged over 3 datasets [PITH_FULL_IMAGE:figures/full_fig_p024_20.png]

**Figure 21.** Figure 21: ATRBO ρ settings [0.65, 0.80, 0.95]. The baseline has ρ = 0.95. The convergence curves are averaged over 5 runs in d = 5. 0 10 0 10 1 F1 10 2 10 3 10 4 10 5 10 6 10 7 F2 10 2 F3 10 2 F4 0 10 0 10 1 10 2 F5 10 1 10 2 10 3 10 4 10 5 10 6 F6 10 1 10 2 F7 10 1 10 2 10 3 1…

**Figure 22.** Figure 22: ATRBO κ settings [1.0, 2.0, 4.0]. The baseline has κ = 2.0. The convergence curves are averaged over 5 runs in d = 5. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_22.png]

**Figure 23.** Figure 23: ATRBO radius settings [1.0, 2.5, 5.0]. The baseline has radius= 2.5. The convergence curves are averaged over 5 runs in d = 5. 0 10 0 10 1 F1 10 2 10 3 10 4 10 5 10 6 10 7 F2 10 2 F3 10 2 F4 0 10 0 10 1 10 2 F5 10 1 10 2 10 3 10 4 10 5 10 6 F6 10 0 10 1 10 2 F7 10 1 1…

**Figure 24.** Figure 24: Adaptation modifications, baseline (adaptive [PITH_FULL_IMAGE:figures/full_fig_p027_24.png]

**Figure 25.** Figure 25: Loss, κ and trust region radius over 5 independent runs with ρ = 0.85 and ρ = 0.95 on f23 in 20 dimensions. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_25.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

LLM-Driven Evolutionary Generation of Multi-Objective Bayesian Optimization Algorithms
cs.NE 2026-07 conditional novelty 6.0 of 10

LLM-driven evolution with SMAC HPO produces MOBO algorithms that match or exceed qParEGO accuracy at roughly 3–60× lower wall-clock cost on synthetic and real-world problems.
A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving
cs.NE 2025-09 conditional novelty 4.0 of 10

A literature survey that classifies LLM-based optimization research into modeling and solving, with solving divided into LLMs as optimizers, low-level components, and high-level managers.

Reference graph

Works this paper leans on

24 extracted references · 9 canonical work pages · cited by 2 Pith papers

[1]

Figure from https://coco-platform.org/testsuites/bbob/ functions/f09.html

in d= 5, showing the optimum near the center of the search domain. Figure from https://coco-platform.org/testsuites/bbob/ functions/f09.html. Figures 9 illustrates how the optimum of f9 lies near the search-space origin, the same is known for f19 Long et al. [2023]. These biased optima locations allows one generated algorithm to basically “cheat” by sampl...

work page 2023
[6]

URLhttps://arxiv.org/abs/2401.02051. M. Feurer and F. Hutter. Hyperparameter Optimization. In F. Hutter, L. Kotthoff, and J. Vanschoren, editors, Automated Machine Learning: Methods, Systems, Challenges, pages 3–33. Springer International Publishing, Cham,

arXiv
[7]

doi: 10.1007/978-3-030-05318-5_1

ISBN 978-3-030-05318-5. doi: 10.1007/978-3-030-05318-5_1. A. I. J. Forrester, A. Sóbester, and A. J. Keane.Engineering Design via Surrogate Modelling - A Practical Guide. John Wiley & Sons Ltd.,

doi:10.1007/978-3-030-05318-5_1
[10]

TrustRegionAdaptiveTempBOv2

Here we have one algorithm called “TrustRegionAdaptiveTempBOv2" with very good performance already from the very first evaluation. This bias in sampling strategies does not generalize normally and is not preferred. We therefore exclude those biased instances from the evaluation procedure of the final generated BO algorithms. Note that we did not have thes...

work page
[11]

doi: https://doi.org/10.1080/10556788.2020.1808977. J. M. Hernández-Lobato, M. A. Gelbart, R. P. Adams, M. W. Hoffman, and Z. Ghahramani. A General Framework for Constrained Bayesian Optimization using Information-based Search, Sept

arXiv 2020
[13]

doi: 10.1007/978-981-15-0731-1_3

ISBN 9789811507311. doi: 10.1007/978-981-15-0731-1_3. M. Lindauer, K. Eggensperger, M. Feurer, A. Biedenkapp, D. Deng, C. Benjamins, T. Ruhkopf, R. Sass, and F. Hutter. SMAC3: A versatile Bayesian optimization package for hyperparameter optimization.J. Mach. Learn. Res., 23(1):54:2475–54:2483, Jan

work page doi:10.1007/978-981-15-0731-1_3
[15]

F. Liu, Y . Yao, P. Guo, Z. Yang, Z. Zhao, X. Lin, X. Tong, M. Yuan, Z. Lu, Z. Wang, and Q. Zhang. A systematic survey on large language models for algorithm design, 2024a. URL https://arxiv.org/abs/ 2410.14716. T. Liu. Tennisonliu/LLAMBO, May

arXiv
[16]

Pluhacek, J

M. Pluhacek, J. Kovac, A. Viktorin, P. Janku, T. Kadavy, and R. Senkerik. Using llm for automatic evolvement of metaheuristics from swarm algorithm soma. InProceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO ’24 Companion, page 2018–2022, New York, NY , USA,

work page 2018

Show all 24 references

[17]

ISBN 9798400704956

Association for Computing Machinery. ISBN 9798400704956. doi: 10.1145/3638530.3664181. URL https://doi. org/10.1145/3638530.3664181. M. C. Ramos, S. S. Michtavy, M. D. Porosoff, and A. D. White. Bayesian Optimization of Catalysts With In-context Learning, Apr
[19]

URL https: //doi.org/10.1145/3731567

doi: 10.1145/3731567. URL https: //doi.org/10.1145/3731567. Just Accepted. H. Wang, B. van Stein, M. Emmerich, and T. Back. A new acquisition function for Bayesian optimization based on the moment-generating function. In2017 IEEE International Conference on Systems, Man, and C...

doi
[21]

doi: 10.1145/3676536.3676816. R. Zhang, F. Liu, X. Lin, Z. Wang, Z. Lu, and Q. Zhang. Understanding the importance of evolutionary search in automated heuristic design with large language models. InInternational Conference on Parallel Problem Solving from Nature, pages 185–202...
[22]

13 A Further Experimental Details This section documents additional details on the evaluation pipeline: Subsection A.1 formalises theAOCCscore and its aggregation, while Subsection A.3 describes the sandbox that prevents untrusted code from hanging or leaking resources. A.1 AO...

2014
[2001]

doi: 10.1023/A: 1010933404324

ISSN 1573-0565. doi: 10.1023/A: 1010933404324. A. I. Cowen-Rivers, W. Lyu, R. Tutunov, Z. Wang, A. Grosnit, R. R. Griffiths, A. M. Maraval, H. Jianye, J. Wang, J. Peters, and H. B. Ammar. HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisation, May

doi
[2006]

doi: 10.1007/s10994-006-6226-1

ISSN 1573-0565. doi: 10.1007/s10994-006-6226-1. J. González, Z. Dai, P. Hennig, and N. D. Lawrence. Batch Bayesian Optimization via Local Penalization, Oct

doi
[2009]

doi: 10.1287/ijoc.1080.0314

ISSN 1091-9856, 1526-5528. doi: 10.1287/ijoc.1080.0314. P. I. Frazier. A Tutorial on Bayesian Optimization, July
[2016]

Hvarfner, D

C. Hvarfner, D. Stoll, A. Souza, M. Lindauer, F. Hutter, and L. Nardi. πBO: AUGMENTING ACQUISITION FUNCTIONS WITH USER BELIEFS FOR BAYESIAN OPTIMIZATION.arXiv preprint arXiv:2204.11051,

arXiv
[2017]

doi: 10.1109/SMC.2017.8122656. S. Watanabe. Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance, May 2023a. S. Watanabe. Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Rol...

2017
[2019]

Anonymized Repository - Anonymous GitHub

Anonymous. Anonymized Repository - Anonymous GitHub. https://anonymous.4open.science/r/LLaMEA-BO- F008/README.md, 2025a. Anonymous. Logs and raw results of llamea-bo, May 2025b. URL https://doi.org/10.5281/zenodo. 15384610. M. Balandat, B. Karrer, D. R. Jiang, S. Daulton, B. L...

doi
[2020]

URLhttp://arxiv.org/abs/1910.06403. L. Breiman. Random Forests.Machine Learning, 45(1):5–32, Oct

1910 arXiv
[2021]

doi: 10.1145/3425501

ISSN 2688-299X, 2688-3007. doi: 10.1145/3425501. J. de Nobel, F. Ye, D. Vermetten, H. Wang, C. Doerr, and T. Bäck. Iohexperimenter: Benchmarking platform for iterative optimization heuristics.Evol. Comput., 32(3):205–210,

doi
[2022]

ISSN 1532-4435. F. Liu, X. Tong, M. Yuan, and Q. Zhang. Algorithm evolution using large language model. arXiv:2311.15249,

arXiv
[2023]

Hansen, S

11 N. Hansen, S. Finck, R. Ros, and A. Auger. Real-parameter black-box optimization benchmarking 2009: Noiseless functions definitions. Technical Report RR6829, INRIA,

2009
[2024]

URL https://doi.org/10.1162/evco_a_00342

doi: 10.1162/EVCO\_A\_00342. URL https://doi.org/10.1162/evco_a_00342. D. Eriksson and M. Jankowiak. High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspaces. arXiv:2103.00349 [cs, stat], June

arXiv
[2025]

2024.3497793

doi: 10.1109/TEVC. 2024.3497793. R. Turner, D. Eriksson, M. McCourt, J. Kiili, E. Laaksonen, Z. Xu, and I. Guyon. Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020, Aug

2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Figure from https://coco-platform.org/testsuites/bbob/ functions/f09.html

in d= 5, showing the optimum near the center of the search domain. Figure from https://coco-platform.org/testsuites/bbob/ functions/f09.html. Figures 9 illustrates how the optimum of f9 lies near the search-space origin, the same is known for f19 Long et al. [2023]. These biased optima locations allows one generated algorithm to basically “cheat” by sampl...

work page 2023

[2] [6]

URLhttps://arxiv.org/abs/2401.02051. M. Feurer and F. Hutter. Hyperparameter Optimization. In F. Hutter, L. Kotthoff, and J. Vanschoren, editors, Automated Machine Learning: Methods, Systems, Challenges, pages 3–33. Springer International Publishing, Cham,

arXiv

[3] [7]

doi: 10.1007/978-3-030-05318-5_1

ISBN 978-3-030-05318-5. doi: 10.1007/978-3-030-05318-5_1. A. I. J. Forrester, A. Sóbester, and A. J. Keane.Engineering Design via Surrogate Modelling - A Practical Guide. John Wiley & Sons Ltd.,

doi:10.1007/978-3-030-05318-5_1

[4] [10]

TrustRegionAdaptiveTempBOv2

Here we have one algorithm called “TrustRegionAdaptiveTempBOv2" with very good performance already from the very first evaluation. This bias in sampling strategies does not generalize normally and is not preferred. We therefore exclude those biased instances from the evaluation procedure of the final generated BO algorithms. Note that we did not have thes...

work page

[5] [11]

doi: https://doi.org/10.1080/10556788.2020.1808977. J. M. Hernández-Lobato, M. A. Gelbart, R. P. Adams, M. W. Hoffman, and Z. Ghahramani. A General Framework for Constrained Bayesian Optimization using Information-based Search, Sept

arXiv 2020

[6] [13]

doi: 10.1007/978-981-15-0731-1_3

ISBN 9789811507311. doi: 10.1007/978-981-15-0731-1_3. M. Lindauer, K. Eggensperger, M. Feurer, A. Biedenkapp, D. Deng, C. Benjamins, T. Ruhkopf, R. Sass, and F. Hutter. SMAC3: A versatile Bayesian optimization package for hyperparameter optimization.J. Mach. Learn. Res., 23(1):54:2475–54:2483, Jan

work page doi:10.1007/978-981-15-0731-1_3

[7] [15]

F. Liu, Y . Yao, P. Guo, Z. Yang, Z. Zhao, X. Lin, X. Tong, M. Yuan, Z. Lu, Z. Wang, and Q. Zhang. A systematic survey on large language models for algorithm design, 2024a. URL https://arxiv.org/abs/ 2410.14716. T. Liu. Tennisonliu/LLAMBO, May

arXiv

[8] [16]

Pluhacek, J

M. Pluhacek, J. Kovac, A. Viktorin, P. Janku, T. Kadavy, and R. Senkerik. Using llm for automatic evolvement of metaheuristics from swarm algorithm soma. InProceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO ’24 Companion, page 2018–2022, New York, NY , USA,

work page 2018