Pith. sign in

REVIEW 2 major objections 7 minor 1 cited by

Providing strong algorithmic code examples substantially improves LLM-driven optimization, and the effect is driven by the code itself, not by linguistic instructions.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Prompting LLMs with strong benchmark algorithm code, rather than relying on linguistic instructions, improves LLM-driven black-box optimization; the proposed BAG method outperforms five baselines on pbo and bbob.

T0 review reviewed 2026-08-02 challenge →

load-bearing objection The paper's core insight about code examples as the strongest prompt signal is plausible and the BAG variant is a reasonable step, but the main comparison is currently confounded by per-problem benchmark-selected priors versus random-search baselines. the 2 major comments →

arxiv 2603.02792 v2 pith:E36JSH7B submitted 2026-03-03 cs.LG cs.NE

From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priors

Qi Huang , Furong Ye , Ananta Shahane , Thomas B\"ack , Niki van Stein This is my paper
classification cs.LG cs.NE
keywords LLM-driven optimizationblack-box optimizationprompt attributioncode priorsbenchmark-guided searchevolutionary algorithm designpseudo-Boolean optimizationcontinuous black-box optimization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that in LLM-driven algorithm design, the example code embedded in a prompt is the dominant influence on the algorithms the LLM produces. Token-level attribution analysis shows that code description and strategy instructions far outweigh task description and fitness feedback. Building on this, the authors introduce a benchmark-assisted guided evolutionary approach (BAG) that injects strong benchmark algorithm codes as priors during initialization and search. BAG consistently outperforms five competing LLM-driven optimization methods across two standard black-box optimization benchmark suites and three different LLMs. The takeaway is practical: integrating established benchmark knowledge into the prompt is a reliable way to make LLMs design better and more robust optimizers.

Core claim

The central claim is that high-quality algorithmic code examples, not careful linguistic phrasing, are what steer LLMs toward useful optimization algorithms. Using an attention-aware attribution method, the paper shows that the code-related parts of a prompt receive the highest relevance scores when the LLM generates new algorithm code. It then demonstrates that refining a strong prior code behaves as a neighborhood search in algorithm space, and that initializing and periodically re-injecting top benchmark algorithms guides the search into promising regions. The proposed BAG method, a (1+1) elitist evolutionary loop with benchmark code priors, achieves superior and more robust performance t

What carries the argument

The central mechanism is the benchmark-assisted guided evolutionary approach (BAG), a simple (1+1) elitist LLM-driven search that keeps an example code in every prompt. At initialization, BAG uses a strong benchmark algorithm for the problem; every q iterations it refines a randomly selected benchmark code; otherwise it refines the current best or generates a novel algorithm. The supporting analytical tool is an attention-aware feature attribution method that produces token-level relevance scores, showing that code-related prompt components receive the highest influence. BAG's design directly exploits this by making the code prior the primary steering mechanism, rather than relying on elabor

Load-bearing premise

The load-bearing assumption is that choosing the top five benchmark algorithms for each problem from a repository's performance data does not leak information about the test instances, so the benchmark-guided selection preserves a genuine black-box setting.

What would settle it

Run BAG with benchmark priors selected from a different set of problems or with the same code but shuffled problem labels, and compare the resulting AUC. If the performance advantage disappears or reverses, the reported gains stem from using the target problem's own benchmark data to pick the prior code, not from a general benefit of code priors.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Prompt design for LLM-driven optimization should focus on supplying high-quality reference code rather than on linguistic prompting strategies.
  • Existing benchmark algorithm collections can be repurposed as reusable priors for LLM-driven algorithm design, linking traditional benchmarking to automated design.
  • Search control in LLM-driven optimization can be achieved through the choice of code prior, because refinement of a strong prior behaves as a local neighborhood search.
  • Because BAG requires only a single LLM query per evaluated candidate, it may gain further advantage under equal query budgets compared to population-based methods.
  • BAG is a unified strategy that can be integrated into population-based LLM-driven approaches, potentially extending the benefit of benchmark priors beyond the (1+1) scheme tested here.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If code priors are the dominant factor, then a simpler baseline that uses benchmark code with random mutations may recover much of BAG's advantage, reframing the contribution as one of code reuse rather than of LLM reasoning.
  • The per-problem selection of top benchmark algorithms from a repository could leak problem-specific information; a fairer black-box test would choose priors from a disjoint set of problems or from generic algorithm families, and the paper's black-box claim hinges on this not being the case.
  • A direct testable extension: replacing BAG's benchmark priors with random but syntactically valid algorithm code of comparable length should erode its advantage if the prior's quality is what matters, while replacing the code with a different strong algorithm should retain it.
  • Attribution scores could be used not only to understand prompts but also to automatically select which code prior to inject at each step, moving beyond uniform random selection.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. This paper studies prompt design in LLM-driven optimization. Using AttnLRP token attribution on prompt-code pairs from two open-source LLMs, the authors find that code examples, rather than linguistic instructions, dominate the influence on generated algorithm code. Motivated by this, they propose BAG, a (1+1) elitist LLM-driven search that initializes from and periodically re-injects code from a benchmark set A_bench. BAG is evaluated on the pbo and bbob suites with three commercial LLMs and compared to EoH, LHNS, LLaMEA, MCTS-AHD, and ReEvo, reporting superior average normalized AUC. A CodeBLEU analysis is used to show the influence of injected benchmark code on later generations, and generalization on unseen instances is reported in the appendix.

Significance. If the claims hold, the paper provides a practically valuable empirical insight: strong code priors, not task descriptions, are the main steering signal in LLM-driven BBO, and benchmark repositories can be exploited as a source of such priors. The token-attribution analysis is a novel application of AttnLRP to LLM code generation in optimization, and the scale of the empirical study (47 problems, 3 LLMs, 5 baselines) is a useful contribution. However, the central performance claim is currently compromised by an experimental design that conflates the quality of the injected prior code with the contribution of LLM-guided search. The attribution result itself is suggestive but is based on a single problem (OneMax) and does not by itself establish the causal claim.

major comments (2)
  1. [Section VII.b / Algorithm 1 / Appendix B] For each pbo problem, A_bench is built from the 'top five algorithms' in the Doerr et al. (2020) repository, while all baselines are initialized with generic random-search code (Appendix B). BAG therefore starts from near-solver-quality code and periodically re-injects it (Algorithm 1, lines 7-9). Its AUC advantage could be caused entirely by the prior code, with the LLM refinement contributing little. This also contradicts the Section I claim that 'no prior knowledge from tested suites is exposed': selecting algorithms using the evaluation suite's own performance data is exactly such prior knowledge. In addition, q=10 is tuned on the same test suites (Appendix D). The comparison must be completed with (i) the AUC of A_bench alone (per-algorithm and best-of-five), (ii) BAG initialized from random search, and (iii) at least one baseline (e.g., LLaMEA) initialized with the same A_bench cod
  2. [Appendix H (Tables XI-XII)] The captions of Tables XI and XII state that 'the best results obtained by the oracle codes are also considered,' but the tables contain no oracle column. The oracle-code baseline is exactly the control needed to separate prior-code quality from LLM-driven refinement, and its absence is a concrete missing piece of evidence for the central claim. Please add explicit oracle results (e.g., the best-of-five A_bench AUC on the test instances) to these tables and to the aggregated comparison.
minor comments (7)
  1. [Section VII.c, Table II] The main-text Table II is identical to Table I (the relevance-score table) and does not contain the reported normalized AUC means and ranks. The actual comparison tables appear only in Appendix F. Please correct the main-text table or refer the reader explicitly to the appendix.
  2. [Section VII.c] In the sentence 'Figure 5 illustrates the convergence process of BGA on Sphere', 'BGA' should be 'BAG'.
  3. [Section IV.A] The text refers to 'the classic LPR assumption'; the acronym should be LRP (layer-wise relevance propagation).
  4. [Appendix F (Tables V-X)] The table captions read 'FOUR LLM-DRIVEN APPROACHES' but each table contains six methods (BAG, EoH, LHNS, LLaMEA, MCTS-AHD, ReEvo). Please correct the captions.
  5. [Section V / Table I] The attribution analysis is performed only on OneMax (F1) and with two open-source LLMs, while the BAG experiments use three proprietary LLMs across 47 problems. Please add an explicit caveat or a few additional attribution measurements on other functions, otherwise the transfer of the attribution conclusion to the main evaluation setting is an assumption.
  6. [Appendix F / Reproducibility] The specific top-five algorithms used as A_bench for each pbo problem are not listed. Since the paper's method depends on these concrete code priors, the exact algorithm names and source code (or a link to the repository entries) should be provided.
  7. [Algorithm 1] The input line 'A set of or a problem and a fitness measure F' is garbled; please rephrase, e.g., 'a problem instance, a fitness measure F, a set of benchmark algorithms A_bench, ...'.

Circularity Check

0 steps flagged

No significant circularity; empirical method with a benchmark-prior confound, not a circular derivation.

full rationale

The paper's central claims are empirical. (1) The AttnLRP token-analysis (Section V) independently measures prompt–code relevance and is not derived from BAG's performance. (2) BAG is an engineering extension of LLaMEA with injected benchmark code, and its superior performance is tested against five external baseline methods across three LLMs and two benchmark suites (Section VII, Tables V–X). There is no equation in which a prediction reduces to a fitted input by construction (e.g., no Eq. X = Eq. Y, no fitted parameter renamed as a prediction). (3) The self-citations (LLaMEA, Doerr et al. 2020) are used as a baseline and as a data source, respectively; they are not invoked as an unverified uniqueness theorem or as the sole support for the central claim. A real methodological concern is that the benchmark priors are selected per problem from the same suite's repository ('we select the top five algorithms reported in the repository of (Doerr et al., 2020) for each pbo problem') and evaluated on the same suite, so part of BAG's advantage could stem from initialization quality rather than LLM-guided search. Appendix H states that oracle-code results 'are also considered' but the oracle column is absent from Tables XI–XII. These are potential leaks/confounds in the experimental comparison, not circularity in the derivation of the results. Therefore, the circularity score is 1.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The central claim rests on a small set of design choices: the attribution method's validity, the (1+1) elitist analogy, and critically the selection of benchmark priors from the same benchmark data used for evaluation. The q hyperparameter and target sets are tuned/derived from the evaluation suites. No new physical or mathematical entities are introduced.

free parameters (4)
  • q (benchmark injection frequency) = 10
    Selected by comparing q ∈ {1,5,10,20,40,100} on the same pbo and bbob suites (Appendix D). This is a hyperparameter tuned to the evaluation data.
  • A_bench composition for pbo = top 5 algorithms per problem from Doerr et al. (2020)
    The five prior algorithms are selected per problem from the same benchmark repository used for evaluation, with no independent selection criterion.
  • AUC target sets Phi = problem-specific, e.g., {50,...,100} for OneMax
    Appendix A states the target sets are 'determined based on the corresponding benchmark data', tying the evaluation metric to the same data used for prior selection.
  • refine/create probability = 0.5
    The probability of choosing refine vs. create is fixed at 0.5, following prior LLM-driven approaches. It is a hand-chosen design constant, not fitted here.
axioms (5)
  • domain assumption AttnLRP relevance scores faithfully reflect the causal contribution of prompt tokens to generated algorithmic code.
    The paper's first contribution relies entirely on AttnLRP as the explainer (Section IV-A), without validating its attributions on this specific code-generation task. If AttnLRP is unreliable, the token-level finding collapses.
  • domain assumption Theoretical guarantees for (1+1) mutation-based evolutionary algorithms transfer to LLM-generated code mutations.
    Section VII cites Witt (2006), Droste et al. (2002), and Hoos/Stützle for the (1+1) elitist strategy, but LLM code mutations are not bit-flip mutations; this is an analogy rather than a proof.
  • ad hoc to paper Per-problem selection of top benchmark algorithms from the evaluation suite does not leak problem identity or prior performance information.
    The paper claims strict black-box settings (Section I and Appendix B), but A_bench for pbo is chosen per problem from the same benchmark repository used to define the test functions and target sets.
  • standard math pbo and bbob suites with the chosen AUC target sets are a valid and representative measure of BBO performance.
    The paper relies on standard benchmarking practice and the anytime AUC measure (Section III), which is broadly accepted but still a modeling choice for what counts as 'performance'.
  • domain assumption LLM outputs with default sampling parameters are representative and stable enough for comparisons across methods.
    All models use default sampling parameters; no seed control or repeated-run variance is reported for the LLM generation stage, yet conclusions are drawn from single evolutionary trajectories.

reviewed 2026-08-02 · how reviews work

0 comments
Cite this review

Pith. "Pith review of From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priors." pith.science (2026). https://pith.science/paper/E36JSH7B

@misc{pith2026260302792,
  author       = {Pith},
  title        = {Pith review of: From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E36JSH7B}},
  note         = {Machine review of arXiv:2603.02792}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large Language Models (LLMs) have already been widely adopted for automated algorithm design, demonstrating strong abilities in generating and evolving algorithms across various fields. Existing work has largely focused on examining their effectiveness in solving specific problems, with search strategies primarily guided by adaptive prompt designs. In this paper, through investigating the token-wise attribution of the prompts to LLM-generated algorithmic codes, we show that providing high-quality algorithmic code examples can substantially improve the performance of the LLM-driven optimization. Building upon this insight, we propose leveraging prior benchmark algorithms to guide LLM-driven optimization and demonstrate superior performance on two black-box optimization benchmarks: the pseudo-Boolean optimization suite (pbo) and the black-box optimization suite (bbob). Our findings highlight the value of integrating benchmarking studies to enhance both efficiency and robustness of the LLM-driven black-box optimization methods.

Figures

Figures reproduced from arXiv: 2603.02792 by Ananta Shahane, Furong Ye, Niki van Stein, Qi Huang, Thomas B\"ack.

Figure 1
Figure 1. Figure 1: The workflow of LLM-driven optimization approaches [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The heatmap of the token-wise relevance of a given prompt to [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Convergence process of refinement-only LHNS and LLaMEA methods for the OneMax problem. x-axis represents the number of algorithms gener￾ated by the LLM, and y-axis indicates the best-so-far AUC value. Results are from using Gemini, GPT, and Qwen, respectively (from Left to Right). the original baselines (LHNS and LLaMEA), achieving the best AUC across three tested LLMs. However, identifying the suitable ex… view at source ↗
Figure 4
Figure 4. Figure 4: Boxplots of the best normalized AUC values obtained by the [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: An illustration of the contribution of queries [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: CodeBLEU similarity scores between ordered pairs of generated algorithms (see Equation 2). Axes present time (i.e., the number of algorithms generated). Each column shows the relevance scores between a newly gener￾ated algorithm and all previous ones. Larger scores indicate higher relevance [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Average performance of BAG with different frequency factor q. The results are aggregated across three LLMs for pbo benchmarks. 1 5 10 20 40 100 Frequency Factor (q) 0.25 0.50 0.75 1.00 AUC [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Average performance of BAG with different frequency factor q. The results are aggregated across three LLMs for bbob benchmarks. of code. CodeBLEU was originally proposed to evaluate the logical and structural quality of generated code by comparing it with a reference implementation. Given a reference code snippet (R) and a generated code snippet (S), CodeBLEU computes a weighted sum of four sub-metrics der… view at source ↗
Figure 9
Figure 9. Figure 9: AUC values of the algorithms obtained by LLM-driven approaches on all pbo problems. The [PITH_FULL_IMAGE:figures/full_fig_p019_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: AUC values of the algorithms obtained by LLM-driven approaches on all pbo problems. The [PITH_FULL_IMAGE:figures/full_fig_p020_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: AUC values of the algorithms obtained by LLM-driven approaches on all pbo problems. The [PITH_FULL_IMAGE:figures/full_fig_p021_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: AUC values of the algorithms obtained by LLM-driven approaches on all bbob problems. The [PITH_FULL_IMAGE:figures/full_fig_p022_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: AUC values of the algorithms obtained by LLM-driven approaches on all bbob problems. The [PITH_FULL_IMAGE:figures/full_fig_p023_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: AUC values of the algorithms obtained by LLM-driven approaches on all bbob problems. The [PITH_FULL_IMAGE:figures/full_fig_p024_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: AUC values obtained by local search method LHNS with six different initial baseline heuristics on F2, F10, F17, F19, and F22 of the pbo suite (from Top to Bottom). The x-axis represents the cumulative number of algorithms generated by the LLM, and the y-axis indicates the best-so-far AUC value. The results are obtained using GPT 5 Nano, Gemini 2.0 Flash, and Qwen3 Coder Flash (from Left to Right) [PITH_F… view at source ↗
Figure 16
Figure 16. Figure 16: AUC values obtained by refinement-only LLaMEA with six different initial baseline heuristics on F2, F10, F17, F19, and F22 of the pbo suite (from Top to Bottom). The x-axis represents the cumulative number of algorithms generated by the LLM, and the y-axis indicates the best-so-far AUC value. The results are obtained using GPT 5 Nano, Gemini 2.0 Flash, and Qwen3 Coder Flash (from Left to Right) [PITH_FUL… view at source ↗
Figure 17
Figure 17. Figure 17: Averaged proportions of failed code generation across 23 pbo problems for the compared LLM-driven approaches. The performances of using Gemin, GPT, and Qwen are plotted from left to right. 0.0 0.2 0.4 0.6 0.8 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Function ID Gemini 0.0 0.2 0.4 0.6 0.8 GPT 0.0 0.2 0.4 0.6 0.8 Qwen BAG EoH LHNS LLaMEA MCTS-AHD ReEvo [PITH_FULL_IMAGE:figures/full_fi… view at source ↗
Figure 18
Figure 18. Figure 18: Averaged proportions of failed code generation across 24 bbob problems for the compared LLM-driven approaches. The performances of using Gemin, GPT, and Qwen are plotted from left to right [PITH_FULL_IMAGE:figures/full_fig_p028_18.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Breaking Validity-Induced Boundaries to Expand Algorithm Search Space: A Two-Stage AST-Based Operator for LLM-Driven Automated Heuristic Evolution

    cs.NE 2026-04 conditional novelty 7.0

    A two-stage AST-based crossover and mutation operator with LLM repair expands the search space in LLM-driven heuristic evolution and improves performance on TSP and online bin packing.

Reference graph

Works this paper leans on

23 extracted references · 8 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Focus only on algorithmic changes, not formatting or comments

    % self.dim fy = func(y) iffy > fx: x, fx = y, fy iffy >= func.optimum.y: break returnfx, x ``` □Strategy ▷Refine the example algorithm to improve its perfor- mance on the given task. Focus only on algorithmic changes, not formatting or comments. APPENDIXC ADDITIONALINFORMATION ONEXPERIMENTALSETUP a) Details on the LLMs:We conduct the token-wise ex- planat...

    2020

  2. [2]

    Approxi- mation algorithms for bin-packing—an updated survey,

    E. G. Coffman Jr, M. R. Garey, and D. S. Johnson, “Approxi- mation algorithms for bin-packing—an updated survey,” in Algorithm Design for Computer System Design. Springer, 1984, pp. 49–106. F. Liu, T. Xialiang, M. Yuan, X. Lin, F. Luo, Z. Wang, Z. Lu, and Q. Zhang, “Evolution of heuristics: Towards efficient automatic algorithm design using large language...

    1984

  3. [5]

    Autopbo: Llm- powered optimization for local search pbo solvers,

    J. Li, Y . Chu, Y . Sun, M. Zou, and S. Cai, “Autopbo: Llm- powered optimization for local search pbo solvers,”arXiv preprint arXiv:2509.04007,

    Pith/arXiv arXiv

  4. [6]

    Understanding the importance of evolutionary search in automated heuristic design with large language models,

    R. Zhang, F. Liu, X. Lin, Z. Wang, Z. Lu, and Q. Zhang, “Understanding the importance of evolutionary search in automated heuristic design with large language models,” in International Conference on Parallel Problem Solving from Nature. Springer, 2024, pp. 185–202. R. Achtibat, S. M. Vakilzadeh Hatefi, M. Dreyer, A. Jain, T. Wiegand, S. R. Lapuschkin, and...

    2024

  5. [7]

    Benchmarking in optimization: Best practice and open issues,

    T. Bartz-Beielstein, C. Doerr, D. v. d. Berg, J. Bossek, S. Chandrasekaran, T. Eftimov, A. Fischbach, P. Kerschke, W. La Cava, M. Lopez-Ibanezet al., “Benchmarking in optimization: Best practice and open issues,”arXiv preprint arXiv:2007.03488,

    Pith/arXiv arXiv 2007

  6. [9]

    Visualizing and understanding neural models in NLP,

    J. Li, X. Chen, E. Hovy, and D. Jurafsky, “Visualizing and understanding neural models in NLP,” inConference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, K. Knight, A. Nenkova, and O. Rambow, Eds., 2016, pp. 681–691. Z. Wu, Y . Chen, B. Kao, and Q. Liu, “Perturbed masking: Parameter-free prob...

    Pith/arXiv arXiv 2016

  7. [14]

    Compar- ing results of 31 algorithms from the black-box optimization benchmarking bbob-2009,

    N. Hansen, A. Auger, R. Ros, S. Finck, and P. Poˇs´ık, “Compar- ing results of 31 algorithms from the black-box optimization benchmarking bbob-2009,” inConference Companion on Genetic and Evolutionary Computation, 2010, pp. 1689–

    2009

  8. [16]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, P. Isabelle, E. Charniak, and D. Lin, Eds. Philadelphia, Pennsylvania, USA: Association for Computational Linguistics, Jul. 2002, pp. 311–318. [Online]...

    2002

  9. [18]

    The cma evolution strategy: A tutorial,

    N. Hansen, “The cma evolution strategy: A tutorial,”arXiv preprint arXiv:1604.00772,

    Pith/arXiv arXiv

  10. [22]

    approximately twice on average. APPENDIXE ANALYSIS OFIMPACT OFBENCHMARK(-INDUCED) ALGORITHM TOGENERATEDHEURISTICS To analyze the impact of benchmark algorithm on the gener- ated heuristics, we use CodeBLEU Ren et al. (2020), a metric for measuring the similarity (relevance) between two pieces 3https://github.com/FeiLiu36/EoH 4https://github.com/Acquent0/L...

    2020

  11. [23]

    Thex-axis represents the cumulative number of algorithms generated by the LLM, and they-axis indicates the best-so-far AUC value

    AUC values of the algorithms obtained by LLM-driven approaches on all bbob problems. Thex-axis represents the cumulative number of algorithms generated by the LLM, and they-axis indicates the best-so-far AUC value. The results are obtained using Qwen3 Coder Flash. 25 TABLE XI THE BEST NORMALIZED(HIGHER IS BETTER) AUCACHIEVED BY FOUR LLM-DRIVEN APPROACHES ...

    1914

  12. [235]

    23 238–23

    PMLR, 21– 27 Jul 2024, pp. 23 238–23

    2024

  13. [255]

    JoPA: Explaining large language model’s generation via joint prompt attribution,

    Y . Chang, B. Cao, Y . Wang, J. Chen, and L. Lin, “JoPA: Explaining large language model’s generation via joint prompt attribution,” inAnnual Meeting of the Association for Computational Linguistics, 2025, pp. 22 106–22

    2025

  14. [1696]

    Codebleu: a method for automatic evaluation of code synthesis,

    S. Ren, D. Guo, S. Lu, L. Zhou, S. Liu, D. Tang, N. Sundare- san, M. Zhou, A. Blanco, and S. Ma, “Codebleu: a method for automatic evaluation of code synthesis,”arXiv preprint arXiv:2009.10297,

    Pith/arXiv arXiv 2009

  15. [2002]

    Stochastic local search,

    H. H. Hoos and T. St¨νtzle, “Stochastic local search,” in Handbook of Approximation Algorithms and Metaheuristics. Chapman and Hall/CRC, 2018, pp. 297–307. H. Ye, J. Wang, Z. Cao, F. Berto, C. Hua, H. Kim, J. Park, and G. Song, “Reevo: Large language models as hyper- heuristics with reflective evolution,” inAdvances in Neural 11 Information Processing Sys...

    2018

  16. [2016]

    Cumulative step-size adaptation on linear functions,

    A. Chotard, A. Auger, and N. Hansen, “Cumulative step-size adaptation on linear functions,” inInternational Conference on Parallel Problem Solving from Nature. Springer, 2012, pp. 72–81. S. Das and P. N. Suganthan, “Differential evolution: A survey of the state-of-the-art,”IEEE Transactions on Evolutionary Computation, vol. 15, no. 1, pp. 4–31,

    2012

  17. [2018]

    Transformers: State-of-the-art natural language processing,

    T. Wolf, L. Debut, V . Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davi- son, S. Shleifer, P. von Platen, C. Ma, Y . Jernite, J. Plu, C. Xu, T. Le Scao, S. Gugger, M. Drame, Q. Lhoest, and A. Rush, “Transformers: State-of-the-art natural language processing,” inConference on Empirical Methods in Natural Language ...

    2020

  18. [2019]

    Automated design of metaheuristic algorithms,

    T. St ¨utzle and M. L ´opez-Ib´a˜nez, “Automated design of metaheuristic algorithms,” inHandbook of metaheuristics. Springer, 2018, pp. 541–579. E. Schede, J. Brandt, A. Tornede, M. Wever, V . Bengs, E. H ¨ullermeier, and K. Tierney, “A survey of methods for automated algorithm configuration,”Journal of Artificial Intelligence Research, vol. 75, pp. 425–487,

    2018

  19. [2020]

    ” why should i trust you?

    M. T. Ribeiro, S. Singh, and C. Guestrin, “” why should i trust you?” explaining the predictions of any classifier,” inACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 1135–1144. E. Kokalj, B. ˇSkrlj, N. Lavra ˇc, S. Pollak, and M. Robnik- ˇSikonja, “BERT meets shapley: Extending SHAP explana- tions to transformer-base...

    Pith/arXiv arXiv 2016

  20. [2021]

    Autonomous 10 code evolution meets np-completeness,

    C. Yu, R. Liang, C.-T. Ho, and H. Ren, “Autonomous 10 code evolution meets np-completeness,”arXiv preprint arXiv:2509.07367,

    Pith/arXiv arXiv

  21. [2022]

    Pretrained optimization model for zero-shot black box optimization,

    X. Li, K. Wu, X. Zhang, H. Wang, J. Liuet al., “Pretrained optimization model for zero-shot black box optimization,” inAdvances in Neural Information Processing Systems, vol. 37, 2024, pp. 14 283–14

    2024

  22. [2024]

    Automatically discovering heuristics in a complex sat solver with large language models,

    Y . Sun, F. Ye, Z. Chen, K. Wei, and S. Cai, “Automatically discovering heuristics in a complex sat solver with large language models,”arXiv preprint arXiv:2507.22876,

    Pith/arXiv arXiv

  23. [2025]

    Multi- objective evolution of heuristic using large language model,

    S. Yao, F. Liu, X. Lin, Z. Lu, Z. Wang, and Q. Zhang, “Multi- objective evolution of heuristic using large language model,” inAAAI Conference on Artificial Intelligence, vol. 39, 2025, pp. 27 144–27

    2025

This paper was first reviewed by deepseek-v4-flash on August 2, 2026.