Latent Heuristic Search: Continuous Optimization for Automated Algorithm Design

Cheikh Ahmed; Mahdi Mostajabdaveh; Zirui Zhou

arxiv: 2605.17137 · v1 · pith:LHIRNJFRnew · submitted 2026-05-16 · 💻 cs.AI

Latent Heuristic Search: Continuous Optimization for Automated Algorithm Design

Cheikh Ahmed , Mahdi Mostajabdaveh , Zirui Zhou This is my paper

Pith reviewed 2026-05-20 14:36 UTC · model grok-4.3

classification 💻 cs.AI

keywords latent heuristic searchcontinuous optimizationautomated algorithm designheuristic discoveryLLM-guided code generationcombinatorial optimizationTSPCVRP

0 comments

The pith

Continuous optimization in a learned latent space discovers heuristics competitive with discrete evolutionary search.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to show that mapping discrete heuristic programs into a continuous latent manifold allows gradient-based search to locate better algorithms than stochastic sampling in program space. An encoder embeds existing code, a surrogate model supplies differentiable performance estimates, and an invertible normalizing flow regularizes the embeddings to a Gaussian prior so that gradient ascent can be performed reliably. The resulting latent vectors are projected to soft prompts that condition a frozen LLM to output fresh executable heuristics. The approach is demonstrated on TSP, CVRP, knapsack, and online bin packing, where it reaches performance levels comparable to strong discrete baselines. A reader would care because the method supplies a complementary route to automated algorithm design that can exploit the smooth structure and gradient information typical of machine-learning pipelines.

Core claim

The central claim is that continuous optimization over a learned latent manifold of heuristics, produced by an encoder and regularized by an invertible normalizing flow to a Gaussian prior, enables gradient ascent on a differentiable performance surrogate; the optimized latent vectors are then mapped to soft prompts that condition a frozen LLM to synthesize novel executable heuristics whose performance is competitive with state-of-the-art discrete evolutionary baselines on TSP, CVRP, KSP, and OBP.

What carries the argument

The learned latent manifold together with its encoder, differentiable performance surrogate, invertible normalizing flow for regularization to a Gaussian prior, and learned mapper from optimized embeddings to soft LLM prompts.

If this is right

Gradient ascent in the latent space supplies a methodological alternative to stochastic discrete sampling for heuristic discovery.
The same pipeline applies directly to multiple combinatorial domains including routing and packing.
Optimized latent points can be decoded into executable code that improves measured performance over baseline heuristics.
Conditioning a frozen LLM with soft prompts generated from the latent space allows synthesis of new algorithms without retraining the language model.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Hybrid discrete-continuous search that alternates latent gradient steps with occasional discrete mutations could be tested for further gains.
The smoothness assumption opens the possibility of adding secondary objectives such as interpretability or generalization directly into the latent optimization.
Varying the latent dimension or flow architecture on larger problem instances would reveal scalability characteristics not addressed in the current benchmarks.

Load-bearing premise

The encoder must produce a latent manifold on which the surrogate model's performance predictions are accurate and smooth enough that gradient ascent reliably yields points that decode to executable heuristics better than those found by discrete search.

What would settle it

Running the full pipeline on any of the four benchmark problems and measuring that the heuristics produced after latent optimization and LLM projection perform no better than the starting population or that the surrogate's predicted scores fail to correlate with actual solution quality or runtime.

Figures

Figures reproduced from arXiv: 2605.17137 by Cheikh Ahmed, Mahdi Mostajabdaveh, Zirui Zhou.

**Figure 1.** Figure 1: Latent Space Visualization. A 2D t-SNE projection of program embeddings for the TSP. The color gradient represents the heuristic score, showing that high-performing programs (lighter colors) cluster together rather than being randomly dispersed. 4.5 Visualization of Heuristics To analyze the topology of the program space, we project the TSP heuristic embeddings (learned by the encoder E) into two dimension… view at source ↗

read the original abstract

The integration of Large Language Models (LLMs) into evolutionary frameworks has established a new paradigm for automated heuristic discovery. Despite their promise, these methods typically search in the discrete space of program syntax, relying on stochastic sampling to navigate a highly non-convex optimization landscape. This work proposes a continuous heuristic discovery framework that shifts optimization to a learned latent manifold. We employ an encoder to map discrete programs into continuous embeddings and train a differentiable surrogate model to predict performance, enabling gradient-based search. To regularize the optimization trajectory, an invertible normalizing flow maps these embeddings to a structured Gaussian prior, where we perform gradient ascent. The resulting optimized latent vectors are projected through a learned mapper into soft prompts, which condition a frozen LLM to synthesize novel executable heuristics. We evaluate the proposed method on the Traveling Salesman Problem (TSP), the Capacitated Vehicle Routing Problem (CVRP), the Knapsack Problem (KSP), and Online Bin Packing (OBP). Empirical results demonstrate that continuous latent-space optimization achieves performance competitive with state-of-the-art discrete evolutionary baselines while offering a complementary methodological alternative for automated algorithm design. The implementation code is available at \url{https://github.com/cheikh025/LHS}.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

This paper moves heuristic search into a continuous latent space with a surrogate and normalizing flow before LLM synthesis, but the abstract gives no numbers or validation that the surrogate actually guides better real-world heuristics.

read the letter

The main thing to know is that this work tries to replace stochastic discrete search over program syntax with gradient ascent inside a learned latent manifold. They encode existing heuristics, train a surrogate to score them, regularize the space with an invertible flow, optimize the latent vectors, then map them to soft prompts that steer a frozen LLM to output new executable code. The claim is that this continuous route matches or approaches the performance of current evolutionary baselines on TSP, CVRP, KSP, and OBP while offering a different search mechanism. Code is linked, which is helpful. The combination of encoder, differentiable surrogate, flow prior, and LLM-conditioned generation from optimized latents does not appear in the cited prior work, so that specific pipeline is the clearest novelty. It frames a methodological alternative cleanly and evaluates on standard external benchmarks rather than inventing its own test cases. The central weakness is the missing evidence. The abstract asserts competitive results but supplies no quantitative scores, error bars, surrogate training details, or ablations. More importantly, the pipeline has a non-differentiable LLM step after the latent optimization, so any gap between surrogate predictions and actual benchmark performance would undermine the whole justification for continuous search. The stress-test note is right to flag the lack of hold-out correlation checks or comparisons between surrogate-guided latents and random ones. Without those, it is hard to know whether the gradient steps are doing useful work or just producing prompts that the LLM happens to turn into decent code anyway. This is aimed at researchers in automated algorithm design and LLM-augmented optimization. Someone already following the discrete evolutionary program search literature would see the contrast and might want to test the idea, but only after seeing the full empirical section. I would send it for peer review. The framing is coherent enough that referees can check whether the surrogate actually correlates with ground truth and whether the continuous path delivers measurable gains over the discrete baselines.

Referee Report

3 major / 2 minor

Summary. The paper proposes Latent Heuristic Search (LHS), a continuous optimization framework for automated heuristic discovery. Discrete programs are mapped to a latent manifold via an encoder; a differentiable surrogate predicts performance; a normalizing flow regularizes the space to a Gaussian prior; gradient ascent is performed in latent space; optimized vectors are mapped to soft prompts that condition a frozen LLM to synthesize executable heuristics. The approach is evaluated on TSP, CVRP, KSP and OBP and claims performance competitive with state-of-the-art discrete evolutionary baselines while providing a complementary methodological alternative.

Significance. If the central empirical claim holds, the work supplies a concrete continuous alternative to discrete evolutionary search in LLM-driven automated algorithm design. The use of a learned latent manifold plus normalizing-flow regularization is a technically coherent way to enable gradient-based search over program space, and the public GitHub link supplies an external reproducibility path.

major comments (3)

The manuscript provides no quantitative results, error bars, training details for the surrogate, or ablation studies in support of the claim that continuous latent-space optimization is competitive with discrete baselines. This absence prevents verification of the central empirical claim from the given text.
No hold-out correlation, calibration plots, or ablation comparing surrogate-guided latent samples against random latent samples or ground-truth evaluations are reported. Because the LLM synthesis step is non-differentiable, any mismatch between surrogate output and true objective value directly undermines the justification for performing gradient ascent on the surrogate rather than discrete search.
The weakest modeling assumption—that the encoder produces a latent manifold on which the surrogate’s predictions are sufficiently accurate and smooth for reliable gradient ascent—is stated but not empirically tested against the downstream benchmark scores on TSP/CVRP/KSP/OBP.

minor comments (2)

The abstract asserts competitive performance without naming the specific baselines, metrics, or numerical gaps; adding a compact results table or key numbers would improve clarity.
Notation for the latent dimension, flow parameters, and mapper is introduced without a consolidated symbol table; a short notation summary would aid readability.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive feedback and for acknowledging the methodological novelty of shifting heuristic search to a continuous latent manifold. We agree that the empirical support requires strengthening to fully substantiate the competitiveness claim and the reliability of the surrogate-guided optimization. The revised manuscript will incorporate the requested quantitative details, validations, and tests.

read point-by-point responses

Referee: The manuscript provides no quantitative results, error bars, training details for the surrogate, or ablation studies in support of the claim that continuous latent-space optimization is competitive with discrete baselines. This absence prevents verification of the central empirical claim from the given text.

Authors: We agree that the presentation of results needs to be more detailed and self-contained. In the revision we will expand Section 4 with explicit tables reporting mean performance and standard deviations (error bars) over repeated runs for LHS versus the discrete evolutionary baselines on all four problems. A new subsection will describe the surrogate architecture, training dataset size, optimizer, learning rate schedule, and convergence behavior. Additional ablation tables will isolate the contribution of latent-space gradient ascent relative to discrete search. revision: yes
Referee: No hold-out correlation, calibration plots, or ablation comparing surrogate-guided latent samples against random latent samples or ground-truth evaluations are reported. Because the LLM synthesis step is non-differentiable, any mismatch between surrogate output and true objective value directly undermines the justification for performing gradient ascent on the surrogate rather than discrete search.

Authors: This concern is well-founded. We will add a dedicated surrogate-validation subsection containing (i) Pearson and Spearman correlations on a held-out set of programs, (ii) calibration plots of predicted versus observed objective values, and (iii) an ablation that compares final heuristic quality obtained from surrogate-optimized latents, randomly sampled latents, and direct ground-truth evaluations. These results will quantify how well the surrogate approximates the true objective despite the non-differentiable LLM decoding step. revision: yes
Referee: The weakest modeling assumption—that the encoder produces a latent manifold on which the surrogate’s predictions are sufficiently accurate and smooth for reliable gradient ascent—is stated but not empirically tested against the downstream benchmark scores on TSP/CVRP/KSP/OBP.

Authors: We will directly test this assumption in the revision by reporting the correlation between surrogate predictions and realized benchmark scores for the final synthesized heuristics on each of the four problem domains. We will also include trajectory plots showing objective improvement along the gradient-ascent path in latent space and quantify the smoothness via local Lipschitz estimates or finite-difference checks around the optimized points. revision: yes

Circularity Check

0 steps flagged

No circularity: derivation relies on external benchmarks and independent surrogate training

full rationale

The paper introduces a latent-space continuous optimization pipeline (encoder + surrogate + normalizing flow + mapper to LLM prompts) and evaluates it on standard external benchmarks (TSP, CVRP, KSP, OBP). No equations, self-citations, or fitted inputs are shown that reduce reported performance gains to quantities defined by the same runs or prior author work. The central claim is an empirical comparison against discrete evolutionary baselines, supported by a public GitHub implementation, satisfying the criteria for a self-contained derivation without load-bearing self-reference or construction-by-definition.

Axiom & Free-Parameter Ledger

2 free parameters · 1 axioms · 1 invented entities

The framework rests on the existence of a useful continuous manifold for discrete heuristics and on the ability of a learned surrogate to guide search; these are introduced by the method rather than derived from prior literature.

free parameters (2)

latent dimension
Dimensionality of the continuous embedding space produced by the encoder; chosen during training.
surrogate architecture hyperparameters
Number of layers and hidden units in the performance-predicting neural network.

axioms (1)

domain assumption A differentiable surrogate can be trained to predict heuristic performance from latent vectors with sufficient accuracy for gradient ascent to be useful.
Invoked when the method replaces discrete search with gradient steps on the surrogate output.

invented entities (1)

learned latent manifold of heuristics no independent evidence
purpose: Provides a continuous space in which gradient-based optimization can be performed on discrete programs.
New representation introduced by the encoder; no independent evidence outside the training procedure is given.

pith-pipeline@v0.9.0 · 5741 in / 1281 out tokens · 47864 ms · 2026-05-20T14:36:30.697255+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

23 extracted references · 23 canonical work pages · 3 internal anchors

[1]

In: Proceedings of the 22nd International Conference on Machine Learning

Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., Hullender, G.: Learning to rank using gradient descent. In: Proceedings of the 22nd International Conference on Machine Learning. p. 89–96. ICML ’05, Association for Computing Machinery, New York, NY, USA (2005). https://doi.org/10.1145/1102351.1102363, https://doi.org/10.1145/11023...

work page doi:10.1145/1102351.1102363 2005
[2]

In: Proceedings of the AAAI Conference on Artificial Intelligence

Dat, P.V.T., Doan, L., Binh, H.T.T.: Hsevo: Elevating automatic heuristic design with diversity-driven harmony search and genetic algorithm using llms. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 39, pp. 26931–26938 (2025), https://github.com/datphamvn/HSEvo

work page 2025
[3]

Dinh, L., Krueger, D., Bengio, Y.: Nice: Non-linear independent components estimation (2015), https://arxiv.org/abs/1410.8516

work page internal anchor Pith review Pith/arXiv arXiv 2015
[4]

In: International Conference on Learning Representations (2017), https://openreview.net/forum?id=HkpbnH9lx

Dinh, L., Sohl-Dickstein, J., Bengio, S.: Density estimation using real NVP. In: International Conference on Learning Representations (2017), https://openreview.net/forum?id=HkpbnH9lx

work page 2017
[5]

In: Rogers, A., Boyd-Graber, J., Okazaki, N

Gu, Y., Feng, X., Ma, S., Zhang, L., Gong, H., Zhong, W., Qin, B.: Controllable text generation via probability density estimation in the latent space. In: Rogers, A., Boyd-Graber, J., Okazaki, N. (eds.) Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 12590–12616. Association for Computa...

work page doi:10.18653/v1/2023.acl-long.704 2023
[6]

Hansen, N.: The cma evolution strategy: A tutorial (2023), https://arxiv.org/abs/1604.00772

work page internal anchor Pith review Pith/arXiv arXiv 2023
[7]

Lee, C.C., Lee, D.T.: A simple on-line bin-packing algorithm. J. ACM32(3), 562–572 (Jul 1985). https://doi.org/10.1145/3828.3833, https://doi.org/10.1145/3828.3833

work page doi:10.1145/3828.3833 1985
[8]

In: Webber, B., Cohn, T., He, Y., Liu, Y

Li, B., Zhou, H., He, J., Wang, M., Yang, Y., Li, L.: On the sentence embeddings from pre-trained language models. In: Webber, B., Cohn, T., He, Y., Liu, Y. (eds.) Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP). pp. 9119–9130. Association for Computational Linguistics, Online (Nov 2020). https://doi.org/10...

work page doi:10.18653/v1/2020.emnlp-main.733 2020
[9]

Li, X.L., Liang, P.: Prefix-tuning: Optimizing continuous prompts for generation (2021)

work page 2021
[10]

Lin, S., Kernighan, B.W.: An effective heuristic algorithm for the traveling-salesman problem. Oper. Res.21(2), 498–516 (Apr 1973). https://doi.org/10.1287/opre.21.2.498, https://doi.org/10.1287/opre.21.2.498

work page doi:10.1287/opre.21.2.498 1973
[11]

Proceedings of the 2020 Genetic and Evolutionary Computation Conference (2020), https://api.semanticscholar.org/CorpusID:220252273

Liskowski, P., Krawiec, K., Toklu, N.E., Swan, J.: Program synthesis as latent continuous optimiza- tion: evolutionary search in neural embeddings. Proceedings of the 2020 Genetic and Evolutionary Computation Conference (2020), https://api.semanticscholar.org/CorpusID:220252273

work page 2020
[12]

In: International Conference on Machine Learning (ICML) (2024), https://arxiv.org/abs/2401.02051

Liu, F., Tong, X., Yuan, M., Lin, X., Luo, F., Wang, Z., Lu, Z., Zhang, Q.: Evolution of heuristics: Towards efficient automatic algorithm design using large language model. In: International Conference on Machine Learning (ICML) (2024), https://arxiv.org/abs/2401.02051

work page arXiv 2024
[13]

Liu, F., Zhang, R., Xie, Z., Sun, R., Li, K., Lin, X., Wang, Z., Lu, Z., Zhang, Q.: Llm4ad: A platform for algorithm design with large language model (2024), https://arxiv.org/abs/2412.17287

work page arXiv 2024
[14]

In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025), https://openreview.net/forum?id=CsXKGIqZtr

Macfarlane, M., Bonnet, C.: Searching latent program spaces. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025), https://openreview.net/forum?id=CsXKGIqZtr

work page 2025
[15]

Mandal, S., Anderson, T.A., Turek, J., Gottschlich, J., Muzahid, A.: Synthesizing programs with con- tinuous optimization (2023), https://arxiv.org/abs/2211.00828

work page arXiv 2023
[16]

Novikov, A., V˜ u, N., Eisenberger, M., Dupont, E., Huang, P.S., Wagner, A.Z., Shirobokov, S., Ko- zlovskii, B., Ruiz, F.J.R., Mehrabian, A., Kumar, M.P., See, A., Chaudhuri, S., Holland, G., Davies, A., Nowozin, S., Kohli, P., Balog, M.: Alphaevolve: A coding agent for scientific and algorithmic dis- covery (2025), https://arxiv.org/abs/2506.13131

work page internal anchor Pith review Pith/arXiv arXiv 2025
[17]

Papamakarios, G., Nalisnick, E., Rezende, D.J., Mohamed, S., Lakshminarayanan, B.: Normalizing flows for probabilistic modeling and inference. J. Mach. Learn. Res.22(1) (Jan 2021)

work page 2021
[18]

https://doi.org/https://doi.org/10.1016/j.cor.2004.03.002, https://www.sciencedirect.com/science/article/pii/S030505480400036X

Pisinger, D.: Where are the hard knapsack problems? Computers & Operations Re- search32(9), 2271–2284 (2005). https://doi.org/https://doi.org/10.1016/j.cor.2004.03.002, https://www.sciencedirect.com/science/article/pii/S030505480400036X

work page doi:10.1016/j.cor.2004.03.002 2005
[19]

doi:10.1038/s41586-023-06924-6 , url =

Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M.P., Dupont, E., Ruiz, F.J.R., Ellenberg, J., Wang, P., Fawzi, O., Kohli, P., Fawzi, A.: Mathematical discoveries from program search with large language models. Nature (2023). https://doi.org/10.1038/s41586-023-06924-6

work page doi:10.1038/s41586-023-06924-6 2023
[20]

In: Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W

Trivedi, D., Zhang, J., Sun, S.H., Lim, J.J.: Learning to synthesize programs as interpretable and generalizable policies. In: Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems (2021), https://openreview.net/forum?id=wP9twkexC3V LSH 15

work page 2021
[21]

Computers & Operations Research140, 105643 (2022)

Vidal, T.: Hybrid genetic search for the cvrp: Open-source imple- mentation and swap* neighborhood. Computers & Operations Research 140, 105643 (2022). https://doi.org/https://doi.org/10.1016/j.cor.2021.105643, https://www.sciencedirect.com/science/article/pii/S030505482100349X

work page doi:10.1016/j.cor.2021.105643 2022
[22]

In: Advances in Neural Information Processing Systems (2024), https://github.com/ai4co/reevo

Ye, H., Wang, J., Cao, Z., Berto, F., Hua, C., Kim, H., Park, J., Song, G.: Reevo: Large language models as hyper-heuristics with reflective evolution. In: Advances in Neural Information Processing Systems (2024), https://github.com/ai4co/reevo

work page 2024
[23]

"" Design a novel algorithm to select the next node in each step

Zheng, Z., Xie, Z., Wang, Z., Hooi, B.: Monte carlo tree search for comprehensive exploration in LLM-based automatic heuristic design. In: Forty-second International Conference on Machine Learning (2025), https://openreview.net/forum?id=Do1OdZzYHr 16 C. Ahmed et al. A Implementation Details We provide a detailed breakdown of the model architectures, train...

work page 2025

[1] [1]

In: Proceedings of the 22nd International Conference on Machine Learning

Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., Hullender, G.: Learning to rank using gradient descent. In: Proceedings of the 22nd International Conference on Machine Learning. p. 89–96. ICML ’05, Association for Computing Machinery, New York, NY, USA (2005). https://doi.org/10.1145/1102351.1102363, https://doi.org/10.1145/11023...

work page doi:10.1145/1102351.1102363 2005

[2] [2]

In: Proceedings of the AAAI Conference on Artificial Intelligence

Dat, P.V.T., Doan, L., Binh, H.T.T.: Hsevo: Elevating automatic heuristic design with diversity-driven harmony search and genetic algorithm using llms. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 39, pp. 26931–26938 (2025), https://github.com/datphamvn/HSEvo

work page 2025

[3] [3]

Dinh, L., Krueger, D., Bengio, Y.: Nice: Non-linear independent components estimation (2015), https://arxiv.org/abs/1410.8516

work page internal anchor Pith review Pith/arXiv arXiv 2015

[4] [4]

In: International Conference on Learning Representations (2017), https://openreview.net/forum?id=HkpbnH9lx

Dinh, L., Sohl-Dickstein, J., Bengio, S.: Density estimation using real NVP. In: International Conference on Learning Representations (2017), https://openreview.net/forum?id=HkpbnH9lx

work page 2017

[5] [5]

In: Rogers, A., Boyd-Graber, J., Okazaki, N

Gu, Y., Feng, X., Ma, S., Zhang, L., Gong, H., Zhong, W., Qin, B.: Controllable text generation via probability density estimation in the latent space. In: Rogers, A., Boyd-Graber, J., Okazaki, N. (eds.) Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 12590–12616. Association for Computa...

work page doi:10.18653/v1/2023.acl-long.704 2023

[6] [6]

Hansen, N.: The cma evolution strategy: A tutorial (2023), https://arxiv.org/abs/1604.00772

work page internal anchor Pith review Pith/arXiv arXiv 2023

[7] [7]

Lee, C.C., Lee, D.T.: A simple on-line bin-packing algorithm. J. ACM32(3), 562–572 (Jul 1985). https://doi.org/10.1145/3828.3833, https://doi.org/10.1145/3828.3833

work page doi:10.1145/3828.3833 1985

[8] [8]

In: Webber, B., Cohn, T., He, Y., Liu, Y

Li, B., Zhou, H., He, J., Wang, M., Yang, Y., Li, L.: On the sentence embeddings from pre-trained language models. In: Webber, B., Cohn, T., He, Y., Liu, Y. (eds.) Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP). pp. 9119–9130. Association for Computational Linguistics, Online (Nov 2020). https://doi.org/10...

work page doi:10.18653/v1/2020.emnlp-main.733 2020

[9] [9]

Li, X.L., Liang, P.: Prefix-tuning: Optimizing continuous prompts for generation (2021)

work page 2021

[10] [10]

Lin, S., Kernighan, B.W.: An effective heuristic algorithm for the traveling-salesman problem. Oper. Res.21(2), 498–516 (Apr 1973). https://doi.org/10.1287/opre.21.2.498, https://doi.org/10.1287/opre.21.2.498

work page doi:10.1287/opre.21.2.498 1973

[11] [11]

Proceedings of the 2020 Genetic and Evolutionary Computation Conference (2020), https://api.semanticscholar.org/CorpusID:220252273

Liskowski, P., Krawiec, K., Toklu, N.E., Swan, J.: Program synthesis as latent continuous optimiza- tion: evolutionary search in neural embeddings. Proceedings of the 2020 Genetic and Evolutionary Computation Conference (2020), https://api.semanticscholar.org/CorpusID:220252273

work page 2020

[12] [12]

In: International Conference on Machine Learning (ICML) (2024), https://arxiv.org/abs/2401.02051

Liu, F., Tong, X., Yuan, M., Lin, X., Luo, F., Wang, Z., Lu, Z., Zhang, Q.: Evolution of heuristics: Towards efficient automatic algorithm design using large language model. In: International Conference on Machine Learning (ICML) (2024), https://arxiv.org/abs/2401.02051

work page arXiv 2024

[13] [13]

Liu, F., Zhang, R., Xie, Z., Sun, R., Li, K., Lin, X., Wang, Z., Lu, Z., Zhang, Q.: Llm4ad: A platform for algorithm design with large language model (2024), https://arxiv.org/abs/2412.17287

work page arXiv 2024

[14] [14]

In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025), https://openreview.net/forum?id=CsXKGIqZtr

Macfarlane, M., Bonnet, C.: Searching latent program spaces. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025), https://openreview.net/forum?id=CsXKGIqZtr

work page 2025

[15] [15]

Mandal, S., Anderson, T.A., Turek, J., Gottschlich, J., Muzahid, A.: Synthesizing programs with con- tinuous optimization (2023), https://arxiv.org/abs/2211.00828

work page arXiv 2023

[16] [16]

Novikov, A., V˜ u, N., Eisenberger, M., Dupont, E., Huang, P.S., Wagner, A.Z., Shirobokov, S., Ko- zlovskii, B., Ruiz, F.J.R., Mehrabian, A., Kumar, M.P., See, A., Chaudhuri, S., Holland, G., Davies, A., Nowozin, S., Kohli, P., Balog, M.: Alphaevolve: A coding agent for scientific and algorithmic dis- covery (2025), https://arxiv.org/abs/2506.13131

work page internal anchor Pith review Pith/arXiv arXiv 2025

[17] [17]

Papamakarios, G., Nalisnick, E., Rezende, D.J., Mohamed, S., Lakshminarayanan, B.: Normalizing flows for probabilistic modeling and inference. J. Mach. Learn. Res.22(1) (Jan 2021)

work page 2021

[18] [18]

https://doi.org/https://doi.org/10.1016/j.cor.2004.03.002, https://www.sciencedirect.com/science/article/pii/S030505480400036X

Pisinger, D.: Where are the hard knapsack problems? Computers & Operations Re- search32(9), 2271–2284 (2005). https://doi.org/https://doi.org/10.1016/j.cor.2004.03.002, https://www.sciencedirect.com/science/article/pii/S030505480400036X

work page doi:10.1016/j.cor.2004.03.002 2005

[19] [19]

doi:10.1038/s41586-023-06924-6 , url =

Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M.P., Dupont, E., Ruiz, F.J.R., Ellenberg, J., Wang, P., Fawzi, O., Kohli, P., Fawzi, A.: Mathematical discoveries from program search with large language models. Nature (2023). https://doi.org/10.1038/s41586-023-06924-6

work page doi:10.1038/s41586-023-06924-6 2023

[20] [20]

In: Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W

Trivedi, D., Zhang, J., Sun, S.H., Lim, J.J.: Learning to synthesize programs as interpretable and generalizable policies. In: Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems (2021), https://openreview.net/forum?id=wP9twkexC3V LSH 15

work page 2021

[21] [21]

Computers & Operations Research140, 105643 (2022)

Vidal, T.: Hybrid genetic search for the cvrp: Open-source imple- mentation and swap* neighborhood. Computers & Operations Research 140, 105643 (2022). https://doi.org/https://doi.org/10.1016/j.cor.2021.105643, https://www.sciencedirect.com/science/article/pii/S030505482100349X

work page doi:10.1016/j.cor.2021.105643 2022

[22] [22]

In: Advances in Neural Information Processing Systems (2024), https://github.com/ai4co/reevo

Ye, H., Wang, J., Cao, Z., Berto, F., Hua, C., Kim, H., Park, J., Song, G.: Reevo: Large language models as hyper-heuristics with reflective evolution. In: Advances in Neural Information Processing Systems (2024), https://github.com/ai4co/reevo

work page 2024

[23] [23]

"" Design a novel algorithm to select the next node in each step

Zheng, Z., Xie, Z., Wang, Z., Hooi, B.: Monte carlo tree search for comprehensive exploration in LLM-based automatic heuristic design. In: Forty-second International Conference on Machine Learning (2025), https://openreview.net/forum?id=Do1OdZzYHr 16 C. Ahmed et al. A Implementation Details We provide a detailed breakdown of the model architectures, train...

work page 2025