Beyond the Hype: Benchmarking LLM-Evolved Heuristics for Bin Packing

Emma Hart; Kevin Sim; Quentin Renau

REVIEW 1 major objections 5 minor 2 cited by

Beyond the Hype: Benchmarking LLM-Evolved Heuristics for Bin Packing

T0 review · 1 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A broad benchmark of bin-packing heuristics finds that the decades-old Best-Fit rule still outperforms most LLM-evolved heuristics, which specialize to the data they were trained on.

desk verdict First large-scale benchmark of LLM-evolved bin-packing heuristics; the qualitative generalization finding is solid, but the main quantitative metric relies on a weak lower bound and exact numbers should be re-verified. read the letter →

arxiv 2501.11411 v1 pith:ZOCFD5U6 submitted 2025-01-20 cs.NE

Kevin Sim , Quentin Renau , Emma Hart This is my paper · ORCID

classification cs.NE

keywords LLM-evolvedheuristicsbinpackingbenchmarkinginstancespaceanalysisgeneralizationonlineevolutionarycomputationBest-Fitheuristic

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper puts five LLM-generated bin-packing heuristics and five classic hand-designed ones to the same test: 6,064 instances from 12 datasets, scored by three metrics. Its central finding is that most LLM-evolved heuristics are specialists that perform well only on instances drawn from the same distribution they were trained on and do not generalise across the full benchmark suite. On the primary metric, average excess bins over a lower bound, the hand-designed Best-Fit heuristic ranks first overall, with the LLM heuristic FS1 a close second. The authors argue that the considerable cost of evolving heuristics with LLMs should be weighed against this narrow coverage.

What carries the argument

The evaluation is carried by a portfolio of ten deterministic heuristics: five classic online bin-packing rules (Next-Fit, First-Fit, Best-Fit, Worst-Fit, Almost-Worst-Fit) and five LLM-evolved ones (FS1, FS2, FSW from FunSearch; EoC and EoH from the Evolution of Heuristics method). Three metrics are applied consistently across all instances: Average Excess Bins (AEB), computed as the mean percentage of bins above the lower bound $\sum_i s_i / C$; the Falkanauer fitness; and the fraction of instances won. The specialization story is drawn from Instance Space Analysis, which projects 209 time-series features into two dimensions, and from an evolutionary instance generator that produces 100 instances won by each heuristic; the gap between 'wins within its niche' and 'performance on the 800 instances it did not win' quantifies generality.

What would settle it

Run the same ten heuristics on freshly generated instances from distributions never seen in training (for example, log-normal or truncated exponential item sizes), or compute exact optimal bin counts for a subset of the smallest benchmarks, and compare; if an LLM-evolved heuristic beats Best-Fit on a majority of such datasets, the claim that most LLM heuristics do not generalise is false.

Watch

Extended reading notes

Core claim

The paper claims that when LLM-evolved heuristics for online bin packing are evaluated across a broad suite of 12 diverse datasets rather than the single distribution used during evolution, most of them fail to match the classic hand-designed heuristics they were meant to improve. On the average excess bins metric, Best-Fit has the lowest overall score; FS1, evolved by the FunSearch program on OR-Library data, ranks second and ties Best-Fit on several datasets. Two heuristics trained on Weibull-distributed data, FSW and EoH, sit at the bottom of the ranking, with FSW's excess bins roughly 20 times that of Best-Fit. The paper also reports that the code-only variant EoC outperforms EoH on the broad suite, even though the original EoH paper found the opposite on Weibull-only data, and that tuning the evolved heuristics' numeric constants with a standard configuration tool does not rescue their performance. An instance-space analysis and an evolutionary search for 'winning' instances show that the LLM heuristics occupy distinct niches in feature space, with FS1 and FS2 generalising well within their training distribution but not across distributions.

Load-bearing premise

The main ranking uses Average Excess Bins, which defines 'optimal' as the lower bound $\sum_i s_i / C$; if that bound is loose for some datasets, the reported excess-bins figures overstate how far each heuristic is from a truly optimal packing.

Editorial extensions

If this is right

If the claim holds, LLM-evolved heuristics should not be deployed blind: their expected advantage exists only when the target instance distribution is known in advance and matches the training distribution.
The reversal of EoC versus EoH on the broad suite warns that single-distribution leaderboards can flip when the test set broadens, so reported rankings in automated heuristic design should be treated as distribution-dependent.
The tuning experiment shows the evolved constants are not the bottleneck; the heuristics' structural logic is what limits them, so tuning alone will not turn a specialist into a generalist.
The instance-space maps provide a recipe for algorithm selection: after computing cheap features of an incoming instance, choose the heuristic that owns that region, potentially combining the best of specialist and generalist rules.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural next test is to run FS1 against Best-Fit on instances from other combinatorial domains, such as scheduling or cutting stock, to see whether its near-generalist behaviour on bin packing extends beyond one problem.
The ISA projection suggests a practical portfolio: a meta-heuristic could compute the selected time-series features online and switch to the region's winning heuristic, which would likely outperform any single LLM heuristic on the full benchmark.
Because the AEB lower bound can be loose, re-ranking the same heuristics with exact optima on small instances (like Scholl_1) would show whether Best-Fit's margin over FS1 is an artifact of the bound or a true gap.
The cost figures cited for the LLM search process imply that an economic case for LLM heuristic design needs a prior on how often a generalist like FS1 emerges; the paper's data suggest that is the exception, not the rule.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First, the useful news: this is the first benchmark that treats LLM-evolved heuristics for bin packing like actual algorithms. Ten heuristics, twelve datasets, 6,064 instances, three metrics, and the authors release their shuffled instances and code. They also exclude the distributions used during evolution, which is the obvious control that the original FunSearch and EoH papers skipped. The headline result—BF, a hand-designed heuristic from the 1970s, beats all five LLM heuristics on average, and most of the LLM heuristics are specialists that fail off-distribution—is credible and worth knowing.

The tuning study is a genuinely nice extra: it shows that the evolved numeric constants are not the reason these heuristics fail to transfer. Tuning gives small improvements where they were already good and small losses elsewhere. The instance-space analysis is well done, and the section that evolves new instances 'won' by each heuristic is explicitly and correctly framed as a stress test, not as evidence of superiority. That is the right way to use that tool.

The soft spot is the AEB metric. The paper defines 'optimal' as sum(item sizes)/C, the trivial cardinality lower bound. For bin-packing instances with large items relative to capacity, that bound is very loose, so the reported 'excess bins' are not true optimality gaps. Because the metric divides by that lower bound per instance, instances where the bound is loose get more weight in the average. The specific quantitative claims—'FSW has an AEB almost 20 times that of BF'—are not safe to cite. A recomputation against exact optima on a stratified sample would settle whether the ranking and the magnitudes survive. My guess is the ranking mostly survives, because BF also wins on the wins metric and the near-wins analysis in Table 4, both of which use actual bin counts. But the factor-20 number should not be quoted in its current form.

Minor: there are no statistical significance tests, though the effect sizes are large enough that this is not a threat to the conclusion.

Bottom line: this is a useful, honest empirical paper. The qualitative claim is well supported and is independent of the AEB metric. The quantitative part needs a fix or a caveat before it is citable. I would send it to peer review, and I would bring it to a reading group.

Referee Report

1 major / 5 minor

Summary. The paper benchmarks five LLM-evolved bin-packing heuristics (FS1, FS2, FSW, EoH, EoC) against five classical online heuristics (FF, BF, WF, NF, AWF) on 6,064 instances from 12 datasets, using three metrics: average excess bins (AEB), Falkenauer fitness, and a wins-based metric. The authors report that the hand-designed heuristic BF ranks first overall on AEB, that FS1 is the best LLM heuristic but does not clearly beat BF, and that most LLM-evolved heuristics are specialist and generalise poorly outside their training distributions. They further tune the evolved constants with irace and use instance-space analysis on evolved 'winning' instances to visualise the niche of each heuristic.

Significance. This is a timely and useful study: it provides the first large-scale, multi-dataset benchmark of LLM-evolved bin-packing heuristics, explicitly excludes datasets from the training distributions, and ships shuffled benchmark data in a public repository. The qualitative conclusion that most LLM-evolved heuristics are specialists rather than generalists is important for the automated heuristic-design community. The main quantitative ranking, however, relies on an AEB metric whose 'optimal' baseline is only a lower bound, so the headline magnitudes (e.g., the factor-20 gap between FSW and BF) need verification before they are cited.

major comments (1)

[Section 4, metric 1; Fig. 2d; Section 7] The AEB metric defines 'optimal' as LB = sum(size_i)/C (Section 4, metric 1). Since LB is only a lower bound, AEB is not a true optimality gap, and for instances with large item sizes it can be very loose (e.g., FSW has AEB values of 134.2 and 450.0 in Fig. 2a). Because the aggregate AEB sums per-instance percentages divided by LB, instances with a loose bound are overweighted in the total, so the aggregate ranking could change if the baseline were the true optimum or a tighter bound. I ask the authors to (i) rename the metric as 'excess over a lower bound' and soften the 'optimal' language, (ii) recompute the ranking on a stratified sample using exact optima or stronger lower bounds (e.g., L2/L3 or dual feasible functions) and report whether BF remains rank 1 and whether the 'almost 20 times' statement for FSW vs BF survives, and (iii) report a baseline-free aggregate such as mean or total bin count. The per-instance ordering by AEB equals the per-instance ordering by bin count, so the qualitative direction is probably robust, but the aggregate weighting and reported magnitudes are not.

minor comments (5)

[Section 4, metric 2] The Falkenauer fitness formula is incomplete because the exponent k is never defined; please specify k (usually 2) and state whether empty bins are included in the average, otherwise the metric is not reproducible.
[Section 4, metric 3; Fig. 2b] Since a 'win' is defined as being better or equal to the best value, the columns of Fig. 2b can sum to well over 1 (e.g., the Angelopoulos column sums to roughly 2.5). Please call this a 'not-worse-than-best' rate or additionally report exclusive wins, especially because Section 6 uses 'won' in the exclusive sense.
[Section 6; Table 4] The evolved instances are generated specifically for each heuristic to win, so the ISA projections and the Table 4 fractions describe synthetic niches rather than general population performance; the paper acknowledges this only in part, and an explicit warning in the Section 6 opening or in the caption would prevent misinterpretation.
[Section 3.2; Table 1] The 'Randomly Generated' dataset is described only by ranges for capacity and item count, with no generator or citation; please provide the exact generation procedure or a reference.
[Section 5] For EoC, the tuning-by-enumeration step is mentioned but the size of the enumeration search space is not given; please state the number of configurations considered.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central benchmarking claim rests on independent datasets and external heuristics; the instance-evolution section is explicitly separated from the ranking.

full rationale

The paper's central claim that most LLM-evolved heuristics do not generalise across a broad benchmark suite is derived from applying the five LLM heuristics and five hand-designed heuristics to 6,064 instances from 12 external datasets. The heuristics themselves are taken as artifacts from prior work [17,25], and the paper even excludes two OR-Library datasets whose generating distributions match the training data, so the evaluation is not defined in terms of the quantity it claims to establish. The AEB metric uses a cardinality lower bound as 'optimal'; this is a correctness/robustness concern about the magnitude of the reported gaps, not a circularity, because the bound is applied identically to all heuristics and the ranking is not constructed from any fitted parameter. Section 6 evolves instances that each heuristic wins, which is tautological by design, but the paper never uses those evolved instances to support the generalization ranking; it explicitly states 'these results contrast to the heuristic ranking obtained using the independent benchmark datasets (Figure 2d)' and uses the evolved instances only for instance-space visualization and niche analysis. The only self-citation is the instance-evolution EA from Alissa et al. [2], a methodological tool that is not load-bearing for the headline result. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported, and no known result is repackaged as a new contribution. The central derivation chain is therefore self-contained and not circular, though interested readers should treat the lower-bound-based AEB as a quantitative caveat rather than a circular flaw.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper rests on standard benchmarking assumptions rather than fitted parameters or new entities. The main premises are dataset representativeness, the AEB lower-bound metric, and faithful implementation of the compared heuristics. The ISA step introduces a methodology with feature-selection choices, but it is auxiliary to the central claim.

assumptions (4)

domain assumption The benchmark suite is representative of online bin packing instances
Section 3.2 selects 12 datasets from BPPLib, OR-Library, and Angelopoulos; the diversity supports the generalization claim, but representativeness of the full problem space is assumed.
domain assumption The lower bound (sum item sizes divided by capacity) is an appropriate baseline for 'optimal' in the AEB metric
Section 4 defines AEB using this lower bound; the metric is standard in the bin-packing literature but may be loose for some datasets.
domain assumption Implementations of the LLM heuristics from [25] and [17] are faithful
Section 3.1 states the authors use the notebook from [25] and code from [17]; validation on training distributions (Table 3) supports correctness, but subtle version mismatches are possible.
standard math Instance Space Analysis projection preserves meaningful structure
Section 6 applies the ISA methodology of [28] with tsfresh features and recursive feature elimination; the projection is assumed to reflect true performance landscapes.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Beyond the Hype: Benchmarking LLM-Evolved Heuristics for Bin Packing." pith.science (2026). https://pith.science/paper/ZOCFD5U6

@misc{pith2026250111411,
  author       = {Pith},
  title        = {Pith review of: Beyond the Hype: Benchmarking LLM-Evolved Heuristics for Bin Packing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZOCFD5U6}},
  note         = {Machine review of arXiv:2501.11411}
}

read the original abstract

Coupling Large Language Models (LLMs) with Evolutionary Algorithms has recently shown significant promise as a technique to design new heuristics that outperform existing methods, particularly in the field of combinatorial optimisation. An escalating arms race is both rapidly producing new heuristics and improving the efficiency of the processes evolving them. However, driven by the desire to quickly demonstrate the superiority of new approaches, evaluation of the new heuristics produced for a specific domain is often cursory: testing on very few datasets in which instances all belong to a specific class from the domain, and on few instances per class. Taking bin-packing as an example, to the best of our knowledge we conduct the first rigorous benchmarking study of new LLM-generated heuristics, comparing them to well-known existing heuristics across a large suite of benchmark instances using three performance metrics. For each heuristic, we then evolve new instances won by the heuristic and perform an instance space analysis to understand where in the feature space each heuristic performs well. We show that most of the LLM heuristics do not generalise well when evaluated across a broad range of benchmarks in contrast to existing simple heuristics, and suggest that any gains from generating very specialist heuristics that only work in small areas of the instance space need to be weighed carefully against the considerable cost of generating these heuristics.

Figures

Figures reproduced from arXiv: 2501.11411 by the authors.

**Figure 2.** (a-c): Heatmaps showing heuristic performance per dataset for 3 metrics: [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

**Figure 3.** Boxplots showing aggregated heuristic performance over all dataset for [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Perspectives on the ISA projection of the instance space: (a) instances [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

BLADE: Benchmark suite for LLM-driven Automated Design and Evolution of iterative optimisation heuristics
cs.SE 2025-04 conditional novelty 6.0 of 10

BLADE is an open-source, modular benchmark suite for evaluating LLM-driven automated algorithm discovery on continuous black-box optimization problems.
Improving Existing Optimization Algorithms with LLMs
cs.AI 2025-02 conditional novelty 5.0 of 10

A GPT-4o-proposed age-aware heuristic for the CMSA metaheuristic outperforms the expert degree-based heuristic on Maximum Independent Set instances across three graph families.

Reference graph

Works this paper leans on

35 extracted references · 28 canonical work pages · cited by 2 Pith papers

[1]

https://site.unibo.it/ operations-research/en/research/bpplib-a-bin-packing-problem-library

BPPLIB – A bin packing problem library. https://site.unibo.it/ operations-research/en/research/bpplib-a-bin-packing-problem-library

work page
[2]

In: Proceedings of the Genetic and Evolutionary Computation Conference

Alissa, M., Sim, K., Hart, E.: Algorithm selection using deep learning without feature extraction. In: Proceedings of the Genetic and Evolutionary Computation Conference. pp. 198–206 (2019)

work page 2019
[3]

Journal of Artificial Intelligence Research78, 1111–1141 (2023)

Angelopoulos, S., Kamali, S., Shadkami, K.: Online bin packing with predictions. Journal of Artificial Intelligence Research78, 1111–1141 (2023)

work page 2023
[4]

https:// github.com/shahink84/BinPackingPredictions/tree/main/Data/Benchmarks (2024)

Angelopoulos, S., Kamali, S., Shadkami, K.: Binpackingpredictions. https:// github.com/shahink84/BinPackingPredictions/tree/main/Data/Benchmarks (2024)

work page 2024
[5]

Anonymous: Beyond the Hype: Benchmarking LLM-Evolved Heuristics for Bin Packing - Data (Nov 2024).https://doi.org/10.5281/zenodo.14162744, https: //doi.org/10.5281/zenodo.14162744

work page doi:10.5281/zenodo.14162744 2024
[6]

Journal of the operational research society41(11), 1069–1072 (1990)

Beasley, J.E.: Or-library: distributing test problems by electronic mail. Journal of the operational research society41(11), 1069–1072 (1990)

work page 1990
[7]

European Journal of Operational Research 290(2), 405–421 (2021)

Bengio, Y., Lodi, A., Prouvost, A.: Machine learning for combinatorial optimiza- tion: a methodological tour d’horizon. European Journal of Operational Research 290(2), 405–421 (2021)

work page 2021
[8]

Journal of the Operational Research Society 64(12), 1695–1724 (2013)

Burke, E.K., Gendreau, M., Hyde, M., Kendall, G., Ochoa, G., Özcan, E., Qu, R.: Hyper-heuristics: A survey of the state of the art. Journal of the Operational Research Society 64(12), 1695–1724 (2013)

work page 2013

Show all 35 references

[9]

Evolutionary computation 20(1), 63–89 (2012)

Burke, E.K., Hyde, M.R., Kendall, G., Woodward, J.: Automating the packing heuristic design process with genetic programming. Evolutionary computation 20(1), 63–89 (2012)

2012
[10]

In: International Conference on Principles and Practice of Constraint Programming

Castiñeiras, I., De Cauwer, M., O’Sullivan, B.: Weibull-based benchmarks for bin packing. In: International Conference on Principles and Practice of Constraint Programming. pp. 207–222. Springer (2012)

2012
[11]

European Journal of Operational Research 255(1), 1–20 (2016)

Delorme, M., Iori, M., Martello, S.: Bin packing and cutting stock problems: Math- ematical models and exact algorithms. European Journal of Operational Research 255(1), 1–20 (2016)

2016
[12]

In: ICRA

Falkenauer, E., Delchambre, A., et al.: A genetic algorithm for bin packing and line balancing. In: ICRA. pp. 1186–1192. Citeseer (1992)

1992
[13]

INFORMS Journal on Computing28(1), 175–194 (2016)

Gschwind, T., Irnich, S.: Dual inequalities for stabilized column generation revis- ited. INFORMS Journal on Computing28(1), 175–194 (2016)

2016
[14]

Optimiza- tion Methods and Software 36(1), 114–144 (2021)

Hansen, N., Auger, A., Ros, R., Mersmann, O., Tusar, T., Brockhoff, D.: COCO: a platform for comparing continuous optimizers in a black-box setting. Optimiza- tion Methods and Software 36(1), 114–144 (2021). https://doi.org/10.1080/ 10556788.2020.1808977

2021
[15]

Evolutionary Computation9(2), 159–195 (2001).https://doi.org/10

Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation9(2), 159–195 (2001).https://doi.org/10. 1162/106365601750190398

2001
[16]

Johnson, D.S.: Near-optimal bin packing algorithms. Ph.D. thesis, Massachusetts Institute of Technology (1973) 16 Sim et. al

1973
[17]

In: Forty-first International Conference on Machine Learning (2024)

Liu, F., Xialiang, T., Yuan, M., Lin, X., Luo, F., Wang, Z., Lu, Z., Zhang, Q.: Evo- lution of heuristics: Towards efficient automatic algorithm design using large lan- guage model. In: Forty-first International Conference on Machine Learning (2024)

2024
[18]

Opera- tions Research Perspectives3, 43 – 58 (2016)

López-Ibáñez, M., Dubois-Lacoste, J., Pérez Cáceres, L., Birattari, M., Stützl, T.: The irace package: Iterated racing for automatic algorithm configuration. Opera- tions Research Perspectives3, 43 – 58 (2016)

2016
[19]

European Journal of Operational Research (2024)

Martín-Santamaría, R., López-Ibáñez, M., Stützle, T., Colmenar, J.M.: On the automatic generation of metaheuristic algorithms for combinatorial optimization problems. European Journal of Operational Research (2024)

2024
[20]

In: European Conference on Evolution- ary Computation in Combinatorial Optimization (Part of EvoStar)

McMenemy, P., Veerapen, N., Adair, J., Ochoa, G.: Rigorous performance analysis of state-of-the-art tsp heuristic solvers. In: European Conference on Evolution- ary Computation in Combinatorial Optimization (Part of EvoStar). pp. 99–114. Springer (2019)

2019
[21]

Journal of Machine Learning Research12, 2825–2830 (2011)

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine learning in Python. Journal of Mach...

2011
[22]

In: ProceedingsoftheGeneticandEvolutionaryComputationConferenceCompanion

Pluhacek, M., Kovac, J., Viktorin, A., Janku, P., Kadavy, T., Senkerik, R.: Using llm for automatic evolvement of metaheuristics from swarm algorithm soma. In: ProceedingsoftheGeneticandEvolutionaryComputationConferenceCompanion. pp. 2018–2022 (2024)

2024
[23]

In: Proceedings of the 2014 annual conference on genetic and evolutionary computa- tion

Polyakovskiy, S., Bonyadi, M.R., Wagner, M., Michalewicz, Z., Neumann, F.: A comprehensive benchmark set and heuristics for the traveling thief problem. In: Proceedings of the 2014 annual conference on genetic and evolutionary computa- tion. pp. 477–484 (2014)

2014
[24]

Chapman and Hall/CRC (2008)

Rinne, H.: The Weibull distribution: a handbook. Chapman and Hall/CRC (2008)

2008
[25]

Nature625(7995), 468–475 (2024)

Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M.P., Dupont, E., Ruiz, F.J., Ellenberg, J.S., Wang, P., Fawzi, O., et al.: Mathematical discoveries from program search with large language models. Nature625(7995), 468–475 (2024)

2024
[26]

IEEE Access8, 128068–128095 (2020)

Sánchez, M., Cruz-Duarte, J.M., carlos Ortíz-Bayliss, J., Ceballos, H., Terashima- Marin, H., Amaya, I.: A systematic review of hyper-heuristics on combinatorial optimization problems. IEEE Access8, 128068–128095 (2020)

2020
[27]

Computers & Operations Research 24(7), 627–645 (1997)

Scholl, A., Klein, R., Jürgens, C.: Bison: A fast hybrid procedure for exactly solv- ing the one-dimensional bin packing problem. Computers & Operations Research 24(7), 627–645 (1997)

1997
[28]

ACM Computing Surveys55(12), 1–31 (2023)

Smith-Miles, K., Muñoz, M.A.: Instance space analysis for algorithm testing: Methodology and software tools. ACM Computing Surveys55(12), 1–31 (2023)

2023
[29]

arXiv preprint arXiv:2405.20132 (2024)

van Stein, N., Bäck, T.: Llamea: A large language model evolutionary algorithm for automatically generating metaheuristics. arXiv preprint arXiv:2405.20132 (2024)

2024 arXiv
[30]

arXiv preprint arXiv:2410.16309 (2024)

van Stein, N., Vermetten, D., Bäck, T.: In-the-loop hyper-parameter optimiza- tion for llm-based automated design of heuristics. arXiv preprint arXiv:2410.16309 (2024)

2024 arXiv
[31]

Journal of Global Optimization11(4), 341–359 (1997)

Storn, R., Price, K.: Differential evolution - A simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization11(4), 341–359 (1997). https://doi.org/10.1023/A:1008202821328

1997 doi
[32]

Applied Soft Computing130, 109696 (2022) Beyond the Hype: Benchmarking LLM-Evolved Heuristics for Bin Packing 17

Ðurasević, M., Ðumić, M.: Automated design of heuristics for the container relo- cation problem using genetic programming. Applied Soft Computing130, 109696 (2022) Beyond the Hype: Benchmarking LLM-Evolved Heuristics for Bin Packing 17

2022
[33]

Operations-Research-Spektrum 18, 131–144 (1996)

Wäscher, G., Gau, T.: Heuristics for the integer one-dimensional cutting stock problem: A computational study. Operations-Research-Spektrum 18, 131–144 (1996)

1996
[34]

arXiv preprint arXiv:2402.01145 (2024)

Ye, H., Wang, J., Cao, Z., Berto, F., Hua, C., Kim, H., Park, J., Song, G.: Reevo: Large language models as hyper-heuristics with reflective evolution. arXiv preprint arXiv:2402.01145 (2024)

2024 arXiv
[35]

In: International Conference on Parallel Problem Solving from Nature

Zhang, R., Liu, F., Lin, X., Wang, Z., Lu, Z., Zhang, Q.: Understanding the im- portance of evolutionary search in automated heuristic design with large language models. In: International Conference on Parallel Problem Solving from Nature. pp. 185–202. Springer (2024)

2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

https://site.unibo.it/ operations-research/en/research/bpplib-a-bin-packing-problem-library

BPPLIB – A bin packing problem library. https://site.unibo.it/ operations-research/en/research/bpplib-a-bin-packing-problem-library

work page

[2] [2]

In: Proceedings of the Genetic and Evolutionary Computation Conference

Alissa, M., Sim, K., Hart, E.: Algorithm selection using deep learning without feature extraction. In: Proceedings of the Genetic and Evolutionary Computation Conference. pp. 198–206 (2019)

work page 2019

[3] [3]

Journal of Artificial Intelligence Research78, 1111–1141 (2023)

Angelopoulos, S., Kamali, S., Shadkami, K.: Online bin packing with predictions. Journal of Artificial Intelligence Research78, 1111–1141 (2023)

work page 2023

[4] [4]

https:// github.com/shahink84/BinPackingPredictions/tree/main/Data/Benchmarks (2024)

Angelopoulos, S., Kamali, S., Shadkami, K.: Binpackingpredictions. https:// github.com/shahink84/BinPackingPredictions/tree/main/Data/Benchmarks (2024)

work page 2024

[5] [5]

Anonymous: Beyond the Hype: Benchmarking LLM-Evolved Heuristics for Bin Packing - Data (Nov 2024).https://doi.org/10.5281/zenodo.14162744, https: //doi.org/10.5281/zenodo.14162744

work page doi:10.5281/zenodo.14162744 2024

[6] [6]

Journal of the operational research society41(11), 1069–1072 (1990)

Beasley, J.E.: Or-library: distributing test problems by electronic mail. Journal of the operational research society41(11), 1069–1072 (1990)

work page 1990

[7] [7]

European Journal of Operational Research 290(2), 405–421 (2021)

Bengio, Y., Lodi, A., Prouvost, A.: Machine learning for combinatorial optimiza- tion: a methodological tour d’horizon. European Journal of Operational Research 290(2), 405–421 (2021)

work page 2021

[8] [8]

Journal of the Operational Research Society 64(12), 1695–1724 (2013)

Burke, E.K., Gendreau, M., Hyde, M., Kendall, G., Ochoa, G., Özcan, E., Qu, R.: Hyper-heuristics: A survey of the state of the art. Journal of the Operational Research Society 64(12), 1695–1724 (2013)

work page 2013