Pith. sign in

REVIEW 2 major objections 6 minor 45 references

LLM bin-packing heuristics reduce to two simple thresholds, undercutting their claimed mathematical novelty.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

The LLM-generated bin packing heuristics from Nature's FunSearch paper reduce to simple two-parameter threshold rules and don't constitute a mathematical discovery.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A careful, reproducible deflation of FunSearch's bin-packing 'discovery' claim; the core conclusion holds, with minor caveats about fitted parameters and the approximate c14 interpretation. the 2 major comments →

arxiv 2510.27353 v2 pith:23FKHEZH submitted 2025-10-31 cs.AI

An In-depth Study of LLM Contributions to the Bin Packing Problem

Julien Herrmann , Guillaume Pallez This is my paper
classification cs.AI MSC 68W2790C27
keywords bin packingonline algorithmslarge language modelsevolutionary searchheuristic interpretabilityprogram searchstochastic optimizationmathematical discovery
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reassesses the claim that LLM-based program search produced mathematical discoveries by solving online bin packing. Through behavioral analysis of heuristics evolved for the Uniform(20,100) and Weibull(3.0,45) instances, the authors show that, despite human-readable code, the heuristics are opaque and resist expert interpretation. All of them can be approximated and outperformed by a simple two-parameter priority rule: fit tightly if a bin leaves little slack, otherwise use a baseline like WorstFit on loose bins, else open a new bin. The authors conclude that the LLM frameworks discovered effective strategies through stochastic search and empirical feedback, not conceptual understanding, and that the instances were previously unstudied, so the 'mathematical discovery' label is misleading. This matters because it establishes a concrete bar for validating AI-generated heuristics: distill them into a compact, parameterized rule and test generalization.

Core claim

The paper claims that heuristics produced by LLM-based evolutionary search for online bin packing do not amount to conceptual or mathematical contributions. On the trained distributions, the behaviors of evolved heuristics — including priority functions c12, c14, and a later variant — collapse to a single principle: place an item in the tightest bin if the leftover space is within a threshold a; otherwise place it in a loose bin (leftover beyond b) using a standard rule; otherwise open a new bin. The authors instantiate this as ab-FirstFit, ab-BestFit, and ab-WorstFit, and show that with two tuned parameters these outperform the LLM heuristics and all baselines on the training distributions

What carries the argument

The central object is the ab-heuristic family, a two-parameter priority rule: parameter a defines a 'tight fit' zone (use BestFit among bins whose residual space after placing item size s is ≤ s+a), parameter b defines a forbidden gap (bins with residual in (s+a, s+b] are skipped), and a baseline strategy (FirstFit, BestFit, or WorstFit) acts on bins with residual > s+b; if no such bin exists, a new bin opens. This family distills the opaque LLM-evolved heuristics: c12 approximates ab-FirstFit with a=5, b=24 on Uniform(20,100), and c14/EoH approximate ab-WorstFit with a≈1, b≈21 on Weibull(3.0,45). The key work of the ab-family is converting an unreadable score function into a testable, trans

Load-bearing premise

The paper's core conclusion depends on its interpretation of c14's opaque score function as equivalent to an ab-WorstFit threshold rule — an approximation the authors themselves state differs non-negligibly from what c14 actually does.

What would settle it

Run c14 side-by-side with ab-WorstFit(a=1, b=21) on a large hold-out sample from Weibull(3.0,45), recording every decision where they diverge (different bin chosen, or one opens a new bin while the other does not). If the divergence rate is high, or if any alternative simple rule explains c14's decisions better than the ab-model, the paper's reduction claim is weakened; conversely, if divergence is rare and bin counts stay within noise, the conclusion is supported.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • The LLM-generated heuristics for the considered bin-packing instances are captured by a two-threshold principle, so their apparent novelty is a reparameterization of familiar strategies rather than new algorithmic concepts.
  • The ab-Baselines heuristic matches or outperforms the LLM-evolved heuristics on the distributions they were trained on and generalizes to other uniform and Weibull instances with a lower bound on item sizes.
  • The paper derives a worst-case gap of c/(c−b) for ab-Baselines when many arriving items fall below the b threshold, explaining the regime in which this style of heuristic degrades.
  • A laptop-based grid search over two parameters accomplishes in minutes what required millions of LLM queries and days of evolutionary search, with better or equal performance.
  • Claims of mathematical discovery from LLM program search need validation against well-established problems with prior literature and simple parameterized baselines; the bin-packing case fails that test.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The distillation strategy used here — approximating an opaque evolved program with a compact parameterized rule and then re-searching the parameter space — could serve as a general post-processing audit step for LLM-evolved heuristics in other combinatorial optimization domains.
  • The two-threshold principle may be specific to distributions with a positive minimum item size and large item counts; a testable extension is to apply ab-Baselines to discretized or bounded distributions with small minimum sizes to map exactly when the tight-fit/loose-bin dichotomy stops paying off.
  • The paper's analysis implies a cheap falsification protocol for future 'AI discovery' claims: run the same evolutionary setup while simply enumerating parameterized variants of known heuristics, and compare — this would make assertions of insight empirically testable.
  • If the pattern generalizes, some celebrated LLM-search results may be explainable as stochastic optimization over a low-dimensional manifold of known algorithm families, tempering expectations for LLM-driven theoretical progress.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. This paper reassesses the claim that LLM-based program search (FunSearch) produced mathematically interesting discoveries for online bin packing. The authors analyze the c12 heuristic for Uniform(20,100) and the c14/EoH heuristics for Weibull(3,45), arguing that these human-readable but opaque programs reduce to two-parameter threshold policies. They introduce ab-FirstFit, ab-WorstFit, and ab-BestFit, and report that on the original distributions these simple heuristics match or slightly exceed the LLM-evolved heuristics (ab-FirstFit 2.1% improvement over BestFit vs c12's 2.0%; ab-WorstFit 3.3% vs c14's 3.3%). The paper concludes that the LLM frameworks did not develop conceptual understanding of bin packing but rediscovered known strategies through stochastic search.

Significance. If the conclusions hold, this is a valuable, sober corrective to high-profile claims about LLM-driven mathematical discovery. The paper's strengths include released code, clearly specified experimental protocols (100-1000 instances, boxplots, heatmaps), a nearly exact simplification of c12, and a concrete, interpretable, two-parameter algorithmic baseline that is orders of magnitude cheaper to produce than the LLM-evolved heuristics. The paper is also candid about the limits of its c14 interpretation. However, two issues in the present version need attention before the stronger claims (generalization, outperformance, 'largely rediscover') are fully supported: the in-sample selection of the ab parameters and the absence of a quantitative fidelity measure for the c14-to-ab-WorstFit approximation.

major comments (2)
  1. [§4.3, Fig. 11] The parameters a,b of ab-FirstFit/ab-WorstFit/ab-BestFit are 'selected separately for each problem instance,' and the heatmaps in Figs. 9-10 appear to use the same 100 instances on which performance is reported. This is in-sample tuning: the ab heuristics effectively see the evaluation set, while c12/c14/EoH are fixed. The claim that 'ab-Baselines outperforms all baselines and LLMs-evolved heuristics,' as well as the word 'generalizable' in the abstract, needs a held-out protocol. Please tune on one set (or use cross-validation) and report results on a disjoint test set, and state explicitly how the parameters in Fig. 11 were chosen.
  2. [§3.3.1, Figs. 5-6] The proposed interpretation of c14 (BestFit on perfect fit, else WorstFit if residual exceeds item size by more than 20, else open a new bin) is admitted to 'differ non negligibly' from c14's actual behavior. Since this interpretation is the main evidence that c14 'largely' rediscovers a known threshold pattern, the paper should quantify its fidelity: for example, the fraction of decisions where c14 and ab-WorstFit choose the same bin, split by the three cases, and a performance comparison between c14 and ab-WorstFit with tuned parameters on held-out instances. Without this, the reader cannot judge how much of c14's behavior is captured, and the 'largely rediscover' claim remains qualitative.
minor comments (6)
  1. [§4.3, Fig. 10] The text reports the best ab-WorstFit parameters as a=1, b=22, while the Figure 10 caption says b=21. Reconcile.
  2. [Algorithm 6] Line 8 uses an undefined variable x; it should be bin or bins. The function declaration also contains the typo 'ab-Worst!rray'.
  3. [Algorithms 4-6] capacity is used in the priority functions but is not passed as an argument; define it or note that it is a global constant.
  4. [§4.3] The sentence 'the performance can degrade to as much as c/(c-b) times the optimal 1' contains a stray '1' and the derivation is only a sketch. State the formal claim and proof more carefully.
  5. [§1 and §2] The introduction says the Nature paper has been cited by more than 800 papers as of October 31st, while §2 says 744 as of October 1st. The discrepancy should be explained or harmonized.
  6. [§2] The 'convenience sample biased toward the most cited' of the remaining 653 papers is not described. If retained, give sampling details or label it as an informal observation.

Circularity Check

0 steps flagged

No significant circularity: the reduction claim is supported by direct code analysis and external benchmarks, with disclosed fitting rather than hidden prediction.

full rationale

The paper's central claim—that FunSearch's bin-packing heuristics largely rediscover simple threshold policies and do not amount to mathematical discovery—is not circular. The c12 analysis is a direct code-level simplification: the priority function in Algorithm 1 is literally a step function that selects bins leaving <=7 residual capacity, ignores bins leaving 7-21, and otherwise applies FirstFit; this is an exact reading, not a fitted construct. The c14 interpretation is explicitly approximate and disclosed: the paper states that its proposed 'BestFit if perfect-fit exists; otherwise WorstFit if residual exceeds item size by more than 20; otherwise open a new bin' interpretation 'differs non negligibly from the actual behavior of c14', so the reduction is a fitted behavioral summary rather than an identity. The ab-Baselines parameters are selected separately per distribution, and the paper does not present these in-sample comparisons as out-of-sample predictions; the robustness of the parameter choice is shown via heatmaps. The conclusion also draws on independent external evidence: EoH's score function visually encodes the same threshold principle, Sim et al.'s benchmarking shows poor generalization of LLM-evolved heuristics, and no follow-up work builds on the discovered heuristics. There are no load-bearing self-citations; all key compared heuristics and benchmarks come from other groups. The closest concern is that the c14-to-ab-WorstFit interpretation is fitted to observed behavior and not independently predictive, but the paper concedes this divergence and the central negative claim does not hinge on exact decision-level equivalence. Thus no circular derivation chain is present; only minor methodological caveats that are disclosed rather than concealed.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 0 invented entities

The paper introduces no new theoretical entities; its new artifacts are algorithms. The load-bearing assumptions are the empirical hypotheses H1/H2, the no-prior-work literature claim, and the fidelity of the c14 threshold approximation. The free parameters are the thresholds a and b (per-distribution fitted) and the ~20 value used in the c14 interpretation.

free parameters (3)
  • a (tight-fit threshold in ab-heuristics) = 5 for Uniform(20,100); 1 for Weibull(3.0,45)
    The ab-FirstFit/WorstFit/BestFit priority functions use threshold a to decide when a bin is 'tight' after placement (bin <= item + a). The paper selects a per distribution in its performance evaluation (Figure 11).
  • b (loose-fit threshold in ab-heuristics) = 24 for Uniform(20,100); 21-22 for Weibull(3.0,45)
    Threshold b controls when a bin is 'too empty' (bin <= item + b yields priority 0, causing new-bin opening). Selected per distribution via grid search over (a,b) (Figures 9, 10).
  • c14 interpretation threshold (~20) = approximately 20
    In Section 3.3.1, the paper infers from Figure 6 that c14 opens a new bin when WorstFit's chosen bin would leave remaining space less than item+20; used to build the interpretable approximation of c14.
axioms (3)
  • domain assumption The performance advantage of the LLM heuristics is explained by hypotheses H1 (large number of items) and H2 (minimum item size imposes a lower bound).
    Section 3.2 and 4.2. The paper supports this with experimental curves (Figures 3, 12, 13) but does not prove it; it is load-bearing for the claim that the instances are simple.
  • domain assumption There is no prior published work on online bin packing with uniform distributions lower-bounded by a>0 or the specific Weibull(3.0,45) distribution.
    Section 2 last paragraph: 'To the best of our knowledge, there is no prior research...' and Section 4.4: 'one cannot entirely rule out the possibility that relevant work does exist'. This assumption supports the critique that FunSearch did not solve an established problem.
  • domain assumption The score functions of the LLM heuristics can be meaningfully approximated by interpretable threshold rules.
    The paper's method approximates c14 by a rule and admits it differs non-negligibly; the 'rediscovery' conclusion depends on this approximation being faithful.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of An In-depth Study of LLM Contributions to the Bin Packing Problem." pith.science (2026). https://pith.science/paper/23FKHEZH

@misc{pith2026251027353,
  author       = {Pith},
  title        = {Pith review of: An In-depth Study of LLM Contributions to the Bin Packing Problem},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/23FKHEZH}},
  note         = {Machine review of arXiv:2510.27353}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Recent studies have suggested that Large Language Models (LLMs) could provide interesting ideas contributing to mathematical discovery. This claim was motivated by reports that LLM-based genetic algorithms produced heuristics offering new insights into the online bin packing problem under uniform and Weibull distributions. In this work, we reassess this claim through a detailed analysis of the heuristics produced by LLMs, examining both their behavior and interpretability. Despite being human-readable, these heuristics remain largely opaque even to domain experts. Building on this analysis, we propose a new class of algorithms tailored to these specific bin packing instances. The derived algorithms are significantly simpler, more efficient, more interpretable, and more generalizable, suggesting that the considered instances are themselves relatively simple. We then discuss the limitations of the claim regarding LLMs' contribution to this problem, which appears to rest on the mistaken assumption that the instances had previously been studied. Our findings instead emphasize the need for rigorous validation and contextualization when assessing the scientific value of LLM-generated outputs.

Figures

Figures reproduced from arXiv: 2510.27353 by Guillaume Pallez, Julien Herrmann.

Figure 1
Figure 1. Figure 1: Priority provided by c12. The scale of the y-axis is not linear to highlight the difference of pri￾ority. performance, relative to BestFit in terms of the num￾ber of bins used, is similar (see [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: Average performance of c12 relative to BestFit over 1000 instances of the Uniform(20, 100) distribution, as the number of items increases (bin capacity = 150). 3.3 Item sizes following a Weibull(3.0, 45) distribution with bin capacity 100 We now analyze the heuristics presented in Romera et al. [6] and Liu et al. [30] for the Weibull(3.0, 45) dis￾tribution. The instances consist of items with integer sizes… view at source ↗
Figure 2
Figure 2. Figure 2: Boxplots showing the heuristics perfor￾mance relative to BestFit over 1000 instances of the Uniform(20, 100) distribution with 500 items and bin capacity 150. We hypothesize that the average 2.0% gain of c12 and Smooth c12 over BestFit is due to several com￾bined factors: • (H1) The large number of items to be scheduled. We hypothesize that the perfor￾mance gain arises when many items must be scheduled. Th… view at source ↗
Figure 4
Figure 4. Figure 4: Plotting f : s, b 7→ score(s, b), where score is the function from line 2 of c14. In these plots we use: c = 100, s1 = 20, s2 = 50. There is an inflexion point at c 1+ 1 s + 1 s2 . The score function defined on line 2 depends on the item size s, the maximum bin capacity c, and the current remaining space b in each bin to determine each bin priority, with the formula: f : (s, b) 7→ (b − c) 2 s + b 2 s 2 + b… view at source ↗
Figure 5
Figure 5. Figure 5: Comparing the behavior item-per-items of [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Remaining size in the bins used by WorstFit after fitting the item in a non-empty bin on the Weibull(3.0, 45) distribution with 50k items and bin capacity 100. perspective: in practice, when WorstFit would sched￾ule an item into a bin whose remaining space is less than the item size plus a certain threshold (approxi￾mately 20), c14 instead opens a new bin. Finally, based on all these evaluations, we propos… view at source ↗
Figure 8
Figure 8. Figure 8: provides a visual representation of this func￾tion. One can notice that, even if the priority func￾tion is different, the underlying principle it captures remains essentially the same. s1 4 · s1 s2 c fEOH(s1, b) fEOH(s2, b) Remaining bin capacity [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: Heatmap showing the relative number of bins used by ab-WorstFit for various values of a and b, compared to BestFit, over 100 instances of Weibull(3.0, 45) (bin capacity = 100, number of items = 5000). The best results are obtained for a = 1 and b = 21, with a relative ratio of 0.967. Weibull(3.0, 45) [PITH_FULL_IMAGE:figures/full_fig_p010_10.png] view at source ↗
Figure 9
Figure 9. Figure 9: Heatmap showing the relative number of bins used by ab-FirstFit for various values of a and b, compared to BestFit, over 100 instances of Uniform(20, 100) (bin capacity = 150, number of items = 500). The best results are obtained for a = 5 and b = 24, with a relative ratio of 0.979. Uniform(20, 100) [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗
Figure 12
Figure 12. Figure 12: Average performance of ab-FirstFit (with a = 5 and b = 24) relative to BestFit over 1000 instances of the Uniform(20, 100) distribution, as the number of items increases (bin capacity = 150) [PITH_FULL_IMAGE:figures/full_fig_p011_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Average performance of ab-WorstFit (with a = 1 and b = 21) relative to BestFit over 1000 instances of the Weibull(3.0, 45) distribution, as the number of items increases (bin capacity = 100). of work, we expect that performance could be further improved, from a computational complexity perspec￾tive, by employing dichotomic search for parameter tuning, as an example. 4.4 Concluding Discussions In their wor… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 6 linked inside Pith

  1. [1]

    Springer, 2018

    Nelishia Pillay and Rong Qu.Hyper-heuristics: theory and applications. Springer, 2018

    2018

  2. [2]

    Hyper-heuristics: A sur- vey of the state of the art.Journal of the Opera- tional Research Society, 64(12):1695–1724, 2013

    Edmund K Burke, Michel Gendreau, Matthew Hyde, Graham Kendall, Gabriela Ochoa, Ender ¨Ozcan, and Rong Qu. Hyper-heuristics: A sur- vey of the state of the art.Journal of the Opera- tional Research Society, 64(12):1695–1724, 2013

    2013

  3. [3]

    Au- tomated design of metaheuristic algorithms

    Thomas St¨ utzle and Manuel L´ opez-Ib´ a˜ nez. Au- tomated design of metaheuristic algorithms. InHandbook of metaheuristics, pages 541–579. Springer, 2018

    2018

  4. [4]

    Springer Science & Business Media, 2013

    William B Langdon and Riccardo Poli.Founda- tions of genetic programming. Springer Science & Business Media, 2013

    2013

  5. [5]

    Survey on genetic programming and machine learning techniques for heuristic design in job shop scheduling.IEEE Transac- tions on Evolutionary Computation, 28(1):147– 167, 2023

    Fangfang Zhang, Yi Mei, Su Nguyen, and Mengjie Zhang. Survey on genetic programming and machine learning techniques for heuristic design in job shop scheduling.IEEE Transac- tions on Evolutionary Computation, 28(1):147– 167, 2023

    2023

  6. [6]

    Mathematical discoveries from program search with large language mod- els.Nature, 625(7995):468–475, 2024

    Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. Mathematical discoveries from program search with large language mod- els.Nature, 625(7995):468–475, 2024

    2024

  7. [7]

    Machine learning for combina- torial optimization: a methodological tour d’horizon.European Journal of Operational Re- search, 290(2):405–421, 2021

    Yoshua Bengio, Andrea Lodi, and Antoine Prouvost. Machine learning for combina- torial optimization: a methodological tour d’horizon.European Journal of Operational Re- search, 290(2):405–421, 2021

    2021

  8. [8]

    Evolutionary compu- tation in the era of large language model: Survey and roadmap.IEEE Transactions on Evolution- ary Computation, 2024

    Xingyu Wu, Sheng-hao Wu, Jibin Wu, Liang Feng, and Kay Chen Tan. Evolutionary compu- tation in the era of large language model: Survey and roadmap.IEEE Transactions on Evolution- ary Computation, 2024

    2024

  9. [9]

    New applications of the poly- nomial method: the cap set conjecture and be- yond.Bulletin of the American Mathematical Society, 56(1):29–64, 2019

    Joshua Grochow. New applications of the poly- nomial method: the cap set conjecture and be- yond.Bulletin of the American Mathematical Society, 56(1):29–64, 2019

    2019

  10. [10]

    Cambridge University Press, 2006

    Terence Tao and Van H Vu.Additive combina- torics, volume 105. Cambridge University Press, 2006. 13

    2006

  11. [11]

    Approximation algorithms for bin-packing—an updated survey

    Edward G Coffman Jr, Michael R Garey, and David S Johnson. Approximation algorithms for bin-packing—an updated survey. InAlgorithm design for computer system design, pages 49–

  12. [12]

    Hyper-heuristics: An emerging direction in mod- ern search technology

    Edmund Burke, Graham Kendall, Jim Newall, Emma Hart, Peter Ross, and Sonia Schulenburg. Hyper-heuristics: An emerging direction in mod- ern search technology. InHandbook of meta- heuristics, pages 457–474. Springer, 2003

    2003

  13. [13]

    Hyper-heuristics: learning to combine simple heuristics in bin- packing problems

    Peter Ross, Sonia Schulenburg, Javier G Mar ´ ın- Bl¨ azquez, and Emma Hart. Hyper-heuristics: learning to combine simple heuristics in bin- packing problems. InProceedings of the 4th annual conference on genetic and evolutionary computation, pages 942–948, 2002

    2002

  14. [14]

    A lifelong learning hyper-heuristic method for bin packing.Evolutionary computation, 23(1):37–67, 2015

    Kevin Sim, Emma Hart, and Ben Paechter. A lifelong learning hyper-heuristic method for bin packing.Evolutionary computation, 23(1):37–67, 2015

    2015

  15. [15]

    Handbook of evolutionary compu- tation.Release, 97(1):B1, 1997

    Thomas B¨ ack, David B Fogel, and Zbigniew Michalewicz. Handbook of evolutionary compu- tation.Release, 97(1):B1, 1997

    1997

  16. [16]

    From evolu- tionary computation to the evolution of things

    Agoston E Eiben and Jim Smith. From evolu- tionary computation to the evolution of things. Nature, 521(7553):476–482, 2015

    2015

  17. [17]

    Explainable artificial in- telligence by genetic programming: A survey

    Yi Mei, Qi Chen, Andrew Lensen, Bing Xue, and Mengjie Zhang. Explainable artificial in- telligence by genetic programming: A survey. IEEE Transactions on Evolutionary Computa- tion, 27(3):621–641, 2022

    2022

  18. [18]

    Learn- ing heuristics with different representations for stochastic routing.IEEE Transactions on Cy- bernetics, 53(5):3205–3219, 2022

    Ya-Hui Jia, Yi Mei, and Mengjie Zhang. Learn- ing heuristics with different representations for stochastic routing.IEEE Transactions on Cy- bernetics, 53(5):3205–3219, 2022

    2022

  19. [19]

    A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and inter- activity.arXiv preprint arXiv:2302.04023, 2023

    Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Love- nia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and inter- activity.arXiv preprint arXiv:2302.04023, 2023

    Pith/arXiv arXiv 2023

  20. [20]

    Evolving code with a large language model.Genetic Programming and Evolvable Ma- chines, 25(2):21, 2024

    Erik Hemberg, Stephen Moskal, and Una-May O’Reilly. Evolving code with a large language model.Genetic Programming and Evolvable Ma- chines, 25(2):21, 2024

    2024

  21. [21]

    When large language models meet evolutionary algo- rithms.arXiv preprint arXiv:2401.10510, 2024

    Wang Chao, Jiaxuan Zhao, Licheng Jiao, Lin- gling Li, Fang Liu, and Shuyuan Yang. When large language models meet evolutionary algo- rithms.arXiv preprint arXiv:2401.10510, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Bridging evolutionary algorithms and reinforcement learning: A com- prehensive survey on hybrid algorithms.IEEE Transactions on evolutionary computation, 2024

    Pengyi Li, Jianye Hao, Hongyao Tang, Xian Fu, Yan Zhen, and Ke Tang. Bridging evolutionary algorithms and reinforcement learning: A com- prehensive survey on hybrid algorithms.IEEE Transactions on evolutionary computation, 2024

    2024

  23. [23]

    Fully autonomous programming with large language models

    Vadim Liventsev, Anastasiia Grishina, Aki H¨ arm¨ a, and Leon Moonen. Fully autonomous programming with large language models. In Proceedings of the Genetic and Evolution- ary Computation Conference, pages 1146–1155, 2023

    2023

  24. [24]

    Eureka: Human-level reward de- sign via coding large language models.arXiv preprint arXiv:2310.12931, 2023

    Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward de- sign via coding large language models.arXiv preprint arXiv:2310.12931, 2023

    Pith/arXiv arXiv 2023

  25. [25]

    Evolution through large models

    Joel Lehman, Jonathan Gordon, Shawn Jain, Kamal Ndousse, Cathy Yeh, and Kenneth O Stanley. Evolution through large models. In Handbook of evolutionary machine learning, pages 331–366. Springer, 2023

    2023

  26. [26]

    Advancing math- ematics by guiding human intuition with ai.Na- ture, 600(7887):70–74, 2021

    Alex Davies, Petar Veliˇ ckovi´ c, Lars Buesing, Sam Blackwell, Daniel Zheng, Nenad Tomaˇ sev, Richard Tanburn, Peter Battaglia, Charles Blundell, Andr´ as Juh´ asz, et al. Advancing math- ematics by guiding human intuition with ai.Na- ture, 600(7887):70–74, 2021

    2021

  27. [27]

    Discovering symbolic models from deep learning with inductive biases

    Miles Cranmer, Alvaro Sanchez Gonzalez, Pe- ter Battaglia, Rui Xu, Kyle Cranmer, David Spergel, and Shirley Ho. Discovering symbolic models from deep learning with inductive biases. Advances in neural information processing sys- tems, 33:17429–17442, 2020

    2020

  28. [28]

    Palm 2 technical report.arXiv preprint arXiv:2305.10403, 2023

    Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm 2 technical report.arXiv preprint arXiv:2305.10403, 2023

    Pith/arXiv arXiv 2023

  29. [29]

    For- mal mathematical reasoning: A new frontier in ai.AI Magazine, 2025

    Alice Q Jiang, Yi Wu, and Mateja Jamnik. For- mal mathematical reasoning: A new frontier in ai.AI Magazine, 2025. 14

    2025

  30. [30]

    Evolution of heuristics: Towards efficient automatic algorithm design using large language model.arXiv preprint arXiv:2401.02051, 2024

    Fei Liu, Xialiang Tong, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang. Evolution of heuristics: Towards efficient automatic algorithm design using large language model.arXiv preprint arXiv:2401.02051, 2024

    Pith/arXiv arXiv 2024

  31. [31]

    Pham Vu Tuan Dat, Long Doan, and Huynh Thi Thanh Binh. Hsevo: Elevating automatic heuristic design with diversity-driven harmony search and genetic algorithm using llms.Pro- ceedings of the AAAI Conference on Artificial Intelligence, 39(25):26931–26938, 2025

    2025

  32. [32]

    Reevo: Large language models as hyper-heuristics with reflec- tive evolution.Advances in neural information processing systems, 37:43571–43608, 2024

    Haoran Ye, Jiarui Wang, Zhiguang Cao, Fed- erico Berto, Chuanbo Hua, Haeyeon Kim, Jinkyoo Park, and Guojie Song. Reevo: Large language models as hyper-heuristics with reflec- tive evolution.Advances in neural information processing systems, 37:43571–43608, 2024

    2024

  33. [33]

    Beyond the hype: Benchmarking llm-evolved heuristics for bin packing

    Kevin Sim, Quentin Renau, and Emma Hart. Beyond the hype: Benchmarking llm-evolved heuristics for bin packing. InProceedings of EvoSTAR 2025, 2025

    2025

  34. [34]

    Detecting hallucinations in large language models using semantic entropy

    Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017):625–630, 2024

    2024

  35. [35]

    Johnson, Alan Demers, Jeffrey D

    David S. Johnson, Alan Demers, Jeffrey D. Ull- man, Michael R Garey, and Ronald L. Graham. Worst-case performance bounds for simple one- dimensional packing algorithms.SIAM Journal on computing, 3(4):299–325, 1974

    1974

  36. [36]

    Near-optimal algo- rithms for stochastic online bin packing.ACM Transactions on Algorithms, 21(2):1–39, 2025

    Nikhil Ayyadevara, Rajni Dabas, Arindam Khan, and KVN Sreenivas. Near-optimal algo- rithms for stochastic online bin packing.ACM Transactions on Algorithms, 21(2):1–39, 2025

    2025

  37. [37]

    The average-case analysis of some on-line algorithms for bin packing.Combinator- ica, 6(2):179–200, 1986

    Peter W Shor. The average-case analysis of some on-line algorithms for bin packing.Combinator- ica, 6(2):179–200, 1986

    1986

  38. [38]

    A stochastic model of bin- packing.Information and control, 44(2):105– 115, 1980

    Edward G Coffman Jr, Kimming So, Micha Hofri, and AC Yao. A stochastic model of bin- packing.Information and control, 44(2):105– 115, 1980

    1980

  39. [39]

    Bin packing with discrete item sizes, part ii: Tight bounds on first fit.Random Structures & Algorithms, 10(1-2):69–101, 1997

    Edward G Coffman Jr, David S Johnson, Pe- ter W Shor, and Richard R Weber. Bin packing with discrete item sizes, part ii: Tight bounds on first fit.Random Structures & Algorithms, 10(1-2):69–101, 1997

    1997

  40. [40]

    Some unexpected expected behavior results for bin packing

    Jon Louis Bentley, David S Johnson, Frank Thomson Leighton, Catherine C Mc- Geoch, and Lyle A McGeoch. Some unexpected expected behavior results for bin packing. In Proceedings of the sixteenth annual ACM sym- posium on Theory of computing, pages 279–288, 1984

    1984

  41. [41]

    Best-fit bin-packing with random order

    Claire Kenyon et al. Best-fit bin-packing with random order. InSODA, volume 96, pages 359– 364, 1996

    1996

  42. [42]

    Or-library: distributing test problems by electronic mail.Journal of the oper- ational research society, 41(11):1069–1072, 1990

    John E Beasley. Or-library: distributing test problems by electronic mail.Journal of the oper- ational research society, 41(11):1069–1072, 1990

    1990

  43. [43]

    Examples are not enough, learn to crit- icize! criticism for interpretability.Advances in neural information processing systems, 29, 2016

    Been Kim, Rajiv Khanna, and Oluwasanmi O Koyejo. Examples are not enough, learn to crit- icize! criticism for interpretability.Advances in neural information processing systems, 29, 2016

    2016

  44. [44]

    Obtaining dynamic scheduling policies with simulation and machine learning

    Danilo Carastan-Santos and Raphael Y De Ca- margo. Obtaining dynamic scheduling policies with simulation and machine learning. InPro- ceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–13, 2017

    2017

  45. [45]

    Qube: En- hancing automatic heuristic design via quality- uncertainty balanced evolution.arXiv preprint arXiv:2412.20694, 2024

    Zijie Chen, Zhanchao Zhou, Yu Lu, Renjun Xu, Lili Pan, and Zhenzhong Lan. Qube: En- hancing automatic heuristic design via quality- uncertainty balanced evolution.arXiv preprint arXiv:2412.20694, 2024. 15

    Pith/arXiv arXiv 2024

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.