Pith. sign in

REVIEW 3 major objections 7 minor 44 references

Successor-Generator Planning with LLM-generated Heuristics

T0 review · 3 major / 7 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM-generated heuristics can match the best numeric planners and reach problems PDDL cannot express.

desk verdict Useful empirical study of LLM-generated heuristics for numeric planning, but the state-of-the-art claim hinges on picking the best of 18 configurations on the same benchmark set. read the letter →

arxiv 2501.18784 v5 pith:7DCIZOW4 submitted 2025-01-30 cs.AI

Alexander Tuisov , Yonatan Vernik , Alexander Shleyfman This is my paper · ORCID
classification cs.AI
keywords automatedplanningheuristicsearchLLM-generatedheuristicssuccessorgeneratornumericgreedybest-firstcodegenerationPDDL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Planners need heuristics to guide search, and building them by hand is the bottleneck that this paper removes using large language models. The task is described as a successor generator, a goal test, and an initial state in Rust; the LLM writes a heuristic function in the same language, which is then compiled into greedy best-first search. The paper reports that this automatic pipeline is competitive with, and in several domains better than, leading numeric planners on the IPC-2023 benchmark set, and that it solves two problems (Twin Prime and Deterministic Pacman) that cannot be adequately expressed in PDDL. The point of the work is that LLMs, which cannot yet plan reliably, can still write the search guidance that makes planning work.

What carries the argument

The central object is an Explicit Successor Generator (ESG): a planning task expressed as a Rust successor generator, a goal-testing function, and a JSON file giving the initial state and goal. The LLM turns this ESG plus a prompt into a heuristic function in Rust, which is compiled and dropped into a standard greedy best-first search, so search remains sound and requires no further model queries. Because the heuristic is ordinary code it can contain arithmetic over numeric fluents, custom distance estimates, or subroutines, which is what lets the method handle infinite state spaces and non-PDDL dynamics. Fallback strategies (FirstCompilation, UntilSuccess, SelfPortfolio-N) cope with the fact that generated heuristics may fail to compile or may be weak; SelfPortfolio-N runs N candidate heuristics in fixed time slices and resamples on failure.

What would settle it

Have independent researchers re-translate the eighteen IPC-2023 numeric domains from PDDL into Rust, run the same LLM-generation pipeline, and validate every returned plan with VAL; if coverage changes materially, the hand-written Rust encoding, not the heuristic synthesis, is responsible for the reported results.

Watch

Extended reading notes

Core claim

The central discovery is that LLM-generated heuristics can replace domain-independent heuristic derivation: given only the executable code of a task's successor generator and goal test, a language model produces a heuristic function that guides greedy best-first search to solutions. The authors show this by encoding eighteen IPC-2023 numeric domains in Rust, prompting several GPT and Claude models for heuristics, and comparing coverage against ENHSP, Metric-FF, and Numeric-FD under identical time and memory limits. They find the best configurations match or exceed the strongest domain-independent planners, doing especially well on Sailing and Zenotravel, and that the approach solves Twin Prime and Deterministic Pacman, tasks they argue no PDDL dialect can express. Because the heuristic is generated once and then compiled, the search itself is sound and requires no further LLM calls.

Load-bearing premise

The load-bearing premise is that the manually written Rust encodings of the IPC-2023 numeric PDDL benchmarks faithfully capture the original PDDL semantics; if a translation subtly changes preconditions, numeric conditions, or effects, the coverage comparison is not against the intended benchmark.

Editorial extensions

If this is right

  • Handcrafted domain knowledge is no longer required: a problem stated as code plus a goal test suffices to obtain a usable heuristic.
  • The representation extends planning to transition systems with unbounded numeric variables, conditional effects, and custom dynamics that PDDL dialects cannot express.
  • Heuristics can be generated once per domain and reused across instances, since instance-specific prompts did not help for the tested non-reasoning models and increased cost.
  • On the IPC-2023 numeric benchmarks the approach is competitive with, and in several domains superior to, the leading domain-independent numeric planners.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If LLM code synthesis continues to improve, planning may move away from PDDL-derived heuristics toward reading executable state-transition functions directly; the paper's explicit dependence on hand-written Rust encodings is the main obstacle, and automating that translation is an obvious next step.
  • The ablation showing that instance-specific prompts hurt most domains suggests the current models use the instance information to overfit or mis-tune coefficients; a stronger reasoning model, or a prompt asking for parameterized heuristics, might reverse this and make each instance's heuristic more precise.
  • Combining an LLM heuristic with a domain-independent heuristic in a parallel search could combine strengths: the paper's own representation control shows the Rust encoding alone is not what drives the gains, pointing to heuristic complementarity rather than simple replacement.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes an approach to satisficing numeric planning in which a planning task is represented as an 'Explicit Successor Generator' in Rust (successor function, goal test, initial state), and an LLM is prompted to generate a Rust heuristic function that is compiled into greedy best-first search. The authors evaluate six LLMs under three fallback strategies (FirstCompilation, UntilSuccess, and SelfPortfolio-N) on the IPC-2023 numeric benchmarks, which are manually translated from PDDL to Rust, plus two non-PDDL domains (Twin Prime and Pacman). Table 1 reports that the best configuration shown there, GPT-4.1 with SelfPortfolio-10, solves 247 of the 360 PDDL instances, above the individual numeric planners NFD (202) and MFF (217), but below the P(3h||3n) portfolio (295), which the authors set aside as post-hoc. The paper also presents ablations of instance-specificity, model/fallback choice, and reasoning effort.

Significance. The idea of using LLMs to synthesize problem-specific heuristic functions is timely and potentially useful: it avoids repeated LLM calls during search, yields inspectable code, and the two non-PDDL domains illustrate representational flexibility. The evaluation has genuine strengths, including external IPC-2023 numeric benchmarks, plan validation with VAL against the original PDDL, multiple LLMs, fallback strategies, and a Monte-Carlo variance analysis of heuristic order. However, the headline state-of-the-art claim is not yet supported because the best model and fallback configuration are selected after observing performance on the same benchmark set, and because the winning configuration is effectively a portfolio of sampled heuristics. With a pre-registered configuration, a held-out evaluation, or an appropriately corrected comparison, the contribution would be significant.

major comments (3)
  1. [Table 1; Discussion] The central claim that LLM-generated heuristics achieve state-of-the-art performance rests on selecting the best of many evaluated configurations after seeing the benchmark results. Table 1 reports three models under three fallback strategies, and the ablation section states that six models and SP-5 were also tested, so the best result (GPT-4.1 with SP-10, 247 on the 360 PDDL instances) is a post-hoc selection. Footnote 4 dismisses the P(3h||3n) portfolio as 'a post-hoc optimization and not a fair comparison,' but the same criticism applies to choosing the best model and fallback after observing performance on the same 360-instance set; for example, FirstCompilation with GPT-4.1 solves only 144 instances. Figure 4 randomizes heuristic order within a single model and fallback policy, so it does not account for selection across models and fallbacks. Please report a pre-registered configuration, a held-out domain split, or a multiplicity-corrected summary over all configurations.
  2. [Table 1; Empirical Evaluation] The comparison in Table 1 is asymmetric: SelfPortfolio-10 allocates ten time slices and may generate and run up to ten distinct heuristics with restarts, whereas the baselines NFD and MFF are single-configuration planners. The paper's own non-portfolio LLM configurations, such as FirstCompilation with GPT-4.1 (144 on the 360 PDDL instances), are far below NFD (202) and MFF (217), so the reported advantage is largely attributable to the portfolio/retry mechanism rather than to an individual LLM-generated heuristic. Please add a fair single-heuristic comparison, or apply the same multi-run portfolio treatment to the baseline planners.
  3. [Discussion; Expressiveness] The abstract and Discussion claim that the approach 'enables the solution of problems that cannot be adequately expressed in any existing PDDL dialect,' but the Twin Prime and Pacman domains are manually encoded in Rust by the authors, and the paper gives no formal expressiveness argument for why these domains cannot be encoded in PDDL or its extensions. The manual encoding step is also acknowledged in the Methodology as requiring human intervention. As a proof-of-concept the two domains are interesting, but the claim should be softened to say that the approach handles tasks for which the authors did not find a practical PDDL encoding, with the human modeling step stated as part of the method.
minor comments (7)
  1. [Discussion] The sentence 'the best setting of our method is to use a non-reasoning model to generate non-IS heuristics and run them as SelfPortfolio-10 (276 with GPT-4.1)' conflicts with Table 1 (247 on the 360 PDDL instances) and Table 2 (281 on the 400 total instances); please clarify which number is intended and why.
  2. [Table 1 caption] The caption says results are from GPT-4.1, GPT-5.1, and Claude Sonnet 4.5, while the text states that six LLMs were tested; please clarify which models appear in the main table and where the full six-model results are reported.
  3. [Abstract] The phrase 'a broad range of established planning benchmarks' overstates the scope of the PDDL evaluation, which is limited to the IPC-2023 numeric track; please qualify the claim as applying to numeric satisficing planning.
  4. [Figure 2] The head-to-head plot in Figure 2 is computed only on instances that both planners solve, which can make the two approaches look more similar than they are across the full benchmark; please state this selection effect explicitly or show a plot that includes unsolved instances.
  5. [Impact of Instance-Specificity] The statement that instance-specificity 'slightly decreased generation time' is supported only by a p-value (P=2e-5) without describing the statistical test, sample size, or multiple-comparison correction; please provide these details.
  6. [Reproducibility] The paper does not provide a link to the code, the Rust translations of the IPC domains, or the full set of generated heuristics; releasing these would substantially improve reproducibility of the SP-10 results.
  7. [Typos] There are several typos and spacing errors, including 'it’s mini variant' (should be 'its'), 'Pacmanand' (missing space), and 'the qualitative measure ... can be seen in Figure 2' which should refer to 'comparisons'; a light copyedit would help.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's claims are empirical benchmark results, validated externally, and no prediction reduces to its inputs by construction.

full rationale

The paper has no formal derivation chain whose output is defined in terms of its input. The LLM-generated heuristics are synthesized from ESG Rust code plus a prompt, then executed in GBFS; coverage is measured on fixed IPC-2023 numeric benchmarks and two new non-PDDL domains, with PDDL solutions checked by VAL against the original PDDL specifications. The manual PDDL-to-Rust translation is a possible correctness risk, but VAL checking means translation errors would tend to invalidate or reduce coverage rather than manufacture solutions. The only candidate concern is post-hoc configuration selection: the paper chooses the best LLM/fallback configuration from Table 1 on the same benchmark set, while dismissing the P(3h||3n) baseline with the footnote that its 'best' heuristics are 'derived from performance on the same problem set used in our evaluation, rather than being fixed independently of it.' That is a legitimate methodological caveat about selection bias, but it is not circularity: no equation makes the reported coverage equal to a fitted parameter, no self-citation is load-bearing for the central result, and no uniqueness theorem or ansatz is imported from the authors' prior work. Under the rule that circularity must be exhibited as a specific reduction, none is present.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim is empirical, so the ledger is short. The paper introduces no fitted constants, but it relies on the faithfulness of manual Rust encodings, the semantic usability of LLM-generated code, and the correctness of its verifiers. The only hidden free values are the coefficients the LLM embeds in each generated heuristic.

free parameters (1)
  • LLM-chosen heuristic penalty coefficients = not reported; vary per generated heuristic
    The ablation section notes that heuristics combine multiple penalizing terms with guessed coefficients; these coefficients materially affect search guidance and are neither reported nor controlled, so they act as unobserved free parameters of the method.
assumptions (3)
  • domain assumption The manual Rust translations of IPC-2023 PDDL benchmarks faithfully implement the intended PDDL semantics.
    Methodology states that PDDL2.1 problems are manually translated into Rust functions. If a translation is wrong, the coverage numbers do not describe the benchmark as intended.
  • domain assumption LLM-generated Rust heuristic functions, once they compile, are semantically well-defined evaluators that do not alter the task's transition system.
    The generated heuristics are compiled into greedy best-first search; while final plans are verified, the performance comparison assumes the heuristic code evaluates the intended state features and terminates in reasonable time.
  • domain assumption The verification tools (VAL for PDDL and the custom sequential validator) correctly certify plan validity.
    Empirical Evaluation describes checking solutions with VAL and a custom procedure. Any bug in verification would inflate the reported solved counts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Successor-Generator Planning with LLM-generated Heuristics." pith.science (2026). https://pith.science/paper/7DCIZOW4

@misc{pith2026250118784,
  author       = {Pith},
  title        = {Pith review of: Successor-Generator Planning with LLM-generated Heuristics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7DCIZOW4}},
  note         = {Machine review of arXiv:2501.18784}
}
read the original abstract

Heuristics are a central component of deterministic planning, particularly in domain-independent settings where general applicability is prioritized over task-specific tuning. This work revisits that paradigm in light of recent advances in large language models (LLMs), which enable the automatic synthesis of heuristics directly from problem definitions -- bypassing the need for handcrafted domain knowledge. We present a method that employs LLMs to generate problem-specific heuristic functions from planning tasks specified through successor generators, goal tests, and initial states written in a general-purpose programming language. These heuristics are compiled and integrated into standard heuristic search algorithms, such as greedy best-first search. Our approach achieves competitive, and in many cases state-of-the-art, performance across a broad range of established planning benchmarks. Moreover, it enables the solution of problems that are difficult to express in traditional formalisms, including those with complex numeric constraints or custom transition dynamics. We provide an extensive empirical evaluation that characterizes the strengths and limitations of the approach across diverse planning settings, demonstrating its effectiveness.

Figures

Figures reproduced from arXiv: 2501.18784 by the authors.

Figure 1
Figure 1. Procedure flowchart. A general problem description is manually written using Rust components—successor generator, [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Per-instance comparisons of the Total Time (left) and expanded states (right) between GPT-4.1 with SelfPortfolio-10 [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Per-instance comparison of the Total Time (gen [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: As a variance analysis we generated 40 heuristics with GPT-4.1 in each domain, and performed a Monte-Carlo [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 32 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    J.; and Yao, Z

    Aghzal, M.; Plaku, E.; Stein, G. J.; and Yao, Z. 2025. A Survey on Large Language Models for Automated Planning. CoRR, abs/2502.12435

    arXiv 2025

  4. [4]

    Aldinger, J.; and Nebel, B. 2017. Interval Based Relaxation Heuristics for Numeric Planning with Action Costs. In SOCS, 155--156

    work page 2017

  5. [5]

    Besta, M.; Blach, N.; Kubicek, A.; Gerstenberger, R.; Podstawski, M.; Gianinazzi, L.; Gajda, J.; Lehmann, T.; Niewiadomski, H.; Nyczyk, P.; and Hoefler, T. 2024. Graph of Thoughts: Solving Elaborate Problems with Large Language Models. In AAAI, 17682--17690. AAAI Press

    work page 2024

  6. [6]

    Bonet, B.; and Geffner, H. 2001. Planning as heuristic search. Artificial Intelligence, 129(1-2): 5--33

    work page 2001

  7. [7]

    Z.; and Thi \' e baux, S

    Chen, D. Z.; and Thi \' e baux, S. 2024. Novelty Heuristics, Multi-Queue Search, and Portfolios for Numeric Planning. In SOCS, 203--207

    work page 2024

  8. [8]

    Z.; Thi \' e baux, S.; and Trevizan, F

    Chen, D. Z.; Thi \' e baux, S.; and Trevizan, F. W. 2024. Learning Domain-Independent Heuristics for Grounded and Lifted Planning. In AAAI, 20078--20086. AAAI Press

    work page 2024

Show all 44 references
  1. [9]

    Chen, X.; Lin, M.; Sch \" a rli, N.; and Zhou, D. 2024. Teaching Large Language Models to Self-Debug. In ICLR. OpenReview.net

    2024

  2. [10]

    B.; Pereira, A

    Corr \^e a, A. B.; Pereira, A. G.; and Seipp, J. 2025. Classical Planning with LLM-Generated Heuristics: Challenging the State of the Art with Python Code. arXiv preprint arXiv:2503.18809

    2025

  3. [11]

    a t Freiburg, Institut f \

    Edelkamp, S.; and Hoffmann, J. 2004. PDDL2.2 : The Language for the Classical Part of the 4th International Planning Competition . Technical Report 195, Albert-Ludwigs-Universit \"a t Freiburg, Institut f \"u r Informatik

    2004

  4. [12]

    Fox, M.; and Long, D. 2003. PDDL2.1 : An Extension to PDDL for Expressing Temporal Planning Domains. JAIR, 20: 61--124

    2003

  5. [13]

    Gerevini, A.; and Long, D. 2005. Plan Constraints and Preferences in PDDL3 . Technical Report R.\,T.\ 2005-08-47, Dipartimento di Elettronica per l'Automazione, Universit\`a degli Studi di Brescia

    2005

  6. [14]

    Gnad, D.; Helmert, M.; Jonsson, P.; and Shleyfman, A. 2023. Planning over Integers: Compilations and Undecidability. In ICAPS, 148--152. AAAI Press

    2023

  7. [15]

    Guan, L.; Valmeekam, K.; Sreedharan, S.; and Kambhampati, S. 2023. Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning. In NeurIPS

    2023

  8. [16]

    J.; Wang, Z.; Wang, D

    Hao, S.; Gu, Y.; Ma, H.; Hong, J. J.; Wang, Z.; Wang, D. Z.; and Hu, Z. 2023. Reasoning with Language Model is Planning with World Model. In EMNLP, 8154--8173. Association for Computational Linguistics

    2023

  9. [17]

    E.; Nilsson, N

    Hart, P. E.; Nilsson, N. J.; and Raphael, B. 1968. A Formal Basis for the Heuristic Determination of Minimum Cost Paths. IEEE Transactions on Systems Science and Cybernetics, 4(2): 100--107

    1968

  10. [18]

    Hoffmann, J. 2003. The Metric-FF Planning System: Translating ''Ignoring Delete Lists'' to Numeric State Variables. JAIR, 20: 291--341

    2003

  11. [19]

    Howey, R.; Long, D.; and Fox, M. 2004. VAL: Automatic Plan Validation, Continuous Effects and Mixed Initiative Planning Using PDDL . In ICTAI, 294--301. IEEE Computer Society

    2004

  12. [20]

    Izhaki, D.; Green, R.; and Shleyfman, A. 2025. Numeric Domains: PDDL and Rust Implementation. Technical report, Computer Science Department, Bar-Ilan University

    2025

  13. [21]

    Katz, M.; Kokel, H.; Srinivas, K.; and Sohrabi, S. 2024. Thought of Search: Planning with Language Models Through The Lens of Efficiency. In NeurIPS

    2024

  14. [22]

    Kuroiwa, R.; Shleyfman, A.; and Beck, J. C. 2022. LM-Cut Heuristics for Optimal Linear Numeric Planning. In ICAPS

    2022

  15. [23]

    Liu, B.; Jiang, Y.; Zhang, X.; Liu, Q.; Zhang, S.; Biswas, J.; and Stone, P. 2023. LLM+P: Empowering Large Language Models with Optimal Planning Proficiency. CoRR, abs/2304.11477

    2023 arXiv

  16. [24]

    Liu, F.; Tong, X.; Yuan, M.; Lin, X.; Luo, F.; Wang, Z.; Lu, Z.; and Zhang, Q. 2024. Evolution of heuristics: Towards efficient automatic algorithm design using large language model. arXiv preprint arXiv:2401.02051

    2024 arXiv

  17. [25]

    P.; Hermann, K.; Welleck, S.; Yazdanbakhsh, A.; and Clark, P

    Madaan, A.; Tandon, N.; Gupta, P.; Hallinan, S.; Gao, L.; Wiegreffe, S.; Alon, U.; Dziri, N.; Prabhumoye, S.; Yang, Y.; Gupta, S.; Majumder, B. P.; Hermann, K.; Welleck, S.; Yazdanbakhsh, A.; and Clark, P. 2023. Self-Refine: Iterative Refinement with Self-Feedback. In NeurIPS

    2023

  18. [26]

    R.; Zheng, Q.; Hui, B.; Zhuo, T

    Muennighoff, N.; Liu, Q.; Zebaze, A. R.; Zheng, Q.; Hui, B.; Zhuo, T. Y.; Singh, S.; Tang, X.; von Werra, L.; and Longpre, S. 2024. OctoPack: Instruction Tuning Code Large Language Models. In ICLR. OpenReview.net

    2024

  19. [27]

    T.; Srinivas, K.; Kokel, H.; Lee, J.; Katz, M.; and Sohrabi, S

    Oswald, J. T.; Srinivas, K.; Kokel, H.; Lee, J.; Katz, M.; and Sohrabi, S. 2024. Large Language Models as Planning Domain Generators. In ICAPS, 423--431. AAAI Press

    2024

  20. [28]

    Pearl, J. 1984. Heuristics: Intelligent Search Strategies for Computer Problem Solving . Addison-Wesley

    1984

  21. [29]

    Scala, E.; Haslum, P.; Thi \' e baux, S.; and Ram \' rez, M. 2020. Subgoaling Techniques for Satisficing and Optimal Numeric Planning. JAIR, 68: 691--752

    2020

  22. [30]

    Sel, B.; Al - Tawaha, A.; Khattar, V.; Jia, R.; and Jin, M. 2024. Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models. In ICML. OpenReview.net

    2024

  23. [31]

    Shinn, N.; Cassano, F.; Gopinath, A.; Narasimhan, K.; and Yao, S. 2023. Reflexion: language agents with verbal reinforcement learning. In NeurIPS

    2023

  24. [32]

    B.; Kaelbling, L

    Silver, T.; Dan, S.; Srinivas, K.; Tenenbaum, J. B.; Kaelbling, L. P.; and Katz, M. 2024. Generalized Planning in PDDL Domains with Pretrained Large Language Models. In AAAI, 20256--20264. AAAI Press

    2024

  25. [33]

    Stojkovic, J.; Choukse, E.; Zhang, C.; Goiri, I.; and Torrellas, J. 2024. Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference. arXiv preprint arXiv:2403.20306

    2024 arXiv

  26. [34]

    Taitler, A.; Alford, R.; Espasa, J.; Behnke, G.; Fiser, D.; Gimelfarb, M.; Pommerening, F.; Sanner, S.; Scala, E.; Schreiber, D.; Segovia - Aguas, J.; and Seipp, J. 2024. The 2023 International Planning Competition. AI Mag. , 45(2): 280--296

    2024

  27. [35]

    Tantakoun, M.; Zhu, X.; and Muise, C. 2025. LLMs as Planning Modelers: A Survey for Leveraging Large Language Models to Construct Automated Planning Models. CoRR, abs/2503.18971

    2025

  28. [36]

    O.; Sreedharan, S.; and Kambhampati, S

    Valmeekam, K.; Hernandez, A. O.; Sreedharan, S.; and Kambhampati, S. 2022. Large Language Models Still Can't Plan (A Benchmark for LLMs on Planning and Reasoning about Change). CoRR, abs/2206.10498

    2022 arXiv

  29. [37]

    V.; Zhou, D.; et al

    Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 24824--24837

    2022

  30. [38]

    Xu, B.; Peng, Z.; Lei, B.; Mukherjee, S.; Liu, Y.; and Xu, D. 2023. ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models. CoRR, abs/2305.18323

    2023 arXiv

  31. [39]

    Yao, S.; Yu, D.; Zhao, J.; Shafran, I.; Griffiths, T.; Cao, Y.; and Narasimhan, K. 2023 a . Tree of Thoughts: Deliberate Problem Solving with Large Language Models. In NeurIPS

    2023

  32. [40]

    R.; and Cao, Y

    Yao, S.; Zhao, J.; Yu, D.; Du, N.; Shafran, I.; Narasimhan, K. R.; and Cao, Y. 2023 b . ReAct: Synergizing Reasoning and Acting in Language Models. In ICLR. OpenReview.net

    2023

  33. [41]

    Ye, H.; Wang, J.; Cao, Z.; Berto, F.; Hua, C.; Kim, H.; Park, J.; and Song, G. 2024. Reevo: Large language models as hyper-heuristics with reflective evolution. Advances in neural information processing systems, 37: 43571--43608

    2024

  34. [42]

    Zhang, K.; Li, Z.; Li, J.; Li, G.; and Jin, Z. 2023. Self-Edit: Fault-Aware Code Editor for Code Generation. In ACL, 769--787. Association for Computational Linguistics

    2023

  35. [43]

    Zhong, L.; Wang, Z.; and Shang, J. 2024. Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step by Step. In ACL (Findings), 851--870. Association for Computational Linguistics

    2024

  36. [44]

    Zhou, A.; Wang, K.; Lu, Z.; Shi, W.; Luo, S.; Qin, Z.; Lu, S.; Jia, A.; Song, L.; Zhan, M.; and Li, H. 2024. Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification. In ICLR. OpenReview.net

    2024

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.