Pith. sign in

REVIEW 5 major objections 6 minor 78 references

Code-Driven Planning in Grid Worlds with Large Language Models

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that iterative refinement of LLM-generated code turns weak grid-world policies into strong, reusable programs, with 10%-to-10x gains over direct code generation on GRASP.

desk verdict A useful code-as-policy refinement study whose headline gains hinge on a train/test split the paper never describes. read the letter →

arxiv 2505.10749 v1 pith:GRKH4CGP submitted 2025-05-15 cs.AI

Ashwath Vaithinathan Aravindan , Zhisheng Tang , Mayank Kejriwal This is my paper · ORCID
classification cs.AI
keywords iterativeprogrammaticplanningLLMcodegenerationpolicysynthesisgrid-worldrefinementGRASPbenchmarkMiniGrid
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that planning in grid worlds can be solved not by asking a language model to output an action plan, but by asking it to write a program that acts as the policy, then repeatedly executing that program, scoring it on sample tasks, and feeding the worst failures back to the model for revision. It claims this iterative programmatic planning (IPP) loop improves over one-shot direct code generation by roughly 10% to 10x on the GRASP benchmark across five of six tested LLMs, and reports a new state-of-the-art result there. A sympathetic reader would care because the resulting policy is a readable, reusable Python program: once synthesized it can be applied to any number of new instances without further model calls, which the paper estimates makes GPT-o3-mini's amortized cost about 400x cheaper than per-instance prompting.

What carries the argument

The load-bearing mechanism is the Iterative Refinement loop, stated as Algorithm 2.1. Starting from a directly generated program $f^{(0)}$, the loop evaluates it on a training set of task instances, computes the average task metric $J$, and if the average improved, feeds the $k=3$ worst-scoring instances to the LLM as revision feedback to produce $f^{(t+1)}$; it stops when the average stops improving. The paper describes this as a discrete, non-gradient search over program space in which worst-case feedback acts as a heuristic gradient. Two auxiliary prompting strategies, pseudocode extension and step-by-step curriculum prompting, seed or stage the synthesis.

What would settle it

Re-run the full IPP pipeline with an explicit split: refine on one set of GRASP and MiniGrid instances, freeze the final program, and score it only on a disjoint held-out set; if the improvement over direct generation collapses or reverses on the held-out set, the central claim of learned generalizable policies is refuted.

Watch

Extended reading notes

Core claim

The central claim is that an LLM that cannot directly produce a good plan can still write a policy program that, when executed and scored, reveals concrete failing instances; feeding those instances back lets the same model correct logical errors, add missing constraints, and restructure control flow. On GRASP, iterative refinement turns Gemini-2.5-Pro's negative direct-generation scores (e.g., -0.48) into positive scores (3.27), lifts Claude-3.7 energy from 0.89 to 0.98 via pseudocode extension and GPT-o3-mini from 0.44 to 1.73 via curriculum prompting, and on MiniGrid takes GPT-o3-mini's Unlock-Pickup from zero reward and zero completion to 0.78 reward and 84% completion. The paper also claims that this beats chain-of-thought and two-step chain-of-thought prompting at lower API cost, because the synthesized program generalizes without per-instance inference.

Load-bearing premise

The load-bearing premise is that the instances used to give refinement feedback differ from the instances used to compute final scores; the paper refines on a 'training set' but never describes a held-out split, so if the sets coincide the reported gains would measure fitting rather than generalization.

Editorial extensions

If this is right

  • The synthesized policies are reusable: one program produced by IPP can be executed on any number of task instances without further LLM calls, which the paper estimates makes GPT-o3-mini's amortized GRASP cost about 400x lower than per-instance reasoning prompting.
  • Weak initial policies are not fatal: GPT-o3-mini moves from 0% to 84% completion on Unlock-Pickup, so refinement can rescue models whose one-shot code fails completely.
  • Most of the benefit arrives in the first one or two refinement iterations, after which scores plateau or decline slightly, so the loop is cheap to run.
  • Structured scaffolding helps: pseudocode extension raises Claude-3.7's GRASP energy from 0.89 to 0.98, and step-by-step curriculum prompting raises GPT-o3-mini from 0.44 to 1.73.
  • If these results hold, code-driven planning offers an interpretable, low-cost alternative to reinforcement learning in discrete grid tasks, with human-readable and editable policies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same generate-score-refine loop should transfer to any setting where an LLM can emit an executable policy and a cheap reward signal exists, such as PDDL planning, robot control programs, or decision-rule synthesis; the paper tests only grid worlds.
  • Because the 16,000 GRASP instances vary by movement type, carry limit, step cost, energy layout, obstacles, and start position, a natural extension is to measure how refinement on one distribution transfers to a different distribution of the same benchmark.
  • The $k=3$ worst-instance selection is an arbitrary choice; testing whether selecting diverse or adversarial failures rather than the three worst changes convergence rate would clarify what the feedback signal is doing.
  • The most decisive follow-up is an explicit train/test split, which the current write-up does not provide; without it, the reported gains could partly reflect refinement on the evaluation distribution.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper proposes Iterative Programmatic Planning (IPP), a framework in which an LLM is prompted to synthesize a Python policy for a grid-world task and then iteratively refine it using execution feedback. The main components are direct generation, pseudocode-conditioned extension, step-by-step curriculum prompting, and an iterative refinement loop (Algorithm 2.1) that selects the k worst-performing training instances and asks the LLM to repair the code. The paper evaluates six LLMs on GRASP and three MiniGrid tasks, reporting that IR improves over DG by 10% to 10x for five of six models, that IPP matches or beats CoT and 2-step CoT at lower API cost, and that it establishes a new state of the art on GRASP. The abstract and introduction also emphasize interpretability, reusability, and the low amortized cost of synthesized policies.

Significance. If the claims hold, the contribution is useful: a simple generate-execute-refine loop can turn weak LLM policies into reusable, interpretable programs and outperform per-instance prompting at lower amortized cost. The appendix is genuinely helpful, with full prompt templates and worked examples of successful refinements, and the breadth across six models and two benchmarks is a strength. However, the headline result rests on the synthesized policy generalizing to instances that did not drive refinement, and the manuscript never establishes that. The unsubstantiated GRASP state-of-the-art claim and unresolved ambiguities in the evaluation population mean that the current evidence supports a promising mechanism rather than the stated conclusions.

major comments (5)
  1. [§2.1/Algorithm 2.1 and §3] The evaluation never specifies a held-out split, so the central DG-vs-IR comparison may be in-sample. Algorithm 2.1 evaluates f(t) on the training set and selects Dfail from that same set (lines 4 and 10), while Section 3 says only that 'we generate 1,000 instances' per MiniGrid task and does not state that the reported numbers come from a disjoint test set. Table 6 makes the problem concrete: for GPT-o3-mini on GRASP, Iter 2 (1.41) is lower than Iter 1 (2.90), which cannot happen under Algorithm 2.1's stopping condition (Jcurr ≤ Jprev) if Jcurr is computed on the same instances used for refinement. This suggests either that the trajectory in Table 6 is evaluated on a different instance population than the one used for refinement or that the algorithm was not followed as written; either way, the population used for evaluation must be defined and reported. As written, the reported 10%–10x gains could be fitting artifacts, and the 'reusable policy' and amortized-cost arguments are unsupported. Please re-run with an explicitly disjoint held-out set and report train and test performance separately.
  2. [Abstract and §4] The claim of establishing a new state of the art on GRASP is not supported by any comparison. Section 4 compares IR only to DG, CoT, 2-step CoT, Random, and Greedy; no prior published result or existing GRASP baseline is cited or tabulated. The phrase 'state-of-the-art' requires a baseline; either add a table with previously reported GRASP numbers under matching settings, including results from the original GRASP benchmark paper, or remove the claim.
  3. [§3 and Table 3] The evaluation population and cost denominator are inconsistent. Section 3 states that 1,000 instances are generated per MiniGrid task, while Table 3 reports results 'averaged over 100 instances per task'; Tables 1–2 do not state the number of instances at all. For the cost comparison, Table 3 reports an IR per-instance API cost (e.g., 8.0e-4), but if the final program is reused, the marginal per-instance cost should be 0 and the reported number must be an amortization of some synthesis cost. Please state the instance count for each table, report the synthesis/refinement cost separately, and define the amortization formula.
  4. [Tables 1 and 5] The GPT-o3-mini rows contain values identical to GPT-o1 rows, which suggests a labeling or data-processing error. In Table 1, GPT-o3-mini's entries under 'With' (e.g., 2.89/2.92, 2.85/2.92) equal GPT-o1's IR entries in the same columns, and in Table 5, GPT-o3-mini's cost-0 DG values (5.75, 5.73, 5.78, ...) equal GPT-o1's cost-0 DG values. Please verify the model assignments and regenerate these tables; if the duplication is not an error, explain why GPT-o3-mini exactly replicates GPT-o1's numbers.
  5. [§4, Tables 2–3] No statistical tests or confidence intervals are reported for the comparisons that back the headline claims, and several reported differences appear to be within noise. For example, GPT-o3-mini Unlock-Pickup IR is 0.78 ± 0.34 versus 2-step CoT 0.84 ± 0.30, a difference of 0.06 that is far smaller than the standard deviations. The abstract's 'equal or better performance' claim needs paired significance tests or bootstrap confidence intervals over instances and seeds, at least for the IR-vs-DG and IR-vs-2-step-CoT comparisons.
minor comments (6)
  1. [§2.1] The text refers to Algorithm 2.1, but the displayed pseudocode is titled 'Algorithm 2.2'; fix the numbering and all cross-references.
  2. [Abstract] The abstract's '10% to as much as 10x' phrasing is imprecise when baselines are negative: for example, Table 1 reports Gemini-2.5-Pro moving from -0.48 to 3.27 as +781%, a percentage based on a negative denominator; report absolute changes or define the improvement metric explicitly.
  3. [Table 1] The main paper's Table 1 omits standard deviations even though Table 4 provides them; move the standard deviations into the main table or state explicitly that Table 4 is the version with error bars.
  4. [Abstract and §4] The cost claims in the abstract ($0.08 per task versus $0.002 per instance, 400x amortization) are not derived from any table in the paper; add a concrete calculation and define what constitutes a 'task' versus an 'instance'.
  5. [Figure 1] Figure 1's caption reports an IR reward of 0.96 but does not identify the task or the reward scale; align it with Table 2's MiniGrid tasks and state which configuration is shown.
  6. [§3] The sentence 'Code for reproducing all experiments is available here' contains no visible URL or repository identifier in the manuscript; include the link or an anonymized repository.

Circularity Check

1 steps flagged · score 6.0 of 10

IR refinement and reported evaluation are not separated by a held-out split, so the headline IR-vs-DG gains are in-sample fits rather than demonstrated predictions; the GRASP SOTA claim also rests on the authors' own benchmark.

  1. fitted input called prediction [Section 2.1, Algorithm 2.2 (referred to in the text as Algorithm 2.1); Section 3; Section 4, Tables 1–3.]
    "D← evaluation results of f (t) on training set ... Dfail← SelectWorstCases(D, k = 3) ... f (t+1)← RefineWithFeedback(f (t),Dfail). Section 3: 'For each task, we generate 1,000 instances and measure two metrics: the average reward and the average completion rate.'"

    The final IR program is iteratively adapted to Dfail, the k worst-scoring instances in the training set (Algorithm 2.2, lines 4–11). Section 3 describes the evaluation benchmarks but never introduces a held-out test split; it states only that 1,000 MiniGrid instances were generated and measured. Tables 1–2 then report IR-vs-DG scores on the benchmark as a whole. As written, the reported IR numbers are evaluations on the same instance population that supplied the refinement feedback, so the 10%–10x gains and the 'reusable policy'/'generalization' claims are in-sample fits, not predictions on unseen instances: DG-vs-IR on that population is effectively a training-vs-DG comparison.

full rationale

The central empirical claim is partially circular as written. Algorithm 2.2 refines code on a 'training set' by selecting the worst k=3 instances and prompting the model to fix failures on exactly those instances, while Section 3 reports benchmark metrics without describing any disjoint held-out evaluation set. If the evaluation instances coincide with the training instances, the reported IR improvements are fitted to the evaluation population, and the paper's recurring claims about reusable, generalizing policies reduce to a training-set performance claim. The NeurIPS checklist asserts that data splits are detailed in the experimental setup, but Section 3 contains no split; this strengthens the concern, though the checklist itself is not a circular step. I did not count the GRASP 'new state-of-the-art' claim as a separate circular step because it is an unsupported, self-benchmarked assertion rather than a derivation forced by a cited result; however, it is worth noting that GRASP is cited from the authors' own prior work and the only programmatic baselines in Table 3 are the hand-coded Greedy and Random. The remaining components—pseudocode extension, step-by-step curriculum, and comparison against the external MiniGrid benchmark and six LLMs—are self-contained and show no circularity.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are introduced. The framework relies on free parameters in the refinement loop and prompting design, plus standard domain assumptions about grid environments and LLM code execution. The main unstated burden is that the evaluation set may be the same as the refinement set, which would make the reported performance a fitted quantity.

free parameters (6)
  • k (number of worst instances selected per refinement round) = 3
    Algorithm 2.1 uses k=3 without ablation; this choice affects the feedback seen by the LLM and thus the refined program.
  • Convergence threshold for iterative refinement = no improvement (Jcurr <= Jprev)
    The stopping rule in Algorithm 2.1 is an arbitrary threshold; it determines the number of LLM calls and the final program.
  • Pseudocode prior for GRASP extension = Greedy BFS pseudocode
    The pseudocode-conditioned prompting supplies a hand-written greedy algorithm as a starting point, injecting a task-specific inductive bias.
  • Curriculum step design for Step-by-Step prompting = Four incremental constraints
    The order and grouping of constraints (diagonals, carry limit, step cost, obstacles) is hand-designed and not ablated.
  • Number of refinement iterations per model = Determined by convergence; not fixed
    The paper reports iteration counts in Figure 2 and Table 6, but the budgets are model-dependent and not specified as experimental controls.
  • MiniGrid instance count per task = 1000
    The number of generated instances is chosen without justification; it affects variance and cost estimates.
assumptions (4)
  • domain assumption Grid environments are fully observable and deterministic, with a known transition function.
    Section 2 defines P with a fully defined, deterministic transition function; real robotic domains may not satisfy this.
  • domain assumption LLM-generated code can be executed directly against the environment without interface errors.
    The method assumes the model produces syntactically and semantically valid Python that correctly interprets the grid format, which is not guaranteed.
  • domain assumption The task performance metric J accurately reflects the desired behavior.
    Algorithm 2.1 uses J as the sole feedback signal; if J is misspecified, refinement optimizes the wrong objective.
  • domain assumption The LLM can understand failure feedback and produce fixes that preserve correct behavior elsewhere.
    The refinement step assumes the model can localize and repair bugs from a small set of worst-case instances; the paper itself notes this is imprecise in Section 5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Code-Driven Planning in Grid Worlds with Large Language Models." pith.science (2026). https://pith.science/paper/GRKH4CGP

@misc{pith2026250510749,
  author       = {Pith},
  title        = {Pith review of: Code-Driven Planning in Grid Worlds with Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GRKH4CGP}},
  note         = {Machine review of arXiv:2505.10749}
}
abstract

We propose an iterative programmatic planning (IPP) framework for solving grid-based tasks by synthesizing interpretable agent policies expressed in code using large language models (LLMs). Instead of relying on traditional search or reinforcement learning, our approach uses code generation as policy synthesis, where the LLM outputs executable programs that map environment states to action sequences. Our proposed architecture incorporates several prompting strategies, including direct code generation, pseudocode-conditioned refinement, and curriculum-based prompting, but also includes an iterative refinement mechanism that updates code based on task performance feedback. We evaluate our approach using six leading LLMs and two challenging grid-based benchmarks (GRASP and MiniGrid). Our IPP framework demonstrates improvements over direct code generation ranging from 10\% to as much as 10x across five of the six models and establishes a new state-of-the-art result for GRASP. IPP is found to significantly outperform direct elicitation of a solution from GPT-o3-mini (by 63\% on MiniGrid to 116\% on GRASP), demonstrating the viability of the overall approach. Computational costs of all code generation approaches are similar. While code generation has a higher initial prompting cost compared to direct solution elicitation (\$0.08 per task vs. \$0.002 per instance for GPT-o3-mini), the code can be reused for any number of instances, making the amortized cost significantly lower (by 400x on GPT-o3-mini across the complete GRASP benchmark).

Figures

Figures reproduced from arXiv: 2505.10749 by the authors.

Figure 1
Figure 1. Example of the Unlock Pickup task from the MiniGrid benchmark. The agent (red triangle) must retrieve a box (purple item) located in a separate room behind a locked door (yellow). Directions in the action sequence indicate the direction the agent turns to face before moving. (a) shows the original environment setup (b) shows actions generated by Claude-3.7 using direct generation, which fails and yields a reward of … view at source ↗
Figure 2
Figure 2. Performance of GPT-o1, Claude-3.7, and GPT-o3-mini across four tasks—GRASP, Unlock, Door-Key, and Unlock-Pickup—measured over successive refinement iterations. For GRASP, the metric is energy collected; for MiniGrid tasks, the metric is reward (maximum = 1.0). Iteration 0 corresponds to the initial output produced by the direct generation method [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Left: Performance comparison of Greedy and Pseudocode Extension strategies on the GRASP task, evaluated across GPT-o1, Claude-3.7, and GPT-o3-mini. The results highlight the performance gains enabled by pseudocode-guided reasoning. Right: Step-by-step performance on GRASP, where environment complexity is introduced incrementally over four steps [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

78 extracted references · 36 canonical work pages

  1. [1]

    Large language model guided self-debugging code generation

    Muntasir Adnan, Zhiwei Xu, and Carlos CN Kuhn. Large language model guided self-debugging code generation. arXiv preprint arXiv:2502.02928, 2025

    arXiv 2025

  2. [2]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    work page 2025

  3. [3]

    Compositional founda- tion models for hierarchical planning

    Anurag Ajay, Seungwook Han, Yilun Du, Shuang Li, Abhi Gupta, Tommi Jaakkola, Josh Tenenbaum, Leslie Kaelbling, Akash Srivastava, and Pulkit Agrawal. Compositional founda- tion models for hierarchical planning. Advances in Neural Information Processing Systems, 36:22304–22325, 2023

    work page 2023

  4. [4]

    Grid-based mobile robot path planning using aging-based ant colony optimization algorithm in static and dynamic environments

    Fatin Hassan Ajeil, Ibraheem Kasim Ibraheem, Ahmad Taher Azar, and Amjad J Humaidi. Grid-based mobile robot path planning using aging-based ant colony optimization algorithm in static and dynamic environments. Sensors, 20(7):1880, 2020

    work page 2020

  5. [5]

    Claude 3.7 sonnet, 2025

    Anthropic. Claude 3.7 sonnet, 2025

    2025

  6. [6]

    Iterative refinement of project-level code context for precise code generation with compiler feedback

    Zhangqian Bi, Yao Wan, Zheng Wang, Hongyu Zhang, Batu Guan, Fangxin Lu, Zili Zhang, Yulei Sui, Hai Jin, and Xuanhua Shi. Iterative refinement of project-level code context for precise code generation with compiler feedback. arXiv preprint arXiv:2403.16792, 2024

    arXiv 2024

  7. [7]

    Evaluating large language models trained on code

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021

    arXiv 2021

  8. [8]

    Teaching large language models to self-debug

    Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug. arXiv preprint arXiv:2304.05128, 2023

    arXiv 2023

Show all 78 references
  1. [9]

    A survey on explainable deep reinforcement learning

    Zelei Cheng, Jiahao Yu, and Xinyu Xing. A survey on explainable deep reinforcement learning. arXiv preprint arXiv:2502.06869, 2025

    2025 arXiv

  2. [10]

    Babyai: A platform to study the sample efficiency of grounded language learning

    Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, and Yoshua Bengio. Babyai: A platform to study the sample efficiency of grounded language learning. arXiv preprint arXiv:1810.08272, 2018

    arXiv

  3. [11]

    Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks

    Maxime Chevalier-Boisvert, Bolun Dai, Mark Towers, Rodrigo de Lazcano, Lucas Willems, Salem Lahlou, Suman Pal, Pablo Samuel Castro, and Jordan Terry. Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks. CoRR, abs/2306.13831, 2023

    2023 arXiv

  4. [12]

    Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks.Advances in Neural Information Processing Systems, 36:73383–73394, 2023

    Maxime Chevalier-Boisvert, Bolun Dai, Mark Towers, Rodrigo Perez-Vicente, Lucas Willems, Salem Lahlou, Suman Pal, Pablo Samuel Castro, and Jordan Terry. Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks.Advances in Neural ...

    2023

  5. [13]

    Palm: Scaling language modeling with pathways

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1– 113, 2023. 10

    2023

  6. [14]

    Leveraging procedural generation to benchmark reinforcement learning

    Karl Cobbe, Chris Hesse, Jacob Hilton, and John Schulman. Leveraging procedural generation to benchmark reinforcement learning. In International conference on machine learning, pages 2048–2056. PMLR, 2020

    2020

  7. [15]

    Quantifying generalization in reinforcement learning

    Karl Cobbe, Oleg Klimov, Chris Hesse, Taehoon Kim, and John Schulman. Quantifying generalization in reinforcement learning. In International conference on machine learning , pages 1282–1289. PMLR, 2019

    2019

  8. [16]

    Gemini 2.5 pro, 2025

    Google DeepMind. Gemini 2.5 pro, 2025

    2025

  9. [17]

    Cycle: Learning to self-refine the code generation

    Yangruibo Ding, Marcus J Min, Gail Kaiser, and Baishakhi Ray. Cycle: Learning to self-refine the code generation. Proceedings of the ACM on Programming Languages, 8(OOPSLA1):392– 418, 2024

    2024

  10. [18]

    Learning of generalizable and interpretable knowledge in grid-based reinforcement learning environments

    Manuel Eberhardinger, Johannes Maucher, and Setareh Maghsudi. Learning of generalizable and interpretable knowledge in grid-based reinforcement learning environments. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, volume...

    2023

  11. [19]

    A formal basis for the heuristic determination of minimum cost paths

    Peter Hart, Nils Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE Transactions on Systems Science and Cybernetics, 4(2):100–107, 1968

    1968

  12. [20]

    Neuro-symbolic approaches in artificial intelligence

    Pascal Hitzler, Aaron Eberhart, Monireh Ebrahimi, Md Kamruzzaman Sarker, and Lu Zhou. Neuro-symbolic approaches in artificial intelligence. National Science Review, 9(6):nwac035, 2022

    2022

  13. [21]

    Gridtopix: Training embodied agents with minimal supervision

    Unnat Jain, Iou-Jen Liu, Svetlana Lazebnik, Aniruddha Kembhavi, Luca Weihs, and Alexan- der G Schwing. Gridtopix: Training embodied agents with minimal supervision. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15141–15151, 2021

    2021

  14. [22]

    A survey on large language models for code generation

    Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. A survey on large language models for code generation. arXiv preprint arXiv:2406.00515, 2024

    2024 arXiv

  15. [23]

    Training llms to better self-debug and explain code

    Nan Jiang, Xiaopeng Li, Shiqi Wang, Qiang Zhou, Soneya Binta Hossain, Baishakhi Ray, Varun Kumar, Xiaofei Ma, and Anoop Deoras. Training llms to better self-debug and explain code. arXiv preprint arXiv:2405.18649, 2024

    2024 arXiv

  16. [24]

    Impact of code language models on automated program repair

    Nan Jiang, Kevin Liu, Thibaud Lutellier, and Lin Tan. Impact of code language models on automated program repair. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 1430–1442. IEEE, 2023

    2023

  17. [25]

    Evaluating open-domain question answering in the era of large language models

    Ehsan Kamalloo, Nouha Dziri, Charles LA Clarke, and Davood Rafiei. Evaluating open-domain question answering in the era of large language models. arXiv preprint arXiv:2305.06984, 2023

    2023 arXiv

  18. [26]

    Large language models are few-shot testers: Ex- ploring llm-based general bug reproduction

    Sungmin Kang, Juyeon Yoon, and Shin Yoo. Large language models are few-shot testers: Ex- ploring llm-based general bug reproduction. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 2312–2323. IEEE, 2023

    2023

  19. [27]

    A survey analyzing generalization in deep reinforcement learning

    Ezgi Korkmaz. A survey analyzing generalization in deep reinforcement learning. arXiv preprint arXiv:2401.02349, 2024

    2024 arXiv

  20. [28]

    Code as policies: Language model programs for embodied control

    Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 9493–9500. IEEE, 2023

    2023

  21. [29]

    Learning to solve and verify: A self-play framework for code and test generation

    Zi Lin, Sheng Shen, Jingbo Shang, Jason Weston, and Yixin Nie. Learning to solve and verify: A self-play framework for code and test generation. arXiv preprint arXiv:2502.14948, 2025

    2025

  22. [30]

    Large language model-based code generation for the control of construction assembly robots: A hierarchical generation approach

    Hanbin Luo, Jianxin Wu, Jiajing Liu, and Maxwell Fordjour Antwi-Afari. Large language model-based code generation for the control of construction assembly robots: A hierarchical generation approach. Developments in the Built Environment, 19:100488, 2024. 11

    2024

  23. [31]

    Self-refine: Iterative refinement with self-feedback

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems , 36:46534–46594, 2023

    2023

  24. [32]

    Audere: Automated strategy decision and realization in robot planning and control via llms

    Yue Meng, Fei Chen, Yongchao Chen, and Chuchu Fan. Audere: Automated strategy decision and realization in robot planning and control via llms. arXiv preprint arXiv:2504.03015, 2025

    2025

  25. [33]

    The monte carlo method

    Nicholas Metropolis and Stanislaw Ulam. The monte carlo method. Journal of the American statistical association, 44(247):335–341, 1949

    1949

  26. [34]

    Gpt-4o, 2024

    OpenAI. Gpt-4o, 2024

    2024

  27. [35]

    Gpt-o1, 2024

    OpenAI. Gpt-o1, 2024

    2024

  28. [36]

    Gpt-o3-mini, 2025

    OpenAI. Gpt-o3-mini, 2025

    2025

  29. [37]

    Openrouter

    OpenRouter. Openrouter. https://openrouter.ai, 2024. https://openrouter.ai

    2024

  30. [38]

    Large language models as planning domain generators

    James Oswald, Kavitha Srinivas, Harsha Kokel, Junkyu Lee, Michael Katz, and Shirin Sohrabi. Large language models as planning domain generators. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 34, pages 423–431, 2024

    2024

  31. [39]

    Curiosity-driven exploration by self-supervised prediction

    Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In International conference on machine learning, pages 2778–

  32. [40]

    Toolllm: Facilitating large language models to master 16000+ real-world apis

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789, 2023

    2023 arXiv

  33. [41]

    Tool learning with large language models: A survey

    Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. Tool learning with large language models: A survey. Frontiers of Computer Science, 19(8):198343, 2025

    2025

  34. [42]

    Code llama: Open foundation models for code

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950, 2023

    2023 arXiv

  35. [43]

    Generalized planning in pddl domains with pretrained large language models

    Tom Silver, Soham Dan, Kavitha Srinivas, Joshua B Tenenbaum, Leslie Kaelbling, and Michael Katz. Generalized planning in pddl domains with pretrained large language models. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 20256–20264, 2024

    2024

  36. [44]

    Toward expert-level medical question answering with large language models

    Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R Pfohl, Heather Cole-Lewis, et al. Toward expert-level medical question answering with large language models. Nature Medicine, pages 1–8, 2025

    2025

  37. [45]

    Generating consistent pddl domains with large language models

    Pavel Smirnov, Frank Joublin, Antonello Ceravola, and Michael Gienger. Generating consistent pddl domains with large language models. arXiv preprint arXiv:2404.07751, 2024

    2024 arXiv

  38. [46]

    Mazebase: A sandbox for learning from games

    Sainbayar Sukhbaatar, Arthur Szlam, Gabriel Synnaeve, Soumith Chintala, and Rob Fergus. Mazebase: A sandbox for learning from games. arXiv preprint arXiv:1511.07401, 2015

    2015 arXiv

  39. [47]

    Value iteration networks

    Aviv Tamar, Yi Wu, Garrett Thomas, Sergey Levine, and Pieter Abbeel. Value iteration networks. Advances in neural information processing systems, 29, 2016

    2016

  40. [48]

    Grasp: A grid-based benchmark for evaluating common- sense spatial reasoning

    Zhisheng Tang and Mayank Kejriwal. Grasp: A grid-based benchmark for evaluating common- sense spatial reasoning. arXiv preprint arXiv:2407.01892, 2024

    2024 arXiv

  41. [49]

    Grid-based mapping and tracking in dynamic environments using a uniform evidential environment representation

    Georg Tanzmeister, Julian Thomas, Dirk Wollherr, and Martin Buss. Grid-based mapping and tracking in dynamic environments using a uniform evidential environment representation. In 2014 IEEE International Conference on Robotics and Automation (ICRA), pages 6090–6095, 2014. 12

    2014

  42. [50]

    Codelutra: Boosting llm code generation via preference-guided refinement

    Leitian Tao, Xiang Chen, Tong Yu, Tung Mai, Ryan Rossi, Yixuan Li, and Saayan Mitra. Codelutra: Boosting llm code generation via preference-guided refinement. arXiv preprint arXiv:2411.05199, 2024

    2024 arXiv

  43. [51]

    Evaluating large language models with grid-based game competitions: an extensible llm benchmark and leaderboard

    Oguzhan Topsakal, Colby Jacob Edell, and Jackson Bailey Harper. Evaluating large language models with grid-based game competitions: an extensible llm benchmark and leaderboard. arXiv preprint arXiv:2407.07796, 2024

    2024 arXiv

  44. [52]

    Care: A collision-aware mobile robot navigation in grid environment using improved breadth first search

    Hrudaya Kumar Tripathy, Sushruta Mishra, Hiren Kumar Thakkar, and Deepak Rai. Care: A collision-aware mobile robot navigation in grid environment using improved breadth first search. Computers & Electrical Engineering, 94:107327, 2021

    2021

  45. [53]

    On the planning abilities of large language models-a critical investigation

    Karthik Valmeekam, Matthew Marquez, Sarath Sreedharan, and Subbarao Kambhampati. On the planning abilities of large language models-a critical investigation. Advances in Neural Information Processing Systems, 36:75993–76005, 2023

    2023

  46. [54]

    Llm^ 3: Large language model-based task and motion planning with motion failure reasoning

    Shu Wang, Muzhi Han, Ziyuan Jiao, Zeyu Zhang, Ying Nian Wu, Song-Chun Zhu, and Hangxin Liu. Llm^ 3: Large language model-based task and motion planning with motion failure reasoning. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1208...

    2024

  47. [55]

    Executable code actions elicit better llm agents

    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better llm agents. In Forty-first International Conference on Machine Learning, 2024

    2024

  48. [56]

    Codet5+: Open code large language models for code understanding and generation

    Yue Wang, Hung Le, Akhilesh Deepak Gotmare, Nghi DQ Bui, Junnan Li, and Steven CH Hoi. Codet5+: Open code large language models for code understanding and generation. arXiv preprint arXiv:2305.07922, 2023

    2023 arXiv

  49. [57]

    Q-learning

    Christopher JCH Watkins and Peter Dayan. Q-learning. Machine learning, 8:279–292, 1992

    1992

  50. [58]

    Codearc: Benchmarking reasoning capabilities of llm agents for inductive program synthesis

    Anjiang Wei, Tarun Suresh, Jiannan Cao, Naveen Kannan, Yuheng Wu, Kai Yan, Thiago SFX Teixeira, Ke Wang, and Alex Aiken. Codearc: Benchmarking reasoning capabilities of llm agents for inductive program synthesis. arXiv preprint arXiv:2503.23145, 2025

    2025 arXiv

  51. [59]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  52. [60]

    Creative robot tool use with large language models

    Mengdi Xu, Peide Huang, Wenhao Yu, Shiqi Liu, Xilun Zhang, Yaru Niu, Tingnan Zhang, Fei Xia, Jie Tan, and Ding Zhao. Creative robot tool use with large language models. arXiv preprint arXiv:2310.13065, 2023

    2023 arXiv

  53. [61]

    Iglu gridworld: Simple and fast environment for embodied dialog agents

    Artem Zholus, Alexey Skrynnik, Shrestha Mohanty, Zoya V olovikova, Julia Kiseleva, Ar- tur Szlam, Marc-Alexandre Coté, and Aleksandr I Panov. Iglu gridworld: Simple and fast environment for embodied dialog agents. arXiv preprint arXiv:2206.00142, 2022

    2022 arXiv

  54. [62]

    ""Checks if a coordinate is within bounds and not an obstacle

    Changzhi Zhou, Xinyu Zhang, Dandan Song, Xiancai Chen, Wanli Gu, Huipeng Ma, Yuhang Tian, Mengdi Zhang, and Linmei Hu. Refinecoder: Iterative improving of large language models via adaptive critique refinement for code generation. arXiv preprint arXiv:2502.09183, 2025. 13 A Ap...

    2025 arXiv

  55. [63]

    There is a dedicated bullet point in the introduction for contributions, with further details throughout the paper

    Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? Answer: [Yes] Justification: The abstract and introduction clearly state the main contributions of the paper. There is a dedicated bullet point in ...

  56. [64]

    Limitations

    Limitations Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: There is a section included specifically for discussion and limitations. Guidelines: • The answer NA means that the paper has no limitation while the...

  57. [65]

    • All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced

    Theory assumptions and proofs Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? 62 Answer: [NA] Justification: [NA] Guidelines: • The answer NA means that the paper does not include theoretical results...

  58. [66]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experimental result reproducibility Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and da...

  59. [67]

    Guidelines: • The answer NA means that paper does not include experiments requiring code

    Open access to data and code 63 Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: We include all prompts u...

  60. [68]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experimental setting/details Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results? Answer: [Yes] Justification: We detail these in the experi...

  61. [69]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiment statistical significance Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [Yes] Justification: Measures of statistical bound are provided with th...

  62. [70]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiments compute resources Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification: Cost of LLM prompting i...

  63. [71]

    Guidelines: • The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics

    Code of ethics Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? Answer: [Yes] Justification: The research conforms to the NeurIPS Code of Ethics. Guidelines: • The answer N...

  64. [72]

    Guidelines: • The answer NA means that there is no societal impact of the work performed

    Broader impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: Details are provided in the discussion and limitation section. Guidelines: • The answer NA means that the...

  65. [73]

    Safeguards Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: [NA] Guidelines...

  66. [74]

    Guidelines: • The answer NA means that the paper does not use existing assets

    Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: The two benchmar...

  67. [75]

    • Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates

    New assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [NA] Justification: [NA] Guidelines: • The answer NA means that the paper does not release new assets. • Researchers should communicate t...

  68. [76]

    Crowdsourcing and research with human subjects Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? A...

  69. [77]

    • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research

    Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals...

  70. [78]

    Answer: [Yes] Justification: LLMs usage are detailed in the experimental setup section

    Declaration of LLM usage 67 Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact t...

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.