Pith. sign in

REVIEW 4 major objections 5 minor 23 references

Assessing LLM code generation quality through path planning tasks

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLM-generated path planning code is not safe to use in safety-critical systems without rigorous human testing.

desk verdict Useful empirical study of LLM code generation for path planning, but the safety-critical hazard claim is undercut by task-design artifacts and a missing baseline. read the letter →

arxiv 2504.21276 v1 pith:KRIBY24I submitted 2025-04-30 cs.SE cs.AI

Wanyi Chen , Meng-Wen Su , Mary L. Cummings This is my paper · ORCID
classification cs.SEcs.AI
keywords LLMcodegenerationpathplanningsafety-criticalsoftwareDijkstra'salgorithmVoronoidiagramsRapidly-exploringRandomTreesbenchmarkshuman-in-the-loopverification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Path planning is a safety-critical function for drones, vehicles, and surgical robots, and this paper asks whether LLM-generated code can do it. Six LLMs were prompted to write Dijkstra, Voronoi, and RRT planners for three increasingly hard 20x20 grid maps, with up to five regenerations allowed. The result is a warning: no model produced a working Voronoi planner, RRT code showed semantic errors like infinite loops and paths leaving the grid, and only some models produced correct shortest paths with Dijkstra. The paper concludes that LLM-generated code should not be applied in safety-critical path planning without rigorous human testing, and that LLMs cannot replace human engineers in these contexts.

What carries the argument

The load-bearing object is the evaluation protocol itself: three path-planning algorithms of increasing complexity (Dijkstra, Voronoi, and RRT) tested on three 20x20 maps with rectangular obstacles, under two prompt conditions, 'a path' versus 'the shortest path'. A generated output is viable only if it compiles, runs, begins and ends at the requested points, stays inside the grid, and keeps a 0.5-unit gap from obstacles; failures trigger up to five regeneration attempts. The mechanism that explains the results is the comparison with online code availability: models succeeded most with Dijkstra and failed entirely with Voronoi, matching the volume of Stack Overflow and GitHub code available for each algorithm. This supports the paper's claim that LLMs pattern-match training code rather than reason about algorithms or library APIs.

What would settle it

Run the same six models on a larger standardized suite of maps that includes non-rectangular and dynamic obstacles, continuous state spaces, and real robot motion constraints; if a large majority of outputs are viable without human fixes, the safety warning would not generalize. A narrower check: find any model that correctly adapts a Voronoi library to polygonal obstacles, which would break the claim that LLMs cannot reason about library use.

Watch

Extended reading notes

Core claim

Across six LLMs and three path-planning algorithms, tested on three grid maps, the paper finds that LLM-generated code is not reliable enough for safety-critical use. None of the models successfully generated a generalized Voronoi planner; most substituted A* or Dijkstra, or imported SciPy's Voronoi library without adapting it to polygonal obstacles. Dijkstra implementations mostly worked, but only some always computed the shortest path; RRT code showed semantic errors such as paths landing outside the grid, circular parent-child trees that caused infinite loops, and incomplete path reconstruction. The paper argues from a source-mechanism-outcome hazard analysis that such code should not be applied in safety-critical contexts without rigorous testing.

Load-bearing premise

The argument assumes that three hand-selected 20x20 grid maps with rectangular obstacles and a manual viability check stand in for the variety, scale, and failure modes of real safety-critical path planning environments.

Editorial extensions

If this is right

  • Teams integrating LLM-written planners into drones, vehicles, or surgical robots must budget for full-cycle human verification; none of the six models produced a working Voronoi planner, and most RRT code required debugging.
  • Prompt wording is a reliability risk: changing one phrase from 'a path' to 'the shortest path' changed default hyperparameters and objectives for several models, so identical prompts must be tested repeatedly before deployment.
  • Dijkstra-style shortest-path code is comparatively safer: only some models produced correct shortest paths on all maps, with failures including a hard-map index error and an incorrect diagonal cost calculation.
  • Code style scores fail production-ready standards on the Pylint scale, so maintainability, not just correctness, is an additional human burden.
  • The strong correlation between online code availability and success implies that LLM-based code generation will improve unevenly: heavily represented algorithms will keep working, while niche safety-critical algorithms remain unreliable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A likely extension: library-adaptation failure will recur in any safety-critical algorithm that requires nontrivial API use, such as visibility graphs or Delaunay-based planning, because none of the models adapted SciPy's Voronoi function to polygonal obstacles.
  • A testable extension: measure the human repair cost in edits and time for each model's output; if repair costs exceed from-scratch implementation, the economic case for LLM code in safety-critical settings weakens even where the output is viable.
  • The one-word prompt sensitivity implies that in practice, requirement ambiguity rather than raw model capability may dominate risk; a standardized prompt harness with explicit constraints could reduce, but not eliminate, failures.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper reports an evaluation of six LLMs (Mistral 7B, Gemini, DeepSeek, GPT-4o, Copilot, and Grok 3) generating Python implementations of three path-planning algorithms: Dijkstra's algorithm, a Voronoi-based planner, and RRT. The models are prompted with a natural-language task specification on 20x20 grids with rectangular obstacles, under two prompt variants (any-path and shortest-path), and are allowed up to five regeneration attempts. The authors manually fix syntax errors, judge output viability, and then evaluate algorithm correctness, satisfaction of prompt requirements, and code style via Pylint. The main finding is that most LLMs implement Dijkstra reasonably well, none successfully implements the generalized Voronoi algorithm, and several RRT implementations exhibit boundary violations, circular trees, or infinite loops. The paper concludes that LLM-generated code presents serious hazards for path planning and should not be used in safety-critical contexts without rigorous testing.

Significance. The paper addresses a relevant and under-explored question: whether LLM-generated code is reliable for safety-critical path planning. Its strengths are a clear algorithm-focused study design, specific bug identification (e.g., circular RRT trees, incorrect diagonal cost in Dijkstra, misuse of SciPy k-d trees), and a hazard-analysis discussion that goes beyond simple pass/fail coding benchmarks. If the external-validity limitations were addressed, the results would be a useful contribution to the growing literature on LLM code reliability. However, the current evidence base is too narrow to fully support the broad safety-critical conclusion, and several protocol decisions reduce the strength of the quantitative claims. The paper is a reasonable starting point but needs substantial revision to make the hazard claim load-bearing.

major comments (4)
  1. [Section 3.2, Figure 1] The experiment uses only three static 20x20 grid maps with rectangular obstacles, yet the conclusion in Section 6 is a blanket statement about path planning applications and safety-critical contexts. No evidence is provided that this small set of maps is representative of the environments faced by drones, vehicles, or surgical robots, and no human or established-library baseline is run on the same prompts. Consequently, the reported failure rates (e.g., 0/6 for Voronoi, RRT success rates in Table 4) cannot by themselves support the broad hazard claim. The paper should either restrict the conclusion to the tested setting or add baselines and a wider range of maps and task types.
  2. [Section 4.3, Table 4] RRT is a continuous-space sampling algorithm, but it is tested on a discrete 20x20 grid with diagonal moves and a 0.5-gap clearance rule (prompt item 7). This is an unusual evaluation regime for RRT: the grid and clearance constraint change the nature of nearest-neighbor and collision checks, and the observed boundary violations and circular trees may be partly induced by the discrete task framing. Because the RRT results are a large part of the hazard conclusion, the paper needs to justify this adaptation, compare with a standard RRT implementation on a continuous map and on the same grid, or report the failure modes separately to make clear which errors are intrinsic to the algorithm generation versus induced by the test harness.
  3. [Section 3.2, Tables 2-4] The protocol allows up to five generations per LLM-prompt combination and manually fixes syntax errors before evaluation, and viability is judged by a single manual inspection. This means the reported results actually measure a human-LLM collaboration pipeline; a code that succeeds only after manual syntax repair cannot be attributed to the LLM alone. The manuscript does not state who performed the fixes, how many raters assessed viability, or how often the least-flawed-of-five selection was used. These details are load-bearing because the central claim is about the hazards of using LLM output without rigorous testing; as written, the reader cannot tell what the raw LLM failure rate is. I encourage the authors to report raw code results and post-fix results separately.
  4. [Section 5, Table 6] The claim that LLM performance roughly corresponds to the amount of available code is not supported by the data shown. Search-result counts on Stack Overflow and GitHub are not a measure of what is in LLM training corpora, no statistical relationship is computed, and the paper does not control for confounds such as algorithm inherent difficulty. Since this claim is used to infer that LLMs do not understand algorithms or coding in general, it should be labeled as a hypothesis or replaced with a direct analysis of training data. This does not affect the hazard conclusion directly, but it is presented as an empirical finding and should be corrected.
minor comments (5)
  1. [Section 1] There is a typo in 'LLMs's ability' that should be 'LLMs' ability'; also, 'V oronoi' appears with an erroneous space throughout the text and should be 'Voronoi'.
  2. [Section 4.3.1] The text refers to 'subsection 4.1.2' and 'subsection 4.1.3' when discussing RRT hyperparameters and goal-biased sampling, but those subsections do not exist in the manuscript; the references should be corrected to the relevant sections or appendix entries.
  3. [Table 4] The column heading 'Req 1' is not defined before use; the text mentions 'Requirement 1', but the mapping between the table heading and the requirement should be explicit.
  4. [Tables 2-4] The color legend uses 'yellow cells indicate partial success or problems that are relatively easy to fix', but the threshold for 'relatively easy' is never defined; the paper should specify the criterion used for yellow versus green versus red.
  5. [Appendix D] The paper would benefit from a supplemental artifact containing the exact prompts and the generated code for each LLM and trial, since manual fixes and regeneration attempts otherwise make the evaluation difficult to reproduce.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical benchmark whose conclusions are direct measurements, not derived from fitted inputs or self-citations.

full rationale

This paper makes no derivation claim that reduces to its own inputs. It prompts six LLMs to generate three path-planning algorithms, runs the code on three maps, and manually records syntax errors, semantic errors, success rates, and Pylint scores (Sections 3.2, 4.1-4.4). The central claim that LLM-generated code is hazardous for safety-critical path planning is an inductive generalization from observed failures, not a quantity predicted from fitted parameters. No parameter is fit to a subset of the data and then relabeled as a prediction. No load-bearing conclusion is justified by a citation to the authors' own prior work; the maps are attributed to Hao et al. [11] and the Voronoi and RRT background to external references [5, 15]. The Section 5.2 limitations (domain dependence, unassessed cybersecurity issues, and the evolving capability of LLMs) are external-validity caveats rather than admissions of circular reasoning; likewise, the absence of a human baseline or the use of discrete 20x20 grids affects whether failures generalize, not whether the measurements are independent of the conclusions. Because the empirical results are self-contained and externally checkable, no circular step can be exhibited.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The ledger is empty of invented entities. The free parameters reflect protocol choices, not fitted model parameters. The axioms are domain assumptions about representativeness and measurement validity that carry the generalization from observed outputs to the safety-critical hazard claim.

free parameters (2)
  • Max regeneration attempts = 5
    Chosen protocol limit; allows best-of-five selection which may overstate LLM capability and understate hazard.
  • RRT success threshold = 70% (used in Table 4 color coding)
    Hand-picked cutoff separating 'partial success' from 'failure'; not statistically derived.
assumptions (6)
  • domain assumption The three maps from Hao et al. [11] are representative of safety-critical path planning difficulty
    Used as the only test environments (Section 3.2); no justification for representativeness.
  • domain assumption A 20x20 grid with rectangular obstacles is a sufficient proxy for real safety-critical path planning
    Underlies all prompts and the hazard conclusion; real environments involve continuous motion, dynamics, uncertainty and higher dimensions.
  • domain assumption Manual inspection of generated paths is an objective and reliable correctness check
    Authors manually verified viability (Section 3.2); no inter-rater reliability or automated check reported.
  • domain assumption Pylint scores measure code quality relevant to safety
    Used as the code style metric (Section 4.4); relationship between style scores and safety is asserted, not demonstrated.
  • domain assumption Stack Overflow and GitHub search result counts approximate LLM training data availability
    Used in Section 5 to explain performance differences; no evidence links search counts to training corpora.
  • domain assumption The prompts used are a fair test of the LLMs' code generation ability
    No pilot testing or prompt optimization across models; wording differences can change outcomes (authors note this).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Assessing LLM code generation quality through path planning tasks." pith.science (2026). https://pith.science/paper/KRIBY24I

@misc{pith2026250421276,
  author       = {Pith},
  title        = {Pith review of: Assessing LLM code generation quality through path planning tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KRIBY24I}},
  note         = {Machine review of arXiv:2504.21276}
}
read the original abstract

As LLM-generated code grows in popularity, more evaluation is needed to assess the risks of using such tools, especially for safety-critical applications such as path planning. Existing coding benchmarks are insufficient as they do not reflect the context and complexity of safety-critical applications. To this end, we assessed six LLMs' abilities to generate the code for three different path-planning algorithms and tested them on three maps of various difficulties. Our results suggest that LLM-generated code presents serious hazards for path planning applications and should not be applied in safety-critical contexts without rigorous testing.

Figures

Figures reproduced from arXiv: 2504.21276 by the authors.

Figure 1
Figure 1. Increasingly difficult path planning maps [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Experiment process diagram • Syntax correctness: We checked whether the generated code contained any syntax errors, such as missing an import statement, that would prevent the code from compiling. • Algorithm correctness : We assessed whether the generated algorithm could find correct paths on all three maps. “Correct" is defined as having the correct start and end points, not running through obstacles, and staying … view at source ↗
Figure 3
Figure 3. Hazard analysis 5.2 Limitations The utility of any hazard analysis ultimately depends on its domain of application. Problems resulting from poor path planning for small, lightweight food delivery robots will necessarily be less hazardous than those for aircraft that are fuel-constrained. Also, due to project scope constraints, we did not assess any cybersecurity issues of LLM-generated code, which could be another s… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Performance comparison between GPT-4o, which optimized for the shortest path with cor [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 16 canonical work pages

  1. [1]

    https://pypi.org/project/pylint/

    pylint 3.3.4. https://pypi.org/project/pylint/. Accessed: 2025-02-25

    work page 2025

  2. [2]

    https://pylint.readthedocs.io/ en/latest/user_guide/messages/messages_overview.html

    Pylint 4.0.0-dev0 documentation: Messages overview. https://pylint.readthedocs.io/ en/latest/user_guide/messages/messages_overview.html. Accessed: 2025-03-01

    work page 2025

  3. [3]

    M., N NOROM , E., U DDIN , G., AND WANG , S

    ALEITHAN , R., X UE, H., M OHAJER , M. M., N NOROM , E., U DDIN , G., AND WANG , S. Swe-bench+: Enhanced coding benchmark for llms. arXiv preprint arXiv:2410.06992 (2024)

    arXiv 2024

  4. [4]

    ATRAMENTOV , A., AND LAVALLE , S. M. Efficient nearest neighbor searching for motion planning. In Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No. 02CH37292) (2002), vol. 1, IEEE, pp. 632–637

    work page 2002

  5. [5]

    V oronoi diagrams—a survey of a fundamental geometric data structure

    AURENHAMMER , F. V oronoi diagrams—a survey of a fundamental geometric data structure. ACM Comput. Surv. 23, 3 (Sept. 1991), 345–405

    work page 1991

  6. [6]

    CHEN , M., T WOREK , J., J UN, H., Y UAN, Q., P INTO , H. P. D. O., K APLAN , J., E DWARDS , H., B URDA , Y., J OSEPH , N., B ROCKMAN , G., ET AL . Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021)

    arXiv 2021

  7. [7]

    A note on two problems in connexion with graphs

    DIJKSTRA, E. A note on two problems in connexion with graphs. Numerische Mathematik 1 (1959), 269–271

    work page 1959

  8. [8]

    H., B ENTLEY , J

    FRIEDMAN , J. H., B ENTLEY , J. L., AND FINKEL , R. A. An algorithm for finding best matches in logarithmic expected time. ACM Transactions on Mathematical Software (TOMS) 3, 3 (1977), 209–226

    work page 1977

Show all 23 references
  1. [9]

    Programming guidelines

    GENEALOGIAL RESEARCH SOFTWARE . Programming guidelines. https://www. gramps-project.org/wiki/index.php/Programming_guidelines. Accessed: 2025- 03-11

    2025

  2. [10]

    E., E VERHART , K., S TEVENS , R., B ABBITT III, N., C LEMENS , P., AND STOUT, L

    GOLDBERG , B. E., E VERHART , K., S TEVENS , R., B ABBITT III, N., C LEMENS , P., AND STOUT, L. System engineering toolbox for design-oriented engineers. Tech. rep., NASA, 1994

    1994

  3. [11]

    A path-planning approach based on potential and dynamic q-learning for mobile robots in unknown environment

    HAO, B., D U, H., Z HAO, J., Z HANG , J., AND WANG , Q. A path-planning approach based on potential and dynamic q-learning for mobile robots in unknown environment. Computational Intelligence and Neuroscience 2022 (06 2022), 1–12

    2022

  4. [12]

    A survey on large language models for code generation

    JIANG , J., W ANG , F., S HEN , J., K IM, S., AND KIM, S. A survey on large language models for code generation. arXiv preprint arXiv:2406.00515 (2024)

    2024 arXiv

  5. [13]

    E., Y ANG , J., W ETTIG , A., Y AO, S., P EI, K., P RESS , O., AND NARASIMHAN , K

    JIMENEZ , C. E., Y ANG , J., W ETTIG , A., Y AO, S., P EI, K., P RESS , O., AND NARASIMHAN , K. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770 (2023)

    2023 arXiv

  6. [14]

    A hazard analysis framework for code synthesis large language models

    KHLAAF , H., M ISHKIN , P., A CHIAM , J., K RUEGER , G., AND BRUNDAGE , M. A hazard analysis framework for code synthesis large language models. arXiv preprint arXiv:2207.14157 (2022)

    2022 arXiv

  7. [15]

    LAVALLE , S. M. Rapidly-exploring random trees : a new tool for path planning. The annual research report (1998)

    1998

  8. [16]

    M., AND KUFFNER JR, J

    LAVALLE , S. M., AND KUFFNER JR, J. J. Randomized kinodynamic planning. The interna- tional journal of robotics research 20, 5 (2001), 378–400

    2001

  9. [17]

    Exploring and evaluating hallucinations in llm-powered code generation

    LIU, F., L IU, Y., S HI, L., H UANG , H., W ANG , R., Y ANG , Z., Z HANG , L., L I, Z., AND MA, Y. Exploring and evaluating hallucinations in llm-powered code generation. arXiv preprint arXiv:2404.00971 (2024)

    2024

  10. [18]

    M., Z I, Y., G UHA , A., A NDERSON , C

    NGUYEN , S., B ABE , H. M., Z I, Y., G UHA , A., A NDERSON , C. J., AND FELDMAN , M. Q. How beginning programmers and code llms (mis) read each other. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (2024), pp. 1–26

    2024

  11. [19]

    M., H ARMAN , M., AND WANG , M

    OUYANG , S., Z HANG , J. M., H ARMAN , M., AND WANG , M. Llm is like a box of chocolates: the non-determinism of chatgpt in code generation. arXiv e-prints (2023), arXiv–2308

    2023

  12. [20]

    Model evaluation for extreme risks

    SHEVLANE , T., FARQUHAR , S., G ARFINKEL , B., P HUONG , M., W HITTLESTONE , J., L EUNG , J., K OKOTAJLO , D., M ARCHAL , N., A NDERLJUNG , M., K OLT, N., ET AL . Model evaluation for extreme risks. arXiv preprint arXiv:2305.15324 (2023). 10

    2023 arXiv

  13. [21]

    C., AND ANTONIOL , G

    TAMBON , F., M ORADI -DAKHEL , A., N IKANJAM , A., K HOMH , F., D ESMARAIS , M. C., AND ANTONIOL , G. Bugs in large language models generated code: An empirical study. Empirical Software Engineering 30, 3 (2025), 1–48

    2025

  14. [22]

    M., AND RASHID , M

    YOUSUF , M. M., AND RASHID , M. Analysis of the change in bugginess and adaptiveness of python software systems. Multimedia Tools and Applications 81, 30 (2022), 43107–43123

    2022

  15. [23]

    Balanced

    ZAMFIRESCU -PEREIRA , J. D., W ONG , R. Y., H ARTMANN , B., AND YANG , Q. Why johnny can’t prompt: how non-ai experts try (and fail) to design llm prompts. In Proceedings of the 2023 CHI conference on human factors in computing systems (2023), pp. 1–21. 11 A Comparison of the ...

    2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.