{"id":"756b8618-6271-415b-b649-a4ed1b0ee31f","arxiv_id":"2504.21276","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"An empirical evaluation of six LLMs on path-planning code generation finds frequent semantic errors, universal failure on Voronoi-based planning, and concludes the code is unsafe for safety-critical use without rigorous testing.","lead":"This paper tests six large language models on their ability to write code for three path-planning algorithms and finds that most models fail on at least one algorithm, with none able to correctly implement Voronoi diagram planning. The authors conclude that LLM-generated code is hazardous for safety-critical navigation and should only be used after rigorous human testing.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central hazard claim is undercut by task-design artifacts: RRT is tested on a discrete grid and Voronoi on an advanced generalized-diagram task, with no human baseline, so failures may not generalize.","rationale":"The reader's CONDITIONAL verdict is reasonable, but the weakest assumption is more specific than map diversity. The paper's strongest evidence of hazard comes from the complete failure of all LLMs on Voronoi and the RRT semantic errors. Both tasks are arguably mismatched to the algorithms' intended domains or unusually hard for a one-sentence prompt. Forcing RRT onto a discrete 20x20 grid is not how RRT is used in safety-critical robotics; the grid discretization and the requirement that path and obstacles be separated by 0.5 may create edge cases that are artifacts of the representation. Likewise, the generalized Voronoi diagram of polygonal obstacles is an advanced computational geometry problem; even the paper's own algorithm description in Section 3.1 is a simplified sketch. Without showing that competent human programmers can reliably satisfy these prompts, the 0/6 Voronoi result does not by itself establish that LLM-generated path planning code is especially hazardous. A human baseline is the cleanest way to settle causal attribution. The map-diversity concern raised by the reader is real but secondary: even if maps were diverse, the task framing issue would remain. Thus I agree partially with the reader and would keep the verdict CONDITIONAL pending this check.","tokens_in":9592,"tokens_out":10550,"duration_ms":124653,"concrete_test":"Run a controlled replication using the same six LLMs and exact prompts on (a) a continuous 2D obstacle field with a non-holonomic vehicle model for RRT and a standard point-based Voronoi roadmap task with an automatically checkable exact solution, and (b) the original prompts given to several human programmers under the same scoring rubric. If LLM failure rates drop sharply in the continuous/standard settings, or if humans also fail the original Voronoi prompt at similar rates, then the paper's conclusion is an artifact of its task design rather than a general property of LLM-generated path planning code.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that LLM-generated path planning code is hazardous and should not be used in safety-critical contexts. The support consists of failure rates from three static 20x20 grid maps with rectangular obstacles (Section 3.2, Figure 1). Two of the three algorithms are evaluated in unnatural regimes: RRT is a continuous-space sampling algorithm, but Section 4.3 evaluates it on a discrete grid with diagonal moves and a 0.5-gap rule; Voronoi is evaluated as construction of a generalized Voronoi diagram for polygonal obstacles, a specialized routine the paper concedes none of the LLMs produced even via the Scipy library (Section 4.2). The observed failures (boundary violations, circular trees, 0/6 Voronoi) may therefore be artifacts of task framing rather than evidence about LLM-generated path planning code. There is no human baseline or standard-library baseline on the same prompts, so the paper cannot distinguish 'LLMs are unreliable' from 'these under-specified advanced tasks are hard for anyone.' Since the Voronoi and RRT results drive the hazard conclusion, this gap is load-bearing.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports an evaluation of six LLMs (Mistral 7B, Gemini, DeepSeek, GPT-4o, Copilot, and Grok 3) generating Python implementations of three path-planning algorithms: Dijkstra's algorithm, a Voronoi-based planner, and RRT. The models are prompted with a natural-language task specification on 20x20 grids with rectangular obstacles, under two prompt variants (any-path and shortest-path), and are allowed up to five regeneration attempts. The authors manually fix syntax errors, judge output viability, and then evaluate algorithm correctness, satisfaction of prompt requirements, and code style via Pylint. The main finding is that most LLMs implement Dijkstra reasonably well, none successfully implements the generalized Voronoi algorithm, and several RRT implementations exhibit boundary violations, circular trees, or infinite loops. The paper concludes that LLM-generated code presents serious hazards for path planning and should not be used in safety-critical contexts without rigorous testing.","tokens_in":9807,"tokens_out":4996,"duration_ms":52196,"significance":"The paper addresses a relevant and under-explored question: whether LLM-generated code is reliable for safety-critical path planning. Its strengths are a clear algorithm-focused study design, specific bug identification (e.g., circular RRT trees, incorrect diagonal cost in Dijkstra, misuse of SciPy k-d trees), and a hazard-analysis discussion that goes beyond simple pass/fail coding benchmarks. If the external-validity limitations were addressed, the results would be a useful contribution to the growing literature on LLM code reliability. However, the current evidence base is too narrow to fully support the broad safety-critical conclusion, and several protocol decisions reduce the strength of the quantitative claims. The paper is a reasonable starting point but needs substantial revision to make the hazard claim load-bearing.","major_comments":[{"comment":"The experiment uses only three static 20x20 grid maps with rectangular obstacles, yet the conclusion in Section 6 is a blanket statement about path planning applications and safety-critical contexts. No evidence is provided that this small set of maps is representative of the environments faced by drones, vehicles, or surgical robots, and no human or established-library baseline is run on the same prompts. Consequently, the reported failure rates (e.g., 0/6 for Voronoi, RRT success rates in Table 4) cannot by themselves support the broad hazard claim. The paper should either restrict the conclusion to the tested setting or add baselines and a wider range of maps and task types.","section":"Section 3.2, Figure 1"},{"comment":"RRT is a continuous-space sampling algorithm, but it is tested on a discrete 20x20 grid with diagonal moves and a 0.5-gap clearance rule (prompt item 7). This is an unusual evaluation regime for RRT: the grid and clearance constraint change the nature of nearest-neighbor and collision checks, and the observed boundary violations and circular trees may be partly induced by the discrete task framing. Because the RRT results are a large part of the hazard conclusion, the paper needs to justify this adaptation, compare with a standard RRT implementation on a continuous map and on the same grid, or report the failure modes separately to make clear which errors are intrinsic to the algorithm generation versus induced by the test harness.","section":"Section 4.3, Table 4"},{"comment":"The protocol allows up to five generations per LLM-prompt combination and manually fixes syntax errors before evaluation, and viability is judged by a single manual inspection. This means the reported results actually measure a human-LLM collaboration pipeline; a code that succeeds only after manual syntax repair cannot be attributed to the LLM alone. The manuscript does not state who performed the fixes, how many raters assessed viability, or how often the least-flawed-of-five selection was used. These details are load-bearing because the central claim is about the hazards of using LLM output without rigorous testing; as written, the reader cannot tell what the raw LLM failure rate is. I encourage the authors to report raw code results and post-fix results separately.","section":"Section 3.2, Tables 2-4"},{"comment":"The claim that LLM performance roughly corresponds to the amount of available code is not supported by the data shown. Search-result counts on Stack Overflow and GitHub are not a measure of what is in LLM training corpora, no statistical relationship is computed, and the paper does not control for confounds such as algorithm inherent difficulty. Since this claim is used to infer that LLMs do not understand algorithms or coding in general, it should be labeled as a hypothesis or replaced with a direct analysis of training data. This does not affect the hazard conclusion directly, but it is presented as an empirical finding and should be corrected.","section":"Section 5, Table 6"}],"minor_comments":[{"comment":"There is a typo in 'LLMs's ability' that should be 'LLMs' ability'; also, 'V oronoi' appears with an erroneous space throughout the text and should be 'Voronoi'.","section":"Section 1"},{"comment":"The text refers to 'subsection 4.1.2' and 'subsection 4.1.3' when discussing RRT hyperparameters and goal-biased sampling, but those subsections do not exist in the manuscript; the references should be corrected to the relevant sections or appendix entries.","section":"Section 4.3.1"},{"comment":"The column heading 'Req 1' is not defined before use; the text mentions 'Requirement 1', but the mapping between the table heading and the requirement should be explicit.","section":"Table 4"},{"comment":"The color legend uses 'yellow cells indicate partial success or problems that are relatively easy to fix', but the threshold for 'relatively easy' is never defined; the paper should specify the criterion used for yellow versus green versus red.","section":"Tables 2-4"},{"comment":"The paper would benefit from a supplemental artifact containing the exact prompts and the generated code for each LLM and trial, since manual fixes and regeneration attempts otherwise make the evaluation difficult to reproduce.","section":"Appendix D"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, the short version: this paper is a useful empirical study of six LLMs generating three path-planning algorithms, and the specific failure modes it documents are the most valuable part. But the headline hazard claim is broader than the evidence supports, mainly because there is no human baseline or standard-library baseline on the same prompts.\n\nThe design is a step beyond the usual competition-style benchmarks: no function signatures, multiple functions, a natural language spec, and evaluation on three maps. The results are genuinely interesting. Dijkstra is mostly fine; Voronoi fails for all LLMs; RRT shows boundary violations, circular trees, and infinite loops. The correlation with online code availability is plausible and worth noting, and the Pylint evaluation adds a useful dimension.\n\nThe soft spots are mostly about interpretation. The Voronoi task is effectively constructing a generalized Voronoi diagram for polygonal obstacles, which is a non-trivial specialized routine. RRT is evaluated on a discrete grid with diagonal moves and a 0.5-gap rule, which is not RRT's native setting. No human coder or standard library implementation was tested on the same prompts, so we cannot tell whether these tasks are genuinely hard for anyone, not just LLMs. The authors also used best-of-five selection and manually fixed syntax errors, which are atypical evaluation choices, and they did not release code or data. The three maps are small and static. These issues do not invalidate the descriptive findings, but they undercut the strong conclusion that LLM-generated code presents serious hazards for path planning applications in general.\n\nI would send this to peer review, because the topic is important and the failure modes are new and concrete. But I would ask for major revision: add a baseline (human or library-based), temper the generalization, and release the artifacts. If the authors do that, the paper becomes a solid reference; if not, it is a cautionary note with limited scope.\n\nFor my own work, I would probably cite it as an example of LLM code generation evaluation, with caveats.","headline":"Useful empirical study of LLM code generation for path planning, but the safety-critical hazard claim is undercut by task-design artifacts and a missing baseline.","tokens_in":10291,"tokens_out":4517,"would_cite":true,"duration_ms":45104,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LLM-generated path planning code is not safe to use in safety-critical systems without rigorous human testing.","keywords":["LLM code generation","path planning","safety-critical software","Dijkstra's algorithm","Voronoi diagrams","Rapidly-exploring Random Trees","code benchmarks","human-in-the-loop verification"],"falsifier":"Run the same six models on a larger standardized suite of maps that includes non-rectangular and dynamic obstacles, continuous state spaces, and real robot motion constraints; if a large majority of outputs are viable without human fixes, the safety warning would not generalize. A narrower check: find any model that correctly adapts a Voronoi library to polygonal obstacles, which would break the claim that LLMs cannot reason about library use.","tokens_in":9406,"feed_emoji":"🧭","tokens_out":5763,"duration_ms":52436,"temperature":0.7,"pith_summary":"Path planning is a safety-critical function for drones, vehicles, and surgical robots, and this paper asks whether LLM-generated code can do it. Six LLMs were prompted to write Dijkstra, Voronoi, and RRT planners for three increasingly hard 20x20 grid maps, with up to five regenerations allowed. The result is a warning: no model produced a working Voronoi planner, RRT code showed semantic errors like infinite loops and paths leaving the grid, and only some models produced correct shortest paths with Dijkstra. The paper concludes that LLM-generated code should not be applied in safety-critical path planning without rigorous human testing, and that LLMs cannot replace human engineers in these contexts.","feed_headline":"LLM path planners fail safety-critical tests","feed_subtitle":"None of six models built a working Voronoi planner; RRT code had boundary bugs and infinite loops.","key_machinery":"The load-bearing object is the evaluation protocol itself: three path-planning algorithms of increasing complexity (Dijkstra, Voronoi, and RRT) tested on three 20x20 maps with rectangular obstacles, under two prompt conditions, 'a path' versus 'the shortest path'. A generated output is viable only if it compiles, runs, begins and ends at the requested points, stays inside the grid, and keeps a 0.5-unit gap from obstacles; failures trigger up to five regeneration attempts. The mechanism that explains the results is the comparison with online code availability: models succeeded most with Dijkstra and failed entirely with Voronoi, matching the volume of Stack Overflow and GitHub code available for each algorithm. This supports the paper's claim that LLMs pattern-match training code rather than reason about algorithms or library APIs.","core_discovery":"Across six LLMs and three path-planning algorithms, tested on three grid maps, the paper finds that LLM-generated code is not reliable enough for safety-critical use. None of the models successfully generated a generalized Voronoi planner; most substituted A* or Dijkstra, or imported SciPy's Voronoi library without adapting it to polygonal obstacles. Dijkstra implementations mostly worked, but only some always computed the shortest path; RRT code showed semantic errors such as paths landing outside the grid, circular parent-child trees that caused infinite loops, and incomplete path reconstruction. The paper argues from a source-mechanism-outcome hazard analysis that such code should not be applied in safety-critical contexts without rigorous testing.","pith_inferences":["A likely extension: library-adaptation failure will recur in any safety-critical algorithm that requires nontrivial API use, such as visibility graphs or Delaunay-based planning, because none of the models adapted SciPy's Voronoi function to polygonal obstacles.","A testable extension: measure the human repair cost in edits and time for each model's output; if repair costs exceed from-scratch implementation, the economic case for LLM code in safety-critical settings weakens even where the output is viable.","The one-word prompt sensitivity implies that in practice, requirement ambiguity rather than raw model capability may dominate risk; a standardized prompt harness with explicit constraints could reduce, but not eliminate, failures."],"forward_implications":["Teams integrating LLM-written planners into drones, vehicles, or surgical robots must budget for full-cycle human verification; none of the six models produced a working Voronoi planner, and most RRT code required debugging.","Prompt wording is a reliability risk: changing one phrase from 'a path' to 'the shortest path' changed default hyperparameters and objectives for several models, so identical prompts must be tested repeatedly before deployment.","Dijkstra-style shortest-path code is comparatively safer: only some models produced correct shortest paths on all maps, with failures including a hard-map index error and an incorrect diagonal cost calculation.","Code style scores fail production-ready standards on the Pylint scale, so maintainability, not just correctness, is an additional human burden.","The strong correlation between online code availability and success implies that LLM-based code generation will improve unevenly: heavily represented algorithms will keep working, while niche safety-critical algorithms remain unreliable."],"supporting_citations":[{"why":"The HumanEval benchmark this study deliberately differs from: no function signature is provided and multiple helper functions are required.","marker":"[6]"},{"why":"The SWE-bench task-based benchmark is contrasted with; the authors avoid its leaked-solution problem by writing their own prompts.","marker":"[13]"},{"why":"Supplies the three increasingly difficult 20x20 grid maps used to test every generated algorithm.","marker":"[11]"},{"why":"Defines Dijkstra's algorithm, one of the three target algorithms and the one most models implemented successfully.","marker":"[7]"},{"why":"Defines Voronoi diagrams, the target algorithm none of the six models implemented correctly.","marker":"[5]"},{"why":"Defines RRT, the target algorithm where semantic errors such as infinite loops and boundary violations appeared.","marker":"[15]"},{"why":"The source-mechanism-outcome hazard-analysis framework used to argue that latent code errors can lead to crashes and accidents.","marker":"[10]"},{"why":"Goal-biased RRT technique that explains one model's 100 percent success rate and grounds the discussion of default optimization choices.","marker":"[16]"}],"fun_headline_variants":["LLM code unsafe for path planning, study finds","Voronoi planner: zero LLMs succeed","RRT code bugs plague LLM path planners","LLM path planning: not ready for safety-critical","Six LLMs fail Voronoi, RRT buggy in tests"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that three hand-selected 20x20 grid maps with rectangular obstacles and a manual viability check stand in for the variety, scale, and failure modes of real safety-critical path planning environments.","fun_headline_variants_meta":{"raw":{"variants":["LLM code unsafe for path planning, study finds","Voronoi planner: zero LLMs succeed","RRT code bugs plague LLM path planners","LLM path planning: not ready for safety-critical","Six LLMs fail Voronoi, RRT buggy in tests"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000143,"raw_usage":{"total_tokens":1079,"prompt_tokens":759,"completion_tokens":320,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":375,"completion_tokens_details":{"reasoning_tokens":242}},"tokens_in":375,"tokens_out":320,"duration_ms":3575,"temperature":1.0,"reasoning_tokens":242,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:07:16.726799+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same six models on a larger standardized suite of maps that includes non-rectangular and dynamic obstacles, continuous state spaces, and real robot motion constraints; if a large majority of outputs are viable without human fixes, the safety warning would not generalize. A narrower check: find any model that correctly adapts a Voronoi library to polygonal obstacles, which would break the claim that LLMs cannot reason about library use.","supporting_citations":[{"cited_title":"A path-planning approach based on potential and dynamic q-learning for mobile robots in unknown environment","cited_arxiv_id":null,"evidence_quote":"Supplies the three increasingly difficult 20x20 grid maps used to test every generated algorithm."},{"cited_title":"A note on two problems in connexion with graphs","cited_arxiv_id":null,"evidence_quote":"Defines Dijkstra's algorithm, one of the three target algorithms and the one most models implemented successfully."},{"cited_title":"V oronoi diagrams—a survey of a fundamental geometric data structure","cited_arxiv_id":null,"evidence_quote":"Defines Voronoi diagrams, the target algorithm none of the six models implemented correctly."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines RRT, the target algorithm where semantic errors such as infinite loops and boundary violations appeared."},{"cited_title":"E., E VERHART , K., S TEVENS , R., B ABBITT III, N., C LEMENS , P., AND STOUT, L","cited_arxiv_id":null,"evidence_quote":"The source-mechanism-outcome hazard-analysis framework used to argue that latent code errors can lead to crashes and accidents."},{"cited_title":"M., AND KUFFNER JR, J","cited_arxiv_id":null,"evidence_quote":"Goal-biased RRT technique that explains one model's 100 percent success rate and grounds the discussion of default optimization choices."}],"review_version":1}