{"id":"55dffbb4-fbfb-470a-8388-2e0af0167c2d","arxiv_id":"2504.12755","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A prompt-engineering pipeline in which an LLM produces a high-level plan and Python code that adapts precomputed robot waypoints to natural language commands, demonstrated in simulation.","lead":"This paper uses a large language model, GPT-4o, to turn human instructions such as 'walk at a distance of at least 10 from the fan' into executable Python code that reshapes existing robot trajectories. It offers a training-free, interpretable way for non-experts to adjust robot paths, but the evidence so far is qualitative simulation examples.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim rests on selected qualitative figures; absent success rates, error metrics, and failure cases, the conclusion that the experiments 'demonstrate capabilities' is not yet supported.","rationale":"The reader's weakest assumption identifies the LLM-generated code being correct and general enough as the key premise; my concern is closely related but focuses on the evidence side: the paper never measures whether that premise holds. Both readings point to the same condition, namely that the demonstrated examples must generalize across the command set, but the reader frames it as a method risk while I frame it as a validation gap. The paper deserves credit for disclosing the full prompt, using two in-context examples, and showing that a parameter-free LLM pipeline can produce visibly plausible adapted trajectories on three robot types. That is genuine evidence of feasibility and interpretability, and the method is not internally inconsistent. However, the strongest claim in the abstract and conclusion uses success language, and the only support is a handful of figures without numerical evaluation. The paper's own limitations section concedes the absence of guarantees and the possibility of non-smooth transitions, which makes quantitative smoothness and constraint checks essential. A concrete rerun with explicit tolerances would settle whether the central claim is robust; if the success rates are high, the claim stands as a useful proof-of-concept; if they are low, the claim must be restricted or the method needs a constraint-checking layer. Since the reader's verdict is already CONDITIONAL and the requested additions match the test I propose, no change to the verdict is needed.","tokens_in":7720,"tokens_out":4360,"duration_ms":50773,"concrete_test":"Re-run the Appendix A.2 prompt on the full Appendix A.1 command list plus the LaTTe subset across the three simulators, using GPT-4o at temperature 0.1 and at least five seeds. Record: (i) the fraction of runs where the generated code executes without error; (ii) constraint-success rates with explicit tolerances, e.g., for 'Go left by 20' require the final waypoint's x-coordinate to shift by 20 ± 2 and intermediate shifts to be gradual; for 'at least 10 from the box' require no waypoint within the prohibited region; for speed commands require proportional velocity changes near the referenced object; (iii) a smoothness metric such as maximum curvature or total variation of the adapted trajectory. If execution or success rates fall below 80%, the 'demonstrate capabilities' claim must be weakened or scoped to the successful subset; if they remain above 80%, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that simulation experiments demonstrate successful free-form trajectory adaptation without fine-tuning. The load-bearing premise is that the three example figures in Section 4.3 and the result figures in Section 4.4 are representative of the full command set rather than favorable selections. Section 4.4 provides no quantitative evaluation: no success rate, no positional or velocity error statistics, no smoothness metric, no failure cases, and no exact list of the tested commands or dataset splits. The paper's own Section 5 concedes the method 'lacks mathematical guarantees' and can produce 'semantically accurate but irregular or non-smooth transitions.' Since the pipeline executes LLM-generated Python code directly (Section 3.2.2), the demonstration depends on the unmeasured assumption that the generated code was syntactically correct and constraint-satisfying for the shown runs, and that it would remain so across the command set with only two in-context exemplars. A capability claim can be supported by selected demonstrations, but 'successfully adapt' and 'demonstrate capabilities' imply systematic success, and the paper offers no evidence of rate or reliability. Independent verification is also impossible because no code, data, or artifact is released.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a language-based pipeline for trajectory adaptation in which a pre-trained LLM (GPT-4o) receives an initial trajectory, object positions, an environment description, and a natural-language instruction, and produces a human-reviewable high-level plan together with Python code that transforms the waypoints and velocities. The generated code is executed directly, and user feedback can be fed back into the LLM. The method requires no task-specific training. Experiments are conducted in PyBullet and Gazebo on a Kuka arm, a Crazyflie drone, and a ground robot, using a subset of the LaTTe dataset plus custom numeric and compound instructions. The central claim is that these simulations demonstrate successful free-form, instruction-driven trajectory reshaping without fine-tuning.","tokens_in":7944,"tokens_out":3991,"duration_ms":39756,"significance":"If the central claim is supported, the work provides a training-free alternative to feature-based sequence-to-sequence models such as LaTTe and ExtRaCT, and it extends the 'code as policy' paradigm to trajectory adaptation with numeric and compound commands. The paper's strengths include a simple and interpretable pipeline, explicit high-level plans that enable human debugging, a full prompt and example plans in the appendix, and demonstrations across three robot morphologies and two simulators. The significance is currently limited by the absence of quantitative evaluation: no success rates, error metrics, ablations, or comparisons with prior methods are reported, and no code or dataset is released. The approach is plausible, but the evidence as presented supports a feasibility demonstration rather than the stronger claim of systematic success.","major_comments":[{"comment":"The central claim that \"LLMs can successfully adapt trajectories to complex human instructions\" is supported only by the qualitative before/after plots in Figures 3-5. No success rate, positional or velocity error, constraint-satisfaction measure, smoothness metric, failure case, or exact list of tested commands is reported. Without such statistics, the figures cannot distinguish a working pipeline from favorable selections. Please add quantitative evaluation over the full command set and, ideally, a comparison with LaTTe [1] and ExtRaCT [2], since Section 4.4 promises an \"advantage over the State-of-the-art methods.\"","section":"Section 4.4"},{"comment":"The dataset description states that commands like \"Upper part\" and \"Top\" are excluded because they are \"not informative enough for a pre-trained model.\" This exclusion is load-bearing for the free-form claim: the method is then evaluated only on a curated subset, and the excluded classes may be exactly those where the approach struggles. The paper should justify the exclusion with examples or failure analyses, and either report results on the excluded commands or scope the central claim to the curated subset.","section":"Section 4.1"},{"comment":"The pipeline executes LLM-generated Python code directly after the user approves only the high-level plan; there is no verification that the code is syntactically correct, respects the two provided functions, or satisfies the stated constraints. The paper itself concedes in Section 5 that the method \"lacks mathematical guarantees\" and can produce \"semantically accurate but irregular or non-smooth transitions.\" The demonstration should report how often code generation required correction or failed across the command set, and what safeguards (e.g., automatic syntax checking, constraint monitors) are in place before execution.","section":"Sections 3.2.2 and 5"}],"minor_comments":[{"comment":"The model is written as \"GPT-40\" in the text but as \"GPT-4o\" in the conclusion; please unify the notation.","section":"Section 4.3"},{"comment":"The coordinate-system rule says \"The Positive Z axis is Up, Negative X axis is Down\"; the second clause should presumably read \"Negative Z axis is Down.\"","section":"Appendix A.2"},{"comment":"The RULES list numbers two items as \"2.\", and the phrase \"detect objects(obj name)\" is inconsistent with \"detect objects(object name)\" elsewhere; please fix the numbering and function signature.","section":"Appendix A.2"},{"comment":"Each figure should state which robot, simulator, command, and environment configuration it corresponds to, and the color legend (blue/red) should be repeated or made accessible for grayscale printing.","section":"Figures 3-5"},{"comment":"Several references are malformed, e.g., [8] has the truncated title \"Robots that use language\" and [9]-[10] contain \"and P. R.\" placeholders; please complete the bibliography.","section":"References"},{"comment":"The paper does not state whether code, data, or prompts beyond the appendix will be released; a reproducibility statement would be valuable given that the evaluation is purely qualitative.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a plausible system demonstration, but its evaluation is currently below the level needed to support the strong claims in the abstract and conclusion. I recommend major revision with quantitative evaluation on the full command set, inclusion of failure cases, and a scoped claim if the numbers cannot be provided. I would not require a new algorithmic contribution if the empirical evidence were substantially strengthened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe short version: this is a plausible pipeline that applies code-as-policies to trajectory adaptation, with a fully disclosed prompt and a clear argument for why it might beat trained baselines. The evaluation, however, is barely more than a demo. The paper shows three figures, claims success, and relies on the reader's goodwill. I'd send it to review because the direction is worth a serious look, but the authors need to do the quantitative homework.\n\nWhat's actually new: extending the LLM-code bridge to dense trajectory adaptation, combining a high-level plan with executable code, and handling numeric and compound commands. The full prompt and sample commands are in the appendix, so a reader can reproduce the approach without guessing. That is real value. The feedback loop that re-prompts with the original instruction is a sensible way to correct misplans.\n\nThe soft spots are the ones you'd expect. There is no success rate, no distance or velocity error, no smoothness metric, no comparison to LaTTe or ExtRaCT, and no failure cases. The paper excludes commands that are 'not informative enough' — that is fine if stated, but it narrows the claim. And the conclusion says the experiments 'demonstrate capabilities'; that is only true if the selected examples are representative, which we have no way to check. The own limitations section concedes no guarantees and possible non-smooth transitions, which is honest but is exactly why the evaluation should have measured smoothness.\n\nI'm not saying the method is wrong. The examples look correct, and the prompt design is careful. But 'LLMs can adapt trajectories' needs to mean 'they do it reliably across a test set', not 'here are four runs where it worked.' Without that, the comparison in the abstract to LaTTe and ExtRaCT is not supported.\n\nWho is this for? Someone working on LLM-based robot control who wants to see a concrete prompt-and-code recipe for trajectory reshaping. Reading group material if you want a case study in what counts as evidence. I would not cite it yet.\n\nRecommendation: Send to peer review. The idea deserves referee time, but the paper needs major revision — at minimum success rates, error metrics, and a proper baseline comparison before it can be accepted.","headline":"A transparent prompt-and-code pipeline for LLM-based trajectory adaptation that is plausible and worth a serious look, but the evaluation is anecdotal and needs quantitative evidence before the capability claim holds.","tokens_in":8454,"tokens_out":2331,"would_cite":false,"duration_ms":24719,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A pretrained LLM, prompted with two examples, reshapes robot trajectories from natural-language commands without task-specific training.","keywords":["trajectory adaptation","large language models","natural language commands","code as policy","waypoint manipulation","human-robot interaction","iterative feedback","numerical constraints"],"falsifier":"Take an unseen compound command with a hard numeric constraint, such as \"keep at least 10 distance from the box and go left by 20,\" run the generated code on a benchmark trajectory, and measure the minimum distance between any adapted waypoint and the box; a distance below 10, or a visibly non-smooth jump between adjacent waypoints, would contradict the claimed adaptation capability.","tokens_in":7502,"feed_emoji":"🤖","tokens_out":5227,"duration_ms":48627,"temperature":0.7,"pith_summary":"The paper sets out to show that a pretrained large language model can act as a trajectory adapter: given an existing path as waypoints plus user instructions, it produces a high-level plan and Python code that reshapes the path accordingly. The authors claim this works across robot types—manipulator arm, ground robot, quadrotor—and across command types, including numeric and compound instructions, with no fine-tuning. The appeal is that non-expert users could restate a desired change in plain language, and the same code-as-policy approach replaces hand-built task frames and retraining. The evidence is limited to simulation experiments on a subset of a published trajectory-language dataset plus newly created numeric and compound commands, with the paper itself conceding there are no mathematical guarantees for smoothness or constraint satisfaction.","feed_headline":"LLMs reshape robot paths from plain English, no training needed","feed_subtitle":"One prompt, two examples: GPT-4o rewrites waypoints for arms, drones, and ground robots in simulation.","key_machinery":"The load-bearing mechanism is a carefully framed prompt: it declares the coordinate system, defines two helper functions—detect_objects(name) returning an object's position and get_trajectory() returning the waypoint list—and gives the LLM two in-context examples of high-level plans for simple commands. From this prompt, the LLM jointly emits a high-level plan and Python code; the plan goes to the user for approval, and if approved the code runs to rewrite the waypoints, with feedback looping back into a new prompt if the user rejects the plan. The prompt also encodes task-agnostic rules: shift waypoints gradually for smoothness, decide whether start or goal positions should change, and allow adding or removing waypoints. This setup does the work of grounding the instruction in geometry, so no task-specific training data or task frame mathematics is needed.","core_discovery":"The central discovery is that a generic trajectory, written as a list of waypoints with velocities, can be adapted by having an LLM generate executable Python that operates directly on those waypoints, rather than by training a sequence-to-sequence model or replanning from scratch. The authors demonstrate that with only two in-context examples of high-level adaptation strategies, GPT-4o can parse a user instruction such as \"Go left by 20 keeping a distance of at least 10 from the box\" into a reviewable plan and code that moves, adds, or removes waypoints and adjusts velocities, while preserving the original trajectory's overall shape. They further show that when the human finds the plan wrong, feeding corrective feedback back into the LLM yields a corrected trajectory, providing an interpretable closed loop. The paper's stated conclusion is that the pipeline achieves free-form task-instructed trajectory reshaping without any fine-tuning, validated in PyBullet and Gazebo simulations across an arm, a drone, and a ground robot.","pith_inferences":["A natural next stress-test is objective constraint verification: run the generated code on a held-out set of numeric commands and check whether every adapted waypoint satisfies the stated minimum-distance or speed bounds, since the paper currently relies on visual inspection.","Because the adaptation is code, the same pipeline could in principle be wrapped in a verifier that rejects generated code violating user constraints, turning the current statistical behavior into a filterable process.","The dependence on two in-context examples raises a testable question: how much of the success comes from the specific exemplars versus the general code-generation ability of the model, which could be probed by removing or changing the examples.","For deployment, one would need to check whether generated code remains correct when object labels or coordinate conventions change, since \"left\" and \"front\" are fixed in the prompt rather than inferred from the robot frame."],"forward_implications":["If correct, a user can reshape a planned path by saying something like \"go slower near the box,\" and the resulting code changes both positions and velocities while keeping the path's overall shape.","The same framework transfers across robots—manipulator arm, drone, ground vehicle—because the adaptation operates on waypoints rather than on platform-specific dynamics.","Handling numeric and compound commands, such as \"go left by 20, keep at least 10 distance from the box,\" suggests the approach covers a wider command space than the simple directional commands used by trained baselines.","The human-in-the-loop plan review turns misinterpretation into a fixable step: the user reads the high-level plan, corrects it, and the LLM regenerates the code instead of silently executing."],"supporting_citations":[{"why":"Supplies the dataset subset used for evaluation and the trained sequence-to-sequence baseline the method is compared against.","marker":"[1]"},{"why":"Represents the state-of-the-art feature-based explainable trajectory-correction approach that requires training and is contrasted with the training-free method.","marker":"[2]"},{"why":"Demonstrates zero-shot LLM trajectory generation, which this work adapts into the trajectory-reshaping setting.","marker":"[14]"},{"why":"Provides the code-as-policy idea the authors extend from task waypoint generation to existing-trajectory adaptation.","marker":"[16]"},{"why":"Motivates the closed-loop feedback mechanism used when the user corrects the high-level plan.","marker":"[17]"},{"why":"Shows online language corrections for manipulation, giving a comparison point for iterative correction without retraining.","marker":"[19]"}],"fun_headline_variants":["LLMs rewrite robot paths from plain English, no retraining","No training: LLM reshapes robot trajectories from human instructions","GPT-4o turns natural language into working robot waypoint edits","Robot path adaptation via LLM code generation, zero fine-tuning","English instructions become robot code: LLM adapts trajectories on the fly"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the LLM-generated Python code will be correct and general enough to handle arbitrary unseen instructions, including numeric and compound ones, with only two in-context examples and optional human feedback; the paper itself warns it lacks mathematical guarantees and can produce semantically correct but irregular trajectories.","fun_headline_variants_meta":{"raw":{"variants":["LLMs rewrite robot paths from plain English, no retraining","No training: LLM reshapes robot trajectories from human instructions","GPT-4o turns natural language into working robot waypoint edits","Robot path adaptation via LLM code generation, zero fine-tuning","English instructions become robot code: LLM adapts trajectories on the fly"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000602,"raw_usage":{"total_tokens":2787,"prompt_tokens":900,"completion_tokens":1887,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":516,"completion_tokens_details":{"reasoning_tokens":1798}},"tokens_in":516,"tokens_out":1887,"duration_ms":12571,"temperature":1.0,"reasoning_tokens":1798,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:22:56.391308+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take an unseen compound command with a hard numeric constraint, such as \"keep at least 10 distance from the box and go left by 20,\" run the generated code on a benchmark trajectory, and measure the minimum distance between any adapted waypoint and the box; a distance below 10, or a visibly non-smooth jump between adjacent waypoints, would contradict the claimed adaptation capability.","supporting_citations":[{"cited_title":"Latte: Language trajectory transformer","cited_arxiv_id":null,"evidence_quote":"Supplies the dataset subset used for evaluation and the trained sequence-to-sequence baseline the method is compared against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Demonstrates zero-shot LLM trajectory generation, which this work adapts into the trajectory-reshaping setting."},{"cited_title":"Code as policies: Language model programs for embodied control","cited_arxiv_id":null,"evidence_quote":"Provides the code-as-policy idea the authors extend from task waypoint generation to existing-trajectory adaptation."},{"cited_title":"No, to the right: Online language corrections for robotic manipula- tion via shared autonomy","cited_arxiv_id":null,"evidence_quote":"Shows online language corrections for manipulation, giving a comparison point for iterative correction without retraining."}],"review_version":1}