{"id":"77f4df22-8027-45b8-8ad3-85ea1fc38189","arxiv_id":"2506.00075","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"A direct ChatGPT-to-ROS2 voice command pipeline for robot navigation is reported to cut latency by 7.01%, but the paper's own data do not reproduce that number.","lead":"This paper connects ChatGPT directly to a simulated robot's navigation system so a person can give spoken commands and the robot obeys. The authors claim their direct link cuts response time by about seven percent, but the paper's own tables show inconsistent numbers that undermine the claim.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table I contradicts the paper's own headline numbers: the per-trial latencies average to about 1.10 s for GPT-3.5 Turbo and 1.29 s for ROSGPT, not the stated 1.18/1.28, making the 7.01% reduction unreproducible.","rationale":"The reader's verdict is REJECT, with high correctness risk. In good faith, this paper proposes a concrete engineering idea: removing the Flask middleware between ChatGPT and ROS 2 to reduce latency. That idea could be sound, and a direct API call might indeed be faster in a controlled comparison. However, the paper's experimental section is the only evidence for the central claim, and that evidence is internally inconsistent. The most load-bearing concern is not merely the undocumented baseline conditions (which the reader's weakest_assumption identified) but the fact that the paper's own Table I and accompanying prose cannot reproduce the reported 7.01% reduction or the success-rate differences. This is a stronger, more direct failure than a need for controlled conditions: even if every condition were perfectly matched, the numbers as presented do not support the conclusion. I therefore partially agree with the reader: they correctly identify high risk, but the decisive flaw is the arithmetic and counting contradictions in Table I and its interpretation. The proposed concrete test, a simple recomputation and recount, would settle the matter. Such a check is inexpensive and should be the first step in any revision. If the errors are typographical, a corrected table might salvage the empirical claim, but as written the paper does not support its headline result.","tokens_in":10987,"tokens_out":4395,"duration_ms":36245,"concrete_test":"Independently sum the 20 latency values in each column of Table I, compute the mean for GPT-3.5 Turbo and ROSGPT, recompute the percent reduction, and recount the FAIL/OK entries in the ROSGPT success column. If the recomputed means differ from the reported 1.18 s and 1.28 s averages, and the reduction is approximately 14.8% rather than 7.01%, or if the success count contradicts the reported 13 misinterpreted orders, then the headline claim is invalid as stated and would require corrected data and controlled baseline measurements.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim, a 7.01% average latency reduction, is not supported by the paper's own data. Three arithmetic checks fail. First, summing the 20 GPT-3.5 Turbo latencies in Table I gives 21.93 s, a mean of 1.0965 s, not the reported 1.18 s. Second, summing the 20 ROSGPT latencies gives 25.73 s, a mean of 1.2865 s, while the text says '1.2865 s with our solution ... to 1.1804 s with ROSGPT', reversing the systems and introducing a 1.1804 figure that matches neither the table nor the recomputation. Third, the claimed 7.01% reduction is not derivable: using the recomputed means gives (1.2865-1.0965)/1.2865 ≈ 14.8%, and using the paper's labeled means (1.18, 1.28) gives ≈ 7.8%. The success narrative is equally inconsistent: the text says ROSGPT 'incorrectly interpreted 13 of the orders', but Table I shows 6 FAILs (rows 7, 8, 10, 17, 18, 20) and 14 OKs; a 13-failure count would imply only 7 successes. Because the quantitative basis for the contribution cannot be reproduced from the evidence provided, the empirical claim is unreliable regardless of whether the architecture itself could reduce latency under controlled conditions.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an architecture for integrating ChatGPT with ROS 2 for natural-language robot navigation, claiming to reduce communication latency by removing middleware such as Flask. The system transcribes voice commands with a speech-to-text module, sends the text to the OpenAI API, parses the structured response, and generates ROS 2 velocity commands. The authors compare their direct-API approach against the ROSGPT baseline using 20 voice commands in a Gazebo simulation, reporting a 7.01% average latency reduction, perfect success (20/20) for their GPT-3.5 Turbo and GPT-4.0 implementations, and 14/20 success for ROSGPT. The manuscript also describes the prompt structure, pseudocode, and experimental setup. The main claimed contribution is the empirical latency improvement from removing the middleware transport platform.","tokens_in":11300,"tokens_out":2670,"duration_ms":25804,"significance":"If the latency reduction and success-rate claims were reproducible, the paper would offer a plausible incremental improvement for LLM-based human-robot interaction: replacing a Flask-based middleware with a direct OpenAI API call is a reasonable design choice that could reduce overhead. The paper also usefully compares two GPT model versions and reports a concrete command-interpretation success metric, and it provides pseudocode that partially documents the system. However, the significance is heavily undermined by internal numerical inconsistencies: the headline 7.01% figure does not follow from Table I, the stated averages are contradicted by the table's own rows, and the success narrative (13 misinterpreted orders versus 14 successes) is internally inconsistent. Because the central empirical claim is not reproducible from the provided data, the paper's contribution cannot be accepted as currently written.","major_comments":[{"comment":"The reported average latencies are not consistent with the per-trial data in Table I. Summing the 20 GPT-3.5 Turbo latencies gives 21.93 s, for a mean of 1.0965 s, not the reported 1.18 s. Summing the 20 ROSGPT latencies gives 25.73 s, for a mean of 1.2865 s, not the reported 1.28 s (or the 1.1804 s attributed to ROSGPT in the text). The claimed 7.01% reduction is therefore not derivable: using the table's labeled means (1.18 vs. 1.28) yields approximately 7.8%, using the recomputed means yields approximately 14.8%, and using the text's 1.2865 vs. 1.1804 yields approximately 8.25%. The paper must correct these numbers and recompute all derived percentages.","section":"Section V, Table I"},{"comment":"The text states that the ROSGPT model 'incorrectly interpreted 13 of the orders,' but Table I shows exactly 6 FAIL entries for ROSGPT (rows 7, 8, 10, 17, 18, 20) and 14 OK entries. If 13 commands were misinterpreted, the success count would be 7, not 14. This direct contradiction undermines the success-rate comparison and must be resolved with a consistent account of which 13 commands failed and which 7 succeeded.","section":"Section V"},{"comment":"The baseline latency measurement for ROSGPT is not documented with sufficient detail to support a causal attribution of the latency difference to the absence of Flask. The text says only that rosgpt.py was modified to insert timing calls. It does not report the network conditions, OpenAI API version, prompt structure, number of repeated trials, hardware, or whether the baseline and proposed systems were run under identical conditions. Without this information, any latency difference could be due to random variation or environmental factors rather than the middleware architecture.","section":"Section V"},{"comment":"The speech recognition pipeline is described inconsistently. Section III states that voice commands are transcribed by the Whisper speech-to-text module, while Section III-A's pseudocode and the 'Voice Recognition' subsection describe using Google's speech recognition service. The paper cannot claim a particular speech-to-text component if the implemented system uses a different one; this ambiguity affects reproducibility and must be corrected with a precise statement of which STT engine was actually used.","section":"Section III and Section III-A"}],"minor_comments":[{"comment":"The caption reads 'Performance Comparistion between approaches'; this should be 'Performance Comparison between Approaches.'","section":"Table I caption"},{"comment":"The word 'randonmly' is a typo for 'randomly.'","section":"Section V"},{"comment":"The phrase 'rol system prompt' and 'rol user prompt' appears twice; 'rol' should be 'role.'","section":"Section III-C"},{"comment":"The second author's affiliation line lists 'Rodrio S. Guerra,' which appears to be a typo for 'Rodrigo S. Guerra.'","section":"Author list"},{"comment":"The sentence 'However, the es=ES model perfectly supports the English language' is confusing: es=ES is a Spanish locale code, and the claim that it 'perfectly supports the English language' needs clarification or correction.","section":"Section III-B"},{"comment":"The pseudocode block contains line-break artifacts and comments interleaved with code, making it difficult to read; a cleaned, properly indented listing would improve the presentation.","section":"Section III-C"},{"comment":"The abstract and conclusions repeat the '7.01%' figure without qualification; these statements should be updated to match the corrected experimental results once the arithmetic errors are resolved.","section":"Abstract and Section V"}],"recommendation":"reject","confidential_remarks":"The manuscript's central quantitative claims are contradicted by its own Table I, and the baseline comparison lacks the experimental controls needed to support a latency-reduction claim. These are not local presentation issues but load-bearing problems with the empirical contribution. Even though the architecture idea (direct API vs. Flask middleware) is plausible, the submitted evidence does not support the stated conclusions. If the authors rerun the experiments with proper controls, correct the table, and reconcile the success-rate narrative, a substantially revised version could be reconsidered. I also note that the paper's writing quality, including inconsistent STT descriptions and multiple typos, suggests the manuscript would benefit from a careful rewrite before any resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the headline number doesn't survive its own table. The paper compares a direct OpenAI API call against ROSGPT's Flask-based integration for LLM-based robot navigation commands, and claims a 7.01% average latency reduction. Summing the 20 GPT-3.5 Turbo latencies in Table I gives a mean of about 1.10 s, not the 1.18 s printed; the ROSGPT column sums to about 1.29 s, not 1.28 s. Worse, the text says '1.2865 s with our solution ... to 1.1804 s with ROSGPT,' reversing the two systems. And the claimed 7.01% doesn't follow from any consistent set of numbers: using the printed means gives roughly 7.8%, using the recomputed means gives about 14.8%. The success count is also off: the text says ROSGPT misinterpreted 13 orders, but Table I shows 6 FAILs and 14 OKs. So the central quantitative contribution is not reproducible from the evidence provided.\n\nWhat's worth crediting: the paper is a straightforward engineering comparison against an existing public package, with per-trial data reported (even if mislabeled). The idea that skipping the Flask middleware reduces latency is plausible and worth testing. The prompt design and parsing logic are described clearly enough to replicate. The related-work survey is broad, though it includes some weakly relevant items.\n\nThe soft spots beyond the arithmetic: no information on the baseline measurement conditions—network state, API version, prompt, number of repeated trials. Without that, even a corrected number wouldn't be interpretable. No code or data are shipped. The sample size (20 commands) is small, and the paper doesn't report variance. The comparison is also only against one baseline; a more meaningful study would include a few variants or at least repeated runs.\n\nBottom line: this is a small engineering report that could be useful if the data were fixed and the methodology documented. As written, the main claim is unsupported. I'd send it back for major revision, not desk-reject it, because the underlying question is legitimate and the authors clearly did the work—they just botched the reporting. A serious referee could help them turn it into something citable.\n\nWho benefits: people building simple voice-command ROS2 prototypes might find the architecture description useful despite the numbers. But I wouldn't cite the latency claim.","headline":"The paper's 7.01% latency reduction is not reproducible from its own table; the underlying idea is plausible but the reported evidence is unreliable.","tokens_in":11821,"tokens_out":4812,"would_cite":false,"duration_ms":41865,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that removing the Flask/JSON middleware between ChatGPT and ROS 2 cuts average voice-command latency by 7.01% and lifts command-interpretation success from 14/20 to 20/20.","keywords":["large language models","ROS 2","human-robot interaction","latency reduction","natural language commands","voice control","prompt engineering","mobile robot navigation"],"falsifier":"A controlled replay of the same twenty voice commands through both pipelines on the same machine, network, API version, and prompt style, with repeated trials, would settle the claim. A simpler check is arithmetic: recompute the column means from Table I; if the published averages do not reproduce from the listed per-command latencies, the measurement basis of the claim is in question.","tokens_in":10800,"feed_emoji":"🤖","tokens_out":10710,"duration_ms":96807,"temperature":0.7,"pith_summary":"The paper tries to establish that the middleware layer in LLM-to-robot voice control is removable without losing functionality, and that removing it buys a measurable speed improvement. The authors connect ChatGPT directly to a ROS 2 navigation node for a simulated mobile robot: speech is transcribed, a prompt with fixed output rules asks the model for a command, and the returned text is split into words that become linear or angular motion commands. On a 20-command voice test they report a 7.01% average latency reduction and 20/20 successful command interpretations for GPT-3.5 Turbo and GPT-4.0, against 14/20 for the Flask-based ROSGPT baseline. The reason to care is that LLM response time is often seen as the irreducible cost in such systems; this work argues that transport and parsing overhead around the model is a separate, reducible cost.","feed_headline":"Direct ChatGPT-to-ROS 2 link trims voice-command latency 7.01%","feed_subtitle":"It also lifts command-interpretation success from 14 of 20 to 20 of 20 versus the Flask-based baseline.","key_machinery":"The carrying mechanism is a fixed-format output contract enforced by a two-role prompt. The system message tells ChatGPT to behave as a command interpreter, to convert magnitudes to meters and m/s, and to return a rigid word sequence; the user message presents the spoken instruction. The parser is not a semantic parser but a word-position reader: if the first token is 'move' it reads direction, distance, and speed from fixed slots; if it is 'rotate' it reads direction, angle, and angular velocity from different fixed slots. This removes the need for JSON and for a Flask transport layer, which is the precise difference from the ROSGPT baseline.","core_discovery":"The central claim is that a direct-API request path, with no Flask server and no JSON serialization between ChatGPT and ROS 2, lowers the latency of natural-language robot commands and makes interpretation more reliable. The system prompt fixes the command vocabulary and formatting so the model's answer can be parsed by word position, and the robot controller converts those parameters into Twist messages on the cmd_vel topic. The authors report that the direct pipeline with GPT-3.5 Turbo reduced average communication latency by 7.01% relative to the ROSGPT baseline, while both tested GPT versions interpreted 20 of 20 commands correctly compared with 14 of 20 for the baseline. They also note that the baseline cannot handle non-standard units or 'left'/'right' turn wording, whereas the prompt rules handle unit conversion and synonyms.","pith_inferences":["The latency advantage is measured only against the original ROSGPT implementation; against a more optimized or asynchronous transport, or a locally hosted model, middleware overhead could shrink or reverse, so 7.01% should not be read as a universal constant.","Word-position parsing is brittle: it depends on the model obeying the exact token order, so robustness and safety in open-vocabulary or noisy deployments would require stronger output validation than a split() call.","The 20/20 success rate is over a small, constrained command set; testing with varied accents, room noise, and ambiguous phrasing would map where the approach stops working."],"forward_implications":["Middleware elimination is a concrete lever for latency reduction in LLM-based robot control.","Prompt-formatted responses can replace JSON parsing for small, fixed command vocabularies.","The direct-API pattern should transfer to other speech-to-text and LLM services that honor a fixed output contract.","The reported 20/20 versus 14/20 success suggests interpretation reliability, not just speed, improves when the output format is tightly constrained.","These changes move natural-language voice control closer to real-time industrial use."],"supporting_citations":[{"why":"Supplies the ROSGPT baseline implementation whose latency and success rates are compared against the proposed direct-API pipeline.","marker":"[20]"},{"why":"Describes the Flask-based ROSGPT architecture that the proposed system removes middleware from, defining the baseline's transport design.","marker":"[19]"},{"why":"Shows how ChatGPT prompts can translate natural-language instructions into executable robot plans, motivating the prompt-based command format used here.","marker":"[4]"},{"why":"Introduces language-model programs that generate robot control code from text, the lineage the paper places its parser-and-publisher design within.","marker":"[5]"}],"fun_headline_variants":["No Flask, no JSON: direct ChatGPT-ROS2 cuts latency 7.01%","Direct ChatGPT-ROS2 trims latency 7.01% and hits 20/20","ChatGPT-ROS2 without middleware: 7.01% latency drop","Direct ChatGPT-ROS2 link lifts command success to 20/20","ChatGPT-ROS2 direct link beats Flask baseline: 7.01% faster, 20/20 vs 14/20"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result rests on the assumption that the ROSGPT baseline was tested under the same API, model, prompt, network, and hardware conditions as the proposed pipeline, so the 7.01% difference really comes from removing the middleware rather than from any other difference in test setup.","fun_headline_variants_meta":{"raw":{"variants":["No Flask, no JSON: direct ChatGPT-ROS2 cuts latency 7.01%","Direct ChatGPT-ROS2 trims latency 7.01% and hits 20/20","ChatGPT-ROS2 without middleware: 7.01% latency drop","Direct ChatGPT-ROS2 link lifts command success to 20/20","ChatGPT-ROS2 direct link beats Flask baseline: 7.01% faster, 20/20 vs 14/20"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001053,"raw_usage":{"total_tokens":4377,"prompt_tokens":857,"completion_tokens":3520,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":473,"completion_tokens_details":{"reasoning_tokens":3401}},"tokens_in":473,"tokens_out":3520,"duration_ms":25115,"temperature":1.0,"reasoning_tokens":3401,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:37:38.337567+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled replay of the same twenty voice commands through both pipelines on the same machine, network, API version, and prompt style, with repeated trials, would settle the claim. A simpler check is arithmetic: recompute the column means from Table I; if the published averages do not reproduce from the listed per-command latencies, the measurement basis of the claim is in question.","supporting_citations":[{"cited_title":"Rosgpt: Next-generation human-robot interaction with chatgpt and ros,","cited_arxiv_id":null,"evidence_quote":"Describes the Flask-based ROSGPT architecture that the proposed system removes middleware from, defining the baseline's transport design."}],"review_version":1}