{"id":"d12d5ca9-c9bd-41ae-a17d-a5c342edc615","arxiv_id":"2505.15793","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"The HCRMP planner feeds LLM semantic hints into state representation and critic weighting instead of letting the LLM decide actions, reporting better CARLA driving metrics.","lead":"This paper introduces a self-driving motion planner that uses a large language model for semantic hints while keeping the reinforcement learning agent in control. In CARLA simulations the planner reports higher success rates and fewer collisions than existing LLM-guided driving methods.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed hallucination-robustness is unsupported because Eq. 2-3 route LLM-generated lambda_i directly into the PPO advantage estimate, so a biased LLM biases the policy gradient rather than being filtered out.","rationale":"The reader's weakest assumption is exactly the one I identify as most load-bearing: the lambda_i weights in Eq. 2 are assumed to be informative enough that the RL policy benefits on average while absorbing errors. My concern goes one step further: the architecture does not merely 'absorb' errors—it directly amplifies them into the policy gradient, because lambda_i multiplies the advantage that trains the actor. There is no learned gating, no confidence estimate, and no empirical hallucination-injection test, so the central claim of hallucination robustness is unsupported. This is not a disagreement with external consensus; it is an internal mechanism problem: Eq. 3 makes the policy explicitly a function of lambda, so a wrong lambda is not an independent suggestion that the RL can choose to ignore. The reader also flagged the unnamed LLM and missing seeded runs, but those are secondary; the hallucination-injection test would settle the primary concern. The paper does provide a concrete architecture and plausible motivation, and the CARLA results, if reproducible, would be meaningful, so the appropriate action is not rejection but conditional acceptance pending the robustness test and code release. I agree with the reader's CONDITIONAL verdict and recommend no change.","tokens_in":13055,"tokens_out":2146,"duration_ms":22137,"concrete_test":"Run a hallucination-injection study in the CARLA trilemma and occluded-pedestrian scenarios: while keeping all other components fixed, replace the LLM's lambda_i with adversarially biased values (e.g., set safety weight to 0.1 and efficiency to 0.9) or add controlled Gaussian noise to the ASR semantic vector s_llm. Measure SR and CR against the nominal HCRMP and against a no-hint baseline with fixed uniform lambda. If SR/CR degrade in proportion to the injected bias, the RL agent is not counteracting the hallucinations and the central robustness claim fails; if performance is unchanged, the claim gains support.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that HCRMP's 'relative independence' between LLM and RL lets the RL agent counteract erroneous semantic indications. But the only mechanism by which LLM hints affect policy learning is through Eq. 2-3: the LLM-produced weights lambda_i scale each critic's advantage, and the PPO clipping objective in Eq. 3 maximizes/minimizes based on this weighted sum. A systematically misleading lambda (for instance, a safety weight near zero in a hazardous situation) changes the sign and magnitude of the advantage for safe actions, so the policy gradient actively pushes the agent away from safe behavior. The actor network does not have a mechanism to detect that lambda is wrong; lambda is treated as ground truth in the loss. The ASR state augmentation s_llm can be corrupted in the same way, and no uncertainty estimate, learned reliability gate, or hallucination-injection experiment is provided. The paper's own motivating statistic (non-hallucination rate only 57.95%) implies that a large fraction of lambda_i will be biased; RAG and Top-3 retrieval reduce but do not eliminate this structural coupling. Therefore the statement that 'the RL agent counteracts potential erroneous semantic indications through policy learning' is not supported by the architecture as written, and the claimed 11.4% collision-rate reduction could be an artifact of a mostly benign LLM in the specific test scenarios rather than a property of the paradigm.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HCRMP, an LLM-hinted reinforcement-learning motion-planning framework for autonomous driving in the CARLA simulator. The architecture augments the RL state with LLM-generated scenario- and object-level semantic hints (ASR module), modulates a multi-critic PPO advantage estimate with LLM-generated per-attribute weights that are anchored by retrieval-augmented generation from a knowledge base (CSA module), and decouples the low-frequency LLM from high-frequency RL control via a semantic cache (SCM). The central claim is that maintaining 'relative independence' between the LLM and the RL agent makes the system robust to LLM hallucinations, with a reported task success rate of up to 80.3% and an 11.4% collision-rate reduction in safety-critical conditions. Experiments compare HCRMP to Vanilla PPO, E2ECLA, AutoReward, and VLM-RL, and include an ablation in a single trilemma scenario.","tokens_in":13492,"tokens_out":5564,"duration_ms":45801,"significance":"If the hallucination-robustness claim were supported, the weakly coupled design of using LLM hints for state augmentation and critic-weight modulation rather than for direct action or reward generation would be a worthwhile contribution to the LLM-RL autonomous-driving literature. The paper also offers a clear taxonomy of LLM-dominated RL methods and a plausible asynchronous execution scheme. However, the current evidence does not establish the headline claim: there are no hallucination-injection experiments, no explicit mechanism for the RL agent to detect or discard unreliable LLM weights, and no statistical confidence measures for the point estimates in the result tables. With additional experiments and architectural clarification, the approach could be of interest to the autonomous-driving and LLM-RL communities.","major_comments":[{"comment":"The claimed ability of the RL agent to 'counteract potential erroneous semantic indications through policy learning' is not realized in the architecture as written. The LLM-generated weights λ_i scale each critic advantage directly in the GAE estimate, and the PPO clipping objective in Eq. (3) maximizes the probability ratio weighted by this advantage. A systematically wrong λ_i, for instance a near-zero safety weight in a hazardous situation, changes the sign and magnitude of the advantage for safe actions, so the policy gradient actively pushes the agent away from safe behavior. The actor network receives no uncertainty estimate or reliability gate for λ_i; the RAG Top-3 retrieval in the CSA module reduces the frequency of erroneous weights but does not eliminate this structural coupling. Thus, the headline robustness claim requires either an architectural mechanism that detects or ignores unreliable weights, or a direct hallucination-injection experiment demonstrating that corrupted λ_i do not degrade policy performance. Neither is provided; the paper's own motivating statistic that a SOTA LLM has only 57.95% non-hallucination rate actually implies that a large fraction of λ_i can be biased.","section":"Section 3.3, Eqs. (2)-(3)"},{"comment":"All reported metrics are single point estimates with no error bars, number of seeds, or episode counts. The key comparative claim of an 11.4% collision-rate reduction appears to be the difference between HCRMP and VLM-RL averaged over the six safety-critical rows (32.3% versus 43.7%), but the text never states this comparison explicitly, and without variance or significance testing the difference could be within random variation. Additionally, the HCRMP Trilemma Low row reports SR=75 and CR=28, which sum to 103 and cannot both be true under the paper's definitions; this internal inconsistency needs correction or explanation.","section":"Section 4.2, Tables 1-2"},{"comment":"The LLM used inside HCRMP is never named. The paper evaluates Gemini-2.5-Pro in Figure 1 to motivate the hallucination problem, but it does not state whether the HCRMP experiments use the same model, a different model, which prompt templates are used, or how the 57.95% non-hallucination rate was measured (dataset, question set, scoring rubric). Without this information, the extent to which HCRMP actually encounters hallucination rates similar to those claimed cannot be assessed, and the experiments are not reproducible.","section":"Section 4.1-4.2 and Figure 1"},{"comment":"The ablation is run only on the medium-density trilemma scenario and reports single-seed point estimates. The reward-curve comparison in Figure 3 has no axis labels or confidence bands, so the claim that CSA 'stabilizes' training rests on qualitative visual inspection. Furthermore, the naming of ablation variants is confusing: 'HCRMP w/ ASR' in Table 3 appears to denote the full model, but the text contrasts it with 'HCRMP without ASR' without clarifying whether CSA is present in both. The table should be relabeled and the exact configuration of each row stated.","section":"Section 4.3, Table 3 and Figure 3"}],"minor_comments":[{"comment":"The reported 'task success rate of up to 80.3%' is not traceable in Tables 1 and 2; the highest tabulated success rate is 99% (overtaking, low density) and the maximum in safety-critical conditions is 75%. Please clarify how the 80.3% figure is computed.","section":"Abstract and Section 5"},{"comment":"The 'semantic compensation mechanism' for low-density traffic is mentioned but never described; its inputs, outputs, and effect on the state vector should be specified.","section":"Section 3.2"},{"comment":"The axes in Figure 3 are unlabeled; the caption should define what 'Reward' and 'Step' refer to and which training or evaluation episodes are plotted.","section":"Figure 3"},{"comment":"The sentence 'LLM poss strong semantic understanding' contains a typo, and the phrase 'intrinsic modulation' undercuts the 'relative independence' terminology used elsewhere in the same paragraph; the intended degree of coupling should be defined precisely.","section":"Section 1"},{"comment":"The sentence attributing 'efficiency and comfort' to ASR and 'extending the state space' to CSA appears to swap the roles of the two modules as defined in Section 3; please correct this attribution.","section":"Section 4.2"},{"comment":"The Semantic Cache Module is not directly evaluated; the paper should report how often the cache fallback occurs and how the nearest-neighbor matching performance affects success and collision rates.","section":"Section 3.4"}],"recommendation":"major_revision","confidential_remarks":"This paper is a candidate for major revision rather than rejection. The weakly coupled LLM-hint idea is plausible and timely, and the taxonomy of LLM-dominated RL methods is useful. However, the central hallucination-robustness claim is currently overstated: the LLM weights enter the PPO advantage directly with no reliability gate, and no hallucination-injection experiments are provided. The empirical section also needs proper statistics and a named LLM. I would urge the editor to require those additions before considering publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nTwo things you should know. First, the core reframing is genuinely useful: instead of letting the LLM generate actions or rewards, HCRMP has it provide semantic state augmentation and adaptive critic weights, with a cache to bridge the LLM's low frequency. That's a sensible middle ground and a clear departure from the LLM-dominated designs it criticizes. Second, the evidence does not back the paper's main robustness claim. The claim that the RL agent 'counteracts' hallucinations is not supported by the architecture: the LLM-produced lambda_i scales the advantage directly in Eq. 2-3, so a wrong lambda changes the sign and magnitude of the policy gradient. RAG anchoring reduces hallucinations, but nothing in the design detects or filters them. The paper's own motivating number—57.95% non-hallucination—means a large fraction of hints can be misleading, and the agent has no mechanism to know which ones.\n\nWhat the paper does well: the three modules are clearly described, the idea of retrieving authoritative text to stabilize critic weights is a nice practical addition, and the ablations, though inconsistent, suggest both ASR and CSA contribute. The writing is readable and the related-work placement is fair.\n\nThe soft spots are significant. There are no error bars, seeds, or episode counts anywhere; the specific LLM used in experiments is never named; and there is no hallucination-injection experiment, which is the one test that would directly support the central claim. Table 2 has a row where SR + CR exceed 100% (Trilemma Low, HCRMP: 75 + 28). Table 3's ablation appears to mislabel or omit the full model, since the 'with ASR' row at 54% is below the full model's 69% in the comparable condition. These are fixable but reinforce the impression that the experiments were not prepared to the level the claims require.\n\nWho's this for? Researchers working on LLM-RL integration for driving, especially those designing against LLM failure modes. The paradigm is worth discussing, but the current form is a workshop-level presentation with conference-level claims. It deserves a serious referee—the idea is important enough to warrant careful evaluation—but the authors need to release code and configs, identify the LLM, add statistical rigor, run hallucination injection, and fix the tables before this is acceptable.\n\nMy recommendation: send it to peer review, but expect heavy revision. If the authors do that work, the weakly coupled hint paradigm could be a real contribution.","headline":"A useful reframing of LLM-RL integration for driving, but the evidence is too thin and the hallucination-robustness claim doesn't follow from the design.","tokens_in":13904,"tokens_out":3196,"would_cite":false,"duration_ms":27661,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that autonomous driving should keep the language model advisory, not authoritative, and reports up to 80.3% task success with an 11.4% collision-rate reduction in safety-critical driving.","keywords":["LLM-hinted reinforcement learning","autonomous driving motion planning","LLM hallucination mitigation","multi-critic PPO","retrieval-augmented semantic grounding","semantic state augmentation","low-frequency LLM high-frequency RL"],"falsifier":"Replace the LLM's semantic hints in the reported simulator runs with deliberately wrong hints, such as reversed scenario classifications and swapped critic weights, and measure success and collision rates; if performance collapses, the claim that RL absorbs hallucinations is not supported, whereas if it stays near the reported levels, the buffering is real.","tokens_in":12843,"feed_emoji":"🚗","tokens_out":6315,"duration_ms":52946,"temperature":0.7,"pith_summary":"The paper argues that existing LLM-RL driving systems fail because the language model's outputs directly drive the policy or define its rewards, so hallucinations become the policy's mistakes. It proposes an LLM-hinted paradigm instead: the language model supplies semantic descriptions of the scene and adaptive weights across safety, efficiency, and comfort critics, while the reinforcement learning agent retains control of the learned policy. In the proposed HCRMP architecture, these hints enter only as state augmentation and as modulation of the multi-critic advantage estimate. Evaluations in an urban driving simulator report a task success rate up to 80.3% across traffic densities and an 11.4% reduction in collision rate in safety-critical scenarios. If correct, LLM common sense can improve RL driving without letting the model's errors dictate actions.","feed_headline":"LLM hints, not commands, cut driving collisions by 11.4%","feed_subtitle":"Keeping the language model advisory lets the driving policy absorb wrong hints instead of following them.","key_machinery":"The load-bearing object is the integrated advantage estimate $\\hat A^{\\mathrm{int}}(s,a) = \\sum_i \\lambda_i A_i^{\\pi_\\theta}(s,a)$ from Eq. 2, where each $\\lambda_i$ is an LLM-generated, knowledge-base-anchored weight between critics for safety, efficiency, and comfort. That estimate feeds the clipped PPO objective in Eq. 3, so the LLM influences the policy only through the multiplier of each critic's advantage. The Augmented Semantic Representation module extends the raw state $s^{\\mathrm{raw}}_t$ with semantic embeddings $s^{\\mathrm{llm}}_t$, and the Semantic Cache module substitutes a nearest-neighbor historical weight vector whenever the LLM's low-frequency output misses its time window.","core_discovery":"The central claim is that maintaining relative independence between the LLM and the RL agent is the key to using LLM knowledge in autonomous driving: the LLM should hint, not decide. HCRMP encodes that separation with three modules: the Augmented Semantic Representation module extends the RL state with multi-level scene abstractions; the Contextual Stability Anchor module retrieves traffic-regulatory passages from a knowledge base and uses them to stabilize the LLM's per-critic weights; and the Semantic Cache module matches historical conditions when the LLM's low-frequency outputs are delayed. The paper reports that this arrangement raises task success rate to 80.3% and cuts collision rate by 11.4% relative to baselines in safety-critical conditions.","pith_inferences":["A direct stress test the paper leaves undone: inject deliberately wrong or inverted semantic hints and measure whether success and collision rates hold; that would quantify how much of the buffering comes from the architecture rather than from the specific LLM's accuracy.","Because the reported non-hallucination rate of roughly 57.95% was measured on a single unnamed model, a natural extension is to vary LLM quality and map the performance degradation curve; the paper's own claim predicts a much flatter curve than LLM-dominated methods would show.","The normalized critic weights mean even a mistaken LLM can shift the trade-off among safety, efficiency, and comfort; adding a learned trust or calibration layer on top of the knowledge base might be the next step beyond this paper."],"forward_implications":["If the 80.3% success figure holds, weakly coupled LLM hints can improve RL driving across low, medium, and high traffic densities without retraining per condition.","The 11.4% collision-rate reduction implies that grounding LLM-generated critic weights in a static knowledge base stabilizes multi-objective policy updates in safety-critical situations.","The asynchronous Semantic Cache shows a concrete way to reconcile LLM latency with real-time control: cache and retrieve past semantic states when fresh hints are unavailable.","The multi-critic PPO formulation suggests a general recipe for LLM-RL integration in other control domains: use LLM outputs as state features and objective multipliers, not as direct action labels or rewards."],"supporting_citations":[{"why":"Supplies the PPO clipped objective into which the multi-critic advantage estimate is plugged.","marker":"[53]"},{"why":"Supplies retrieval-augmented generation, used to anchor LLM-generated critic weights in traffic-regulatory text.","marker":"[54]"},{"why":"Provides the similarity-search method used for both knowledge-base retrieval and historical cache matching.","marker":"[57]"},{"why":"Provides the open urban driving simulator where all reported experiments and baselines are evaluated.","marker":"[58]"},{"why":"Provides the lightweight visual backbone used to encode multimodal observations into the RL state.","marker":"[52]"},{"why":"Is the LLM-dominated reward-design baseline whose collision and success numbers HCRMP is compared against.","marker":"[29]"},{"why":"Is the LLM-dominated vision-language-RL baseline used in the safety-critical comparisons.","marker":"[25]"}],"fun_headline_variants":["LLM hints, not orders, cut collision rate by 11.4%","Advisory AI cuts driving collisions by 11.4% in CARLA","LLM as advisor, not driver: 80.3% task success, 11.4% fewer crashes","Language model hints boost driving success to 80.3%, cut crashes 11.4%","HCRMP: LLM suggests, RL decides—collisions drop 11.4%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the language model's semantic hints are informative enough on average, and wrong often enough only in ways the RL update can treat as noise, so the scaled advantage in the PPO objective still points toward better driving.","fun_headline_variants_meta":{"raw":{"variants":["LLM hints, not orders, cut collision rate by 11.4%","Advisory AI cuts driving collisions by 11.4% in CARLA","LLM as advisor, not driver: 80.3% task success, 11.4% fewer crashes","Language model hints boost driving success to 80.3%, cut crashes 11.4%","HCRMP: LLM suggests, RL decides—collisions drop 11.4%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000422,"raw_usage":{"total_tokens":2195,"prompt_tokens":998,"completion_tokens":1197,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":614,"completion_tokens_details":{"reasoning_tokens":1079}},"tokens_in":614,"tokens_out":1197,"duration_ms":8126,"temperature":1.0,"reasoning_tokens":1079,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:10:42.656382+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the LLM's semantic hints in the reported simulator runs with deliberately wrong hints, such as reversed scenario classifications and swapped critic weights, and measure success and collision rates; if performance collapses, the claim that RL absorbs hallucinations is not supported, whereas if it stays near the reported levels, the buffering is real.","supporting_citations":[{"cited_title":"Autoreward: Closed-loop reward design with large language models for autonomous driving","cited_arxiv_id":null,"evidence_quote":"Is the LLM-dominated reward-design baseline whose collision and success numbers HCRMP is compared against."}],"review_version":1}