{"id":"3fbe8dab-3158-4a39-b67a-9c6ce62c0ba8","arxiv_id":"2502.01387","paper_version":3,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A teacher LLM guides an attention-based DRL student for autonomous driving, reporting improved success rates in simulation, though with several unstated implementation details.","lead":"TeLL-Drive blends a large language model 'teacher' with a deep reinforcement learning 'student' to make driving decisions, using the LLM's reasoning to guide early training. The paper reports higher success rates than four baselines in three simulated traffic scenarios, but the gains are inconsistent across metrics and the method leaves key details unspecified.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table II contradicts the abstract's 'average returns' claim: Dilu's Eval Reward is higher than TeLL-Drive's in the intersection (6.59 vs 5.68) and highway (29.53 vs 27.17) scenarios, so the central claim is unsupported as stated.","rationale":"I read the paper as proposing a teacher-student framework and claiming empirical superiority over DRL and LLM baselines. The most load-bearing condition for that claim is that the paper's own experiments support it. They do not do so without qualification: Table II's Eval Reward entries contradict the abstract's 'average returns' statement. This is not a matter of differing from external consensus; it is an internal inconsistency between the stated conclusion and the reported numbers. I do not read this as evidence of bad faith, but the manuscript as written asks the reader to accept a claim that its own table refutes. The success-rate and decision-time portions of the claim are supported by Table II, but the average-return portion is not, and the paper does not explain the discrepancy. I also examined the KL/pi_T gap. It is a genuine reproducibility and soundness problem: Eq. (13) is a constrained objective involving pi_T, but nothing in Algorithm 1 or Section IV defines a distribution over the discrete action set A. Section IV.C.2 introduces a learned pi_hat_T, but the text does not state that Eq. (13)'s pi_T equals pi_hat_T, nor how the constraint is enforced. If pi_T were a one-hot distribution, the KL divergence with the student policy would be undefined whenever the student assigns positive probability to an unselected action; if it were token-level softmax, the support would not be A. That gap would prevent faithful reimplementation. However, the empirical contradiction is more immediately load-bearing for the central claim, which is why this critique focuses there. The framework idea is plausible at a high level, and the vehicle-in-loop demonstration is a positive step, but the current text requires either a corrected abstract or additional evidence for the average-return claim. The reader's REJECT verdict stands. I do not see a basis to soften it.","tokens_in":13512,"tokens_out":7060,"duration_ms":66904,"concrete_test":"Request the exact reward definition and per-seed evaluation-return logs, then recompute Table II's 'Eval Reward' for Dilu and TeLL-Drive under an identical reward function. If Dilu remains higher in the intersection and highway rows, the abstract's 'outperforms ... average returns' claim is false; if the table's 'Eval Reward' is not the 'average return' claimed in the abstract, the paper must state the metric actually used and supply the missing comparison.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract) is that TeLL-Drive outperforms baselines, including LLM-based Dilu, on success rate, average return, and real-time feasibility. The paper's own evaluation in Section V.C.1, Table II, undermines the 'average return' conjunct. In the unsignalized intersection, Dilu's Eval Reward is 6.59 versus TeLL-Drive's 5.68; in four-lane adaptive cruise, Dilu's is 29.53 versus TeLL-Drive's 27.17. Only in high-speed ramp merging does TeLL-Drive (5.60) exceed Dilu (3.21). The text introduces Table II as a numerical summary of success rate, evaluation return, average speed, TTCP, and decision-making time, so unless 'average return' in the abstract is a different quantity, the headline claim is contradicted. Compounding this, no reward function is defined in Section III or Section IV; without it, the reader cannot tell whether 'Eval Reward' is a shared, comparable quantity or whether 'average return' refers to the training curves in Fig. 5. A second, related gap is Eq. (13): the KL constraint requires a teacher distribution pi_T, but the LLM Teacher (Algorithm 1) emits an action, and Section IV.C.2 later states that the teacher does not directly provide action probabilities, introducing only a learned approximation pi_hat_T via Eq. (15) without connecting it to Eq. (13). If pi_T were one-hot, the KL would be degenerate; if it were token-level softmax, it would not be a distribution over the action space A. This gap blocks implementation, but the Table II mismatch is the most direct threat to the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TeLL-Drive, a hybrid decision-making framework for autonomous driving in which a GPT-4o-mini-based Teacher LLM provides high-level action guidance to an attention-based student deep reinforcement learning (DRL) agent. The method constructs risk-aware prompts using time-to-conflict-point estimates, memory retrieval, and chain-of-thought reasoning, and constrains the student policy toward the teacher's demonstrations via a KL-divergence term, with a learned approximation of the teacher's policy. The authors evaluate the method in three Highway-Env scenarios (unsignalized intersection, ramp merging, four-lane adaptive cruise) against DQN, A2C, RecurrentPPO, and the LLM-based Dilu, and report simulation results, an ablation study, and a qualitative vehicle-in-loop experiment. The main claimed contributions are improved success rate, average return, and real-time feasibility over existing baselines.","tokens_in":13899,"tokens_out":5655,"duration_ms":48121,"significance":"If the performance claims were fully supported, the idea of using an LLM teacher to accelerate DRL convergence and improve robustness in driving scenarios would be of practical interest to the autonomous-driving and robot-learning communities. The paper includes a useful comparison against both conventional DRL and an LLM-based baseline, a clear three-scenario evaluation, and an ablation that isolates the contribution of the attention mechanism and the LLM guidance. The vehicle-in-loop setup is a positive step toward real-world validation. However, the current manuscript does not substantiate the headline claim on average returns, and the undefined teacher distribution and reward function prevent the method from being reproduced or properly assessed. With these gaps closed, the framework could become a solid contribution; in its present form the evidence is insufficient.","major_comments":[{"comment":"The abstract claims that TeLL-Drive outperforms baselines \"in terms of success rates, average returns, and real-time feasibility,\" but Table II shows Dilu attaining higher Eval Reward in both the unsignalized intersection (6.59 vs. 5.68) and four-lane adaptive cruise (29.53 vs. 27.17) scenarios. The text introducing Table II calls these columns \"Evaluation Return,\" which is the paper's apparent measure of average return. The authors must clarify what \"average returns\" in the abstract refers to, and either correct the claim, re-run the comparison, or present a consistent metric across all scenarios that supports the stated superiority.","section":"Section V.C.1, Table II"},{"comment":"The constrained optimization uses a KL divergence between the student policy πS(s_t) and a teacher policy πT(s_t), but Algorithm 1 outputs only a discrete action a_t ∈ A, not a probability distribution over the action space. Section IV.C.2 explicitly states that the teacher does not directly provide action probabilities and instead introduces a learned approximation π̂T via Eq. (15). The paper never specifies how πT in Eq. (13) is obtained (e.g., one-hot distribution, token-level softmax, or the learned π̂T), nor how the constraint is enforced during optimization. Without this, the central guidance mechanism is not implementable as described.","section":"Section IV.C, Eq. (13)"},{"comment":"The reward function r(s_t, a_t) is listed as part of the POMDP tuple in Section III but is never defined or described anywhere in the manuscript. This leaves the \"Eval Reward\" in Table II and the \"average return\" in Fig. 5 without a verifiable meaning, and prevents the reader from confirming that all baselines were optimized under the same reward. The authors should provide the reward specification (or explicitly reference the Highway-Env default reward and any modifications) and state whether the same reward is used for the teacher's guidance and the student's objective.","section":"Section III and Section IV"},{"comment":"The TTCP (time-to-conflict-point) metric is defined inconsistently. Eq. (4) writes τ = arg min_{t≥0} || p_ego(t)/v_ego(t) − p_other(t)/v_other(t) ||, which is dimensionally ambiguous and not a standard collision measure, while Algorithm 1 line 4 uses τ_i = arg min_{Δt≥0} || p_ego(t+Δt) − p_i(t+Δt) ||, a different quantity. Since TTCP drives the risk metric in Eq. (8) and the teacher's prompting, this inconsistency must be corrected and a precise, implementable definition provided.","section":"Section IV.B.1, Eq. (4) and Algorithm 1"}],"minor_comments":[{"comment":"The phrase \"gradient verification scenario\" in the Introduction and Section V.A appears to be a typo; it should probably read \"graded verification scenario\" or \"progressively difficult scenario.\"","section":"Section I, Section V.A"},{"comment":"Table I does not report values for the risk weighting β, the high-risk threshold δ, or the KL tolerance σ, even though these parameters appear in the algorithm's key equations; please include them for reproducibility.","section":"Table I vs. Eqs. (8) and (13)"},{"comment":"The notation for the teacher policy is inconsistent: Eq. (13) and its caption use πT, while the immediately following sentence refers to πE; standardize the symbol.","section":"Eq. (13) and surrounding text"},{"comment":"The Abstract and Section VI refer to \"Our Website\" and a footnote reads \"Vehicle-in-Loop Experimental Validation Video Weblink,\" but no actual URL is provided; include a working link or remove the reference.","section":"Abstract and Section VI"},{"comment":"The caption contains a typo: \"TeLL-Dirve\" should be \"TeLL-Drive.\"","section":"Fig. 9 caption"},{"comment":"The text says \"proposes a domin-specific adjustment\"; this should be \"domain-specific adjustment.\"","section":"Section IV.B.3"},{"comment":"The ablation study is presented only as training curves in Fig. 6 without a quantitative table of final rewards or success rates; adding end-of-training numbers would strengthen the comparison.","section":"Section V.C.2"},{"comment":"The vehicle-in-loop experiments are reported qualitatively through two cases; no quantitative metrics (e.g., success rate, task completion time, safety margin) are provided for the real-vehicle runs, so the claim of \"robustness and reliability\" is not quantified.","section":"Section VI.B"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has no code or data release, and the promised \"Our Website\" and video weblink are placeholders; if reproducibility is expected, the authors should be required to provide a working link or repository. The main concerns from the reader's report—particularly the Table II mismatch with the abstract and the undefined teacher distribution in Eq. (13)—are also present in my reading and are significant enough to block acceptance in the current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper builds a teacher-student framework where a GPT-4o-mini teacher, prompted with risk metrics and memory retrieval, guides a PPO student via a KL constraint and self-attention fusion. The idea is sensible, and the ablation shows the attention and LLM guidance help in these Highway-Env scenarios. Success rates are genuinely higher than the baselines, including Dilu, and the vehicle-in-loop demo is a nice bonus.\n\nThe central claim, however, is overstated. The abstract says TeLL-Drive beats baselines on average returns, yet Table II shows Dilu's Eval Reward is higher in two of the three scenarios: 6.59 vs 5.68 in the unsignalized intersection and 29.53 vs 27.17 on the four-lane highway. Only in ramp merging does TeLL-Drive lead. Unless \"average return\" means the training curves in Fig. 5, the claim is contradicted by the paper's own evaluation. That is a serious accuracy problem.\n\nThe method also has a load-bearing gap. Equation (13) constrains the student policy with a KL divergence to a teacher policy πT, but the LLM teacher emits a discrete action, not a distribution. Section IV.C.2 admits the teacher does not provide action probabilities and introduces a learned approximation π̂_T via Eq. (15), but never connects it back to Eq. (13). As written, the KL constraint is not implementable. The reward function R is never defined anywhere, so the reader cannot tell what \"Eval Reward\" measures or why the teacher guidance should align with the RL objective. These are fixable in revision, but they block reproduction now.\n\nThe paper does not release code or data, which makes the quantitative claims hard to verify independently. The vehicle-in-loop section is a qualitative case study rather than a rigorous evaluation.\n\nThat said, the core idea has merit, and the success-rate improvements are not trivially explained by the baselines. A serious referee could help the authors clean up the definitions and re-run the comparison. I would not cite this in its current form, but I would not dismiss the direction either.\n\nRecommendation: send to peer review, with the expectation that reviewers will demand substantial revision: define πT, state the reward, and reconcile the abstract's average-returns claim with Table II.","headline":"A sensible teacher-student idea with real success-rate gains, but the abstract overclaims on average returns and the KL constraint is under-specified.","tokens_in":14464,"tokens_out":2746,"would_cite":false,"duration_ms":24111,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A teacher LLM guides a deep-reinforcement-learning student to success rates of 88–100% across three driving tasks.","keywords":["autonomous driving","large language models","deep reinforcement learning","teacher-student learning","self-attention policy fusion","sample efficiency","vehicle-in-the-loop testing"],"falsifier":"Hold everything fixed except replace the LLM teacher's advice with uniformly random maneuvers drawn from the same five-action set, and retrain the student under the same KL schedule and attention architecture. If the random-teacher version still reaches roughly 88%, 91%, and 100% success in the three scenarios, then the content of the LLM's reasoning is not what produces the reported gains; if the random-teacher version collapses, the teacher's semantic guidance is load-bearing.","tokens_in":1985,"feed_emoji":"🚗","tokens_out":7676,"duration_ms":105802,"temperature":0.7,"pith_summary":"TeLL-Drive claims that the two standard weaknesses of autonomous-driving agents—DRL's slow, sample-hungry learning and LLMs' slow, random decision-making—can be resolved by pairing them as teacher and student. A large language model is prompted with risk metrics, retrieved past scenarios, and driving heuristics to propose high-level maneuvers; an attention-based deep reinforcement learning agent uses those proposals as a soft constraint during early training and then acts as the fast final decision maker. The paper reports that this hybrid reaches 88% success in unsignalized left turns, 91% in high-speed ramp merging, and 100% in four-lane adaptive cruise, beating DRL baselines and the LLM-based Dilu baseline while keeping per-decision inference under about four milliseconds. If true, the framework shows that LLM common-sense reasoning can be used as a training-time bootstrap without sacrificing the real-time response that safety-critical driving demands.","feed_headline":"Teacher LLM gets self-driving agent to 88–100% success","feed_subtitle":"Hybrid framework uses LLM reasoning to guide early training, then a fast DRL student decides in real time.","key_machinery":"The load-bearing mechanism is the constrained fusion of two policies: a teacher policy $\\pi_T$, approximated by a learned teacher embedding $h_t^T$ built from the LLM's discrete action suggestions, and a student policy $\\pi_S$ built on its own embedding $h_t^S$. A multi-head self-attention layer computes $h_t = \\alpha_t W_V h_t^T + h_t^S$ and feeds the fused representation into the student's policy and Q estimates, while Eq. (13) constrains the student by $D_{KL}(\\pi_S(s_t), \\pi_T(s_t)) \\le \\sigma$, with $\\sigma$ small early and larger later. Around this core sit a risk-aware prompt constructor that uses TTCP, memory retrieval, and chain-of-thought reasoning; a reflective evaluator that flags episodes with $\\max_t \\Omega(s_t, a_t) \\ge \\delta$ and writes revised policies back to memory; and an auxiliary distillation loss that teaches the teacher embedding to reproduce the LLM's choices. The attention mechanism is what lets the student keep its own learned strategy instead of being overwritten by the teacher.","core_discovery":"The paper's central claim is that high-level LLM reasoning and low-latency DRL control are complements, not competitors, and that a teacher-student architecture can capture both. The LLM teacher never drives the vehicle; it produces contextual maneuver proposals from prompts built on a Time-to-Conflict-Point risk metric, cosine-similarity retrieval from a memory repository, and chain-of-thought reasoning. The DRL student is an actor-critic whose objective includes a KL-divergence constraint (Eq. 13) that keeps the student policy near the teacher's proposals early in training, with the bound loosened as learning proceeds; a multi-head self-attention layer fuses a learned teacher embedding with the student's own state embedding, and an auxiliary distillation loss (Eq. 22) makes the teacher embedding approximate the LLM's implicit policy. The paper reports that the student surpasses the teacher in all three test scenarios and that the full framework beats DQN, A2C, RecurrentPPO, and Dilu on success rate and return while keeping decision time at 0.003–0.004 seconds, and that vehicle-in-loop experiments at an unsignalized intersection confirm safe yielding and priority-taking behaviors on real hardware.","pith_inferences":["A testable extension the paper leaves implicit: any cheap expert signal—rule-based heuristics, human demonstrations, or a smaller model—might yield most of the same early-training boost, since the attention and KL mechanism do not logically require an LLM; comparing teacher sources would isolate what the LLM's reasoning adds.","The paper's KL constraint requires a distribution $\\pi_T$ that the LLM never directly emits; until the paper specifies whether $\\pi_T$ is a one-hot over the chosen action, a token-probability softmax, or a separately trained estimator, the mechanism in Eq. (13) is underdetermined and reimplementation is not uniquely defined.","The memory repository and reflective evaluator produce a trace of which past scenarios and risk thresholds influenced each maneuver, which could be repurposed as an interpretability and safety-audit log for the deployed policy.","Because the student surpasses the teacher, the framework suggests a general pattern: expert guidance should be scheduled to decay, and the final policy should be evaluated independently of the teacher; future multi-agent or open-road studies could test whether this decay schedule needs retuning in more interactive traffic."],"forward_implications":["If the reported results hold, autonomous driving planners can get LLM-level reasoning without LLM latency: the teacher is used only in the first 10% of training, and the student makes decisions in about 0.004 seconds.","Training data requirements shrink because the KL constraint steers exploration early, which addresses DRL's sample-complexity bottleneck in safety-critical scenarios.","The teacher-student split means the expensive LLM can be swapped or replaced after training, and the deployed vehicle needs only the student policy network.","The student's consistent outperformance of the teacher implies LLM guidance is a bootstrap rather than a performance ceiling, so the framework can in principle improve as the DRL agent accumulates experience.","Vehicle-in-loop demonstrations at an unsignalized intersection suggest the approach transfers from simulation to real hardware for at least one high-risk scenario."],"supporting_citations":[{"why":"It supplies the GPT-4o-mini backbone used as the LLM teacher.","marker":"[11]"},{"why":"It supplies the Time-to-Conflict-Point risk metric and the interactive LLM decision paradigm the teacher builds on.","marker":"[23]"},{"why":"It is the LLM-based Dilu baseline that TeLL-Drive must outperform on success rate, return, and latency.","marker":"[26]"},{"why":"It justifies the chain-of-thought prompting used to stabilize LLM reasoning.","marker":"[31]"},{"why":"It introduces the expert-prior KL divergence constraint that becomes Eq. (13)'s guidance mechanism.","marker":"[33]"},{"why":"It provides the simulation scenarios used for all training and evaluation.","marker":"[34]"},{"why":"It defines PPO, the base algorithm behind the Vanilla-PPO ablation and the student's actor-critic core.","marker":"[18]"},{"why":"It is the value-based DQN baseline.","marker":"[19]"},{"why":"It is the policy-gradient A2C baseline.","marker":"[35]"},{"why":"It is the recurrent-memory RecurrentPPO baseline.","marker":"[36]"}],"fun_headline_variants":["Teacher LLM boosts DRL driving success and real-time control","LLM-teacher + DRL-student: hybrid driving outdoes baselines","LLM coaches DRL for safer, faster autonomous driving","TeLL-Drive: LLM-guided RL hits real-time and high success"],"cache_read_input_tokens":16384,"weakest_assumption_plain":"The load-bearing premise is that the LLM's discrete action outputs can be converted into a well-defined teacher distribution $\\pi_T$ for the KL constraint in Eq. (13), but the paper never specifies how that distribution is formed.","fun_headline_variants_meta":{"raw":{"variants":["Teacher LLM boosts DRL driving success and real-time control","LLM-teacher + DRL-student: hybrid driving outdoes baselines","LLM coaches DRL for safer, faster autonomous driving","TeLL-Drive: LLM-guided RL hits real-time and high success"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000236,"raw_usage":{"total_tokens":1539,"prompt_tokens":1013,"completion_tokens":526,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":629,"completion_tokens_details":{"reasoning_tokens":449}},"tokens_in":629,"tokens_out":526,"duration_ms":5711,"temperature":1.0,"reasoning_tokens":449,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T15:27:58.906287+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold everything fixed except replace the LLM teacher's advice with uniformly random maneuvers drawn from the same five-action set, and retrain the student under the same KL schedule and attention architecture. If the random-teacher version still reaches roughly 88%, 91%, and 100% success in the three scenarios, then the content of the LLM's reasoning is not what produces the reported gains; if the random-teacher version collapses, the teacher's semantic guidance is load-bearing.","supporting_citations":[{"cited_title":"Efficient deep reinforcement learning with imitative expert priors for autonomous driving","cited_arxiv_id":null,"evidence_quote":"It introduces the expert-prior KL divergence constraint that becomes Eq. (13)'s guidance mechanism."}],"review_version":1}