{"id":"756c37fa-64fe-4d43-93da-125c538e1a76","arxiv_id":"2607.10438","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"An IL planner with agent-centric data reuse, complexity-aware async LLM semantics, and residual differentiable optimization reports top nuPlan Hard20 closed-loop scores and real-time CARLA-ROS execution.","lead":"This paper builds an autonomous-driving planner that mixes imitation learning, a complexity-triggered LLM for scene meaning, and a differentiable trajectory optimizer. On hard nuPlan closed-loop tests it reports the best overall scores among the baselines it compares, and a CARLA-ROS setup shows it can run online.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.5","headline":"The headline Hard20 gains rest on a hand-tuned complexity scheduler whose necessity is only weakly isolated from the larger residual-optimizer contribution.","rationale":"The paper is a competent empirical systems integration: agent-centric label reuse, residual Theseus refinement with backprop, and async LLM features are clearly described, and Tables II–III plus component ablations give real support for strong Hard20 numbers. I agree with the reader that the hand-tuned C_t / K schedule is a soft spot and that missing variance, code, and thin CARLA evidence keep the verdict CONDITIONAL rather than ACCEPT. I only partially agree on which assumption is most load-bearing: the scheduler is a free-parameter risk, but the stronger threat to the strongest claim is attribution—whether complexity-aware refresh (vs. fixed async or no LLM) is actually necessary for the reported best scores under real-time budgets. Tables IV–V already hint that Diff. Opt. + aug carry most of the gain and that fixed high-frequency LLM can score higher. A single fixed-schedule re-run would settle whether the proposed adaptive policy is essential or mainly a latency knob. That does not overturn the empirical ranking under the authors' chosen settings, so the verdict stays CONDITIONAL; it does not warrant REJECT or UNCHANGED without that check.","tokens_in":19615,"tokens_out":812,"duration_ms":7599,"concrete_test":"Re-evaluate the full model on nuPlan reactive Hard20 with the LLM forced to a fixed 9-frame schedule (Table V medium-frequency setting) and report Score, Coll., TTC, and wall-clock ms under identical hardware/batch settings as Table V. If Score falls below PLUTO (76.88) or the gap to M2 shrinks below ~2 points while runtime stays near 155 ms, the complexity-aware scheduler is not load-bearing for the headline ranking; if Score remains ≥78 and still leads PLUTO at matched latency, the concern is largely defused.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that the full stack (agent-centric augmentation + complexity-aware AsyncLLM + residual differentiable optimization) is what produces the best Hard20 closed-loop scores (83.63 nonreactive / 78.29 reactive). The reader's weakest assumption correctly flags the hand-tuned complexity score C_t (Eq. 7, equal α_i, τ_l=0.35, τ_h=0.65, K∈{3,9,29}) as a free-parameter proxy for when LLM semantics must be refreshed. That concern is real, but the more load-bearing issue for the strongest claim is that the ablations do not cleanly show the scheduler is required for the headline ranking. Table IV shows M2 (aug + Diff. Opt., no LLM) already reaches 75.19 reactive; full M3 only adds +3.1. Table V shows complexity-aware AsyncLLM (78.29 / 172 ms) is close to fixed-3-frame (79.61 / 237 ms) and well below synchronous (80.46 / 477 ms). Table VII shows threshold and factor ablations move score by a few points, but never re-ranks the method against PLUTO (76.88) under a fixed schedule with matched compute. Thus the claim that the proposed complexity-aware design is what enables the best overall scores under real-time budgets is only partially supported: most of the lift is from residual optimization and augmentation, while the scheduler mainly trades a small quality drop for latency. Without a fixed-schedule, matched-budget re-run of the full model, the IoT/real-time framing of the strongest claim over-attributes performance to the hand-tuned policy.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes an imitation-learning trajectory planner for IoT-enabled autonomous driving that combines three components: (i) surrounding-agent-centric data augmentation that reindexes filtered non-ego trajectories as additional planning supervision, (ii) a complexity-aware asynchronous LLM module that injects scene-associated semantic features with adaptive reuse lengths, and (iii) a residual-based differentiable nonlinear optimizer (Theseus LM) that refines the selected ego trajectory and backpropagates optimization gradients to the upstream planner. On nuPlan closed-loop Hard20, the full system reports best overall scores of 83.63 (nonreactive) and 78.29 (reactive) among the listed baselines, with component ablations, scheduler/residual studies, qualitative closed-loop comparisons, and CARLA-ROS SIL tests supporting online 5 Hz deployment.","tokens_in":20063,"tokens_out":1238,"duration_ms":16735,"significance":"If the reported closed-loop gains hold under fair comparison, the work is a solid systems contribution at the intersection of IL planning, LLM semantic guidance, and differentiable trajectory refinement. Strengths include full nuPlan Hard20 closed-loop tables (II–III), stepwise component ablations (IV), augmentation design (VI), LLM schedule and scheduler-factor studies (V, VII), residual-category ablations (VIII), and a separate CARLA-ROS stack that demonstrates real-time closed-loop execution rather than offline metrics alone. The residual optimizer with gradient flow through LM iterations and the agent-centric reuse of logged trajectories are practically useful ideas even if the LLM scheduler is secondary. The IoT/real-time framing is relevant to IEEE IoT Journal, but the significance of the complexity-aware LLM design specifically depends on cleaner isolation from the larger optimizer/augmentation lift.","major_comments":[{"comment":"The strongest claim attributes best Hard20 scores under real-time budgets to the full stack, especially complexity-aware AsyncLLM. Table IV shows most reactive gain already at M2 (aug + Diff. Opt.: 75.19 vs M0 60.43); full M3 adds only +3.1 to 78.29. Table V shows complexity-aware AsyncLLM (78.29 / 172 ms) is below fixed-3-frame (79.61 / 237 ms) and synchronous (80.46 / 477 ms). Without a fixed-schedule, matched-budget re-run of the full model that still ranks above PLUTO (76.88 reactive), the IoT/real-time framing over-attributes the headline ranking to the hand-tuned scheduler rather than residual optimization and augmentation.","section":"§V-A, Tables II–V"},{"comment":"Eq. (7) defines C_t with equal α_i and fixed thresholds τ_l=0.35, τ_h=0.65 and K∈{3,9,29} (Table I). Table VII shows threshold/factor changes move score by a few points, but the scheduler remains a free-parameter rule without a learned policy or validation that these proxies are necessary for the claimed ranking under a fixed compute budget. Please either (a) report matched-budget fixed-K full-model results, or (b) soften claims that complexity-aware scheduling is what enables best overall real-time performance.","section":"§III-C, Eq. (7), Tables I, V, VII"},{"comment":"Safety residual (Eq. 15) uses a selective local corridor with a fixed +5.0 m buffer and explicitly defers rear-end risk to upstream prediction/closed-loop feedback. Given that reactive Hard20 emphasizes interaction, please quantify how often rear-end or multi-agent conflicts fall outside the selected corridor, and whether residual design choices (not only LLM scheduling) drive Coll./TTC differences vs PLUTO in Tables II–III.","section":"§III-D, Eq. (15), Tables II–III, VIII"}],"minor_comments":[{"comment":"Fig. 1 scenario taxonomy (SST/CIST/CST/RST) is used to motivate long-tail imbalance, but the manuscript does not state how these labels are assigned or whether augmentation preferentially samples RST/CST.","section":"Fig. 1, §I, §III-B"},{"comment":"CARLA-ROS reports 80% success, <5% collision, <30 s traversal at 5 Hz, but scenario count, route set, and comparison against a non-LLM or non-optimizer baseline on the same SIL stack are not specified.","section":"§V-B, Fig. 5"},{"comment":"Several loss weights and LM solver settings are listed as free parameters; a short sensitivity note (beyond residual-category removal) would help reproducibility.","section":"§III-E, §IV-B, Table I"},{"comment":"Minor presentation: spacing/hyphenation issues in the abstract (“sur rounding”, “asyn chronous”) and occasional notation overload (K^sem_t vs Δt) should be cleaned.","section":"Abstract, §III-C"}],"recommendation":"major_revision","confidential_remarks":"Fit for IEEE IoT Journal is plausible via the real-time/IoT framing, but novelty relative to PLUTO + AsyncDriver is incremental; the main risk is overselling the hand-tuned LLM scheduler. I would accept after the authors either provide matched-budget fixed-schedule full-model results or reframe the contribution so residual optimization and agent-centric augmentation carry the primary claim. No integrity concerns; empirical package is above average for this area."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is straightforward: they stack three known pieces—IL planning, residual differentiable refinement, and async LLM features—add agent-centric label reuse, and post the best Hard20 closed-loop numbers among the baselines they report (83.63 nonreactive / 78.29 reactive), with a CARLA-ROS SIL check that the stack actually runs closed-loop.\n\nWhat is actually new is the combination and two design choices, not a new planning theory. Surrounding-agent-centric augmentation reindexes real non-ego trajectories as extra planning supervision after interaction-aware filtering; that is a clean data-centric move and the ablations show it helps. The residual optimizer (Theseus LM, soft efficiency/comfort/safety/kinematic terms, gradients back to the planner) is the workhorse: Table IV jumps from ~65 to 75.19 once it is added. The adaptive gate on LLM cross-attention is a sensible detail. Writing is clear, tables are complete, residual categories and schedule ablations are present, and the citation line (PlanTF, PLUTO, AsyncDriver, DIPP-style work) is honest.\n\nSoft spots, in proportion. The complexity score C_t is hand-tuned (equal α_i, fixed τ and K), and the stress-test note is right that the scheduler is not cleanly load-bearing for the headline ranking: M2 without LLM is already near PLUTO, full model adds ~3 points, and fixed high-frequency LLM scores higher at higher latency. So the IoT/real-time framing over-attributes the win to the adaptive policy; most of the lift is augmentation + residual opt. Safety residual is selective and soft (+5 m buffer, local corridor), CARLA is thin (5 Hz, success/collision anecdotes, no full protocol), no variance or code, and many free parameters. None of that collapses the empirical claim; it just means the strongest sentence in the abstract should be read as “this stack scores best under our settings,” not “complexity-aware LLM is the reason.”\n\nThis is for people building IL + refinement planners who care about Hard20 and online latency tradeoffs. It is not a theory paper and not a deployment paper. I would send it to peer review; the evidence is real enough for referees to argue over attribution and baselines. Worth reading if you work in this line; I would cite the numbers and the augmentation idea, not the scheduler as a general solution.","headline":"Solid hybrid planner with real Hard20 gains; most lift is from residual optimization and agent-centric labels, not the hand-tuned LLM scheduler the abstract foregrounds.","tokens_in":20758,"tokens_out":588,"would_cite":true,"duration_ms":7840,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A planner that reuses nearby drivers as teachers, refreshes language scene cues only when traffic gets hard, and trains through a differentiable trajectory optimizer tops the nuPlan Hard20 closed-loop scores.","keywords":["imitation learning","differentiable optimization","large language model","trajectory planning","connected autonomous driving","data augmentation","closed-loop evaluation","IoT intelligent transportation"],"falsifier":"On the same Hard20 reactive split, replace the fixed thresholds and reuse lengths with a learned or oracle refresh policy (or force full synchronous language-model updates) and check whether overall score or safety metrics rise enough to erase the claimed advantage while runtime stays inside the online envelope.","tokens_in":20431,"feed_emoji":"🚗","tokens_out":948,"duration_ms":14148,"temperature":0.7,"pith_summary":"Imitation-learning planners for city driving often miss rare multi-agent interactions, ignore high-level scene meaning under real-time limits, and train without feedback from the constraints that will later refine their trajectories. This paper claims those three gaps can be closed together without new raw data or constant large-model calls. Surrounding vehicles that already interact in complex ways are recentered as extra planning subjects so one log episode yields more hard training cases. A large language model supplies scene-and-instruction features, but a simple complexity score decides how often to refresh them so latency stays controlled. A residual optimizer then reshapes the chosen trajectory under speed, comfort, safety, and bicycle-model costs, and because the solver is differentiable those costs train the upstream network. On the hard closed-loop splits the combined system records the highest overall scores among the compared methods, and a live CARLA-ROS loop confirms it can run online.","feed_headline":"Planner reuses nearby drivers, sparse LLM cues, tops Hard20","feed_subtitle":"Agent-centric logs, complexity-gated semantics, and residual optimization hit 83.63 / 78.29 closed-loop","key_machinery":"The residual-based differentiable optimizer: a Levenberg–Marquardt solver that refines the highest-confidence ego trajectory under soft penalties for speed, reference-line consistency, comfort, selective safety buffers, and bicycle kinematics, and that back-propagates residual gradients into the upstream planner and cost-weight network so generation and refinement are learned jointly.","core_discovery":"The authors establish that an imitation planner trained with surrounding-agent-centric trajectory reuse, complexity-aware asynchronous language-model semantic features, and residual differentiable optimization produces safer, more feasible closed-loop trajectories than strong baselines on nuPlan Hard20, reaching overall scores of 83.63 (nonreactive) and 78.29 (reactive) while remaining deployable in real time.","pith_inferences":["The same agent-reindexing idea could be applied to other multi-agent logs (pedestrians, cyclists, or mixed fleets) wherever rare interactions dominate failure modes.","A learned complexity policy trained to maximize planning score per unit of language-model latency would test how much headroom remains in the hand-tuned scheduler.","Because gradients flow through the residual solver, the same pattern could be used to co-train prediction and planning under richer multi-directional safety costs without changing the online inference graph."],"forward_implications":["Existing logged multi-agent episodes can be turned into denser long-tail planning supervision simply by re-indexing surrounding vehicles as ego, without new collection.","High-level language semantics can be injected into real-time planners if a lightweight complexity gate keeps invocation frequency low in simple traffic.","Training through a differentiable residual optimizer aligns the network’s proposals with the same constraints used at execution time, reducing the usual train–refine mismatch.","Closed-loop Hard20 leadership plus a working CARLA-ROS loop implies the stack is already a practical candidate for software-in-the-loop IoT vehicle testing."],"fun_headline_variants":["Agent reuse and gated LLM cues lift planner to Hard20 83.63/78.29","Surrounding-path training plus residual opt tops nuPlan Hard20","Sparse async LLM semantics refine imitation trajectories to 83.63","Complexity-aware LLM features and agent logs hit 83.63 closed-loop","Differentiable residual planner with LLM cues scores 83.63/78.29"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That a hand-tuned rule scoring traffic density, conflicts, time-to-collision, intersections, navigation changes, and short-term variation is a good enough proxy for when language-model semantics must be refreshed, so reusing the last feature for many frames still keeps planning quality under a real-time budget.","fun_headline_variants_meta":{"raw":{"variants":["Agent reuse and gated LLM cues lift planner to Hard20 83.63/78.29","Surrounding-path training plus residual opt tops nuPlan Hard20","Sparse async LLM semantics refine imitation trajectories to 83.63","Complexity-aware LLM features and agent logs hit 83.63 closed-loop","Differentiable residual planner with LLM cues scores 83.63/78.29"]},"model":"grok-4.5","effort":"low","cost_usd":0.004252,"raw_usage":{"total_tokens":1310,"prompt_tokens":806,"num_sources_used":0,"completion_tokens":86,"cost_in_usd_ticks":42520000,"prompt_tokens_details":{"text_tokens":806,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":418,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":806,"tokens_out":86,"duration_ms":3908,"temperature":1.0,"reasoning_tokens":418,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T11:43:24.510975+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On the same Hard20 reactive split, replace the fixed thresholds and reuse lengths with a learned or oracle refresh policy (or force full synchronous language-model updates) and check whether overall score or safety metrics rise enough to erase the claimed advantage while runtime stays inside the online envelope.","supporting_citations":[],"review_version":1}