{"id":"9557a869-8c65-44cf-a068-569873f24d0c","arxiv_id":"2607.02841","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Residual waypoint RL around a frozen VLA prior, scaled via heterogeneous CARLA/H100 infrastructure, raises closed-loop driving score and success rate on longest6 v2 and Bench2Drive.","lead":"CLEAR finetunes vision-language-action driving models with closed-loop RL by learning residual corrections to pretrained waypoints, using a heterogeneous sim/learner pipeline to scale to 64 parallel CARLA environments. It reports large gains on long-horizon CARLA routes where pure imitation learning fails.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"SOTA gains rest on residual corrections to a frozen VLA prior under a metric-aligned reward; without multi-seed variance or controller ablation the closed-loop claim is under-supported.","rationale":"The Reader correctly isolates the residual-prior + frozen-backbone + metric-aligned-reward premise as the weakest link and assigns a CONDITIONAL verdict with MODERATE confidence. My stress test finds the same load-bearing assumption: the empirical SOTA numbers are impressive but rest on single-run ablations and a reward known to optimize the evaluation metric. No stronger internal inconsistency appears—the residual formulation is coherent, the heterogeneous pipeline is a genuine systems contribution, and the paper does not over-claim real-world closed-loop driving. Therefore the Reader’s verdict needs no change; the concrete multi-seed + residual-magnitude check would simply convert the current conditional acceptance into a firmer one if the numbers hold, or expose fragility if they do not.","tokens_in":15299,"tokens_out":610,"duration_ms":6188,"concrete_test":"Re-train CLEAR (InternVL3-1B residual) for three independent random seeds under the exact 100 M-sample / 16384-batch protocol of Table 5; report mean ± std of DS/SR/RC on longest6 v2. If any seed falls below DS 30 or SR 15 %, or if the residual-magnitude histogram shows frequent clipping at the Γ bounds of Eqs. 16–17, the claim that bounded residuals around a frozen prior reliably recover closed-loop competence is weakened.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim (residual waypoint RL around a frozen InternVL3-1B prior + 64-env heterogeneous pipeline yields SOTA closed-loop DS/SR) depends on the premise in §3.3 that the open-loop waypoint prior remains a sufficiently good base plan so that only m=4 clipped residual deltas (lateral offsets + speed) need be learned while vision encoder and LLM stay frozen and a fixed deterministic controller maps waypoints to controls. Tables 1–2 show large jumps (longest6 SR 0 % → 25 %, Bench2Drive DS 86.8), yet Table 4’s control-vs-waypoint ablation and Table 5’s scaling ablation report only single-run point estimates; no multi-seed standard deviations, no controller-sensitivity study, and no residual-magnitude statistics appear. Because the reward (Eq. 21) is taken from Jaeger et al. and is known to share the same global optimum as the reported metrics, the observed gains could be an artifact of reward–metric alignment plus a single lucky seed rather than robust recovery of closed-loop competence from the residual formulation. The limitations section itself flags Sim2Real and reward design as open issues, confirming that the transfer premise is untested.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper presents CLEAR, a two-stage system for closed-loop RL fine-tuning of Vision-Language-Action (VLA) policies for end-to-end autonomous driving. A SimLingo-style VLA (InternVL3-1B backbone) is first pretrained with imitation learning on expert waypoints. Closed-loop fine-tuning then freezes the vision encoder and LLM and learns a residual waypoint policy (m=4 longitudinal anchors, clipped lateral offsets and speed) around the frozen prior; residuals are optimized with PPO (DD-PPO) under a simple route-completion reward taken from prior work, and a fixed controller maps the corrected waypoints to low-level controls. A heterogeneous pipeline places CARLA servers on V100 hosts and the learner on H100s, linked by SSH tunnels, enabling 64 parallel environments and 100 M samples. On CARLA longest6 v2 the method raises driving score from 18.43 (IL) to 39.89 and success rate from 0 % to 25 %; on Bench2Drive it reaches DS 86.8 / SR 69.5, claimed as new SOTA among non-privileged planners. Zero-shot nuScenes L2 is competitive. Ablations compare residual waypoints versus direct control and show gains with scale.","tokens_in":15646,"tokens_out":1200,"duration_ms":11106,"significance":"If the residual-plus-scale recipe is robust, the work supplies a practical, reproducible path for closed-loop post-training of large VLAs that previously relied almost exclusively on open-loop imitation. The heterogeneous pipeline is a concrete systems contribution that removes a well-known resource-contention bottleneck for vision-based RL in CARLA. Large absolute gains on two standard long-horizon and multi-ability benchmarks, obtained with a deliberately simple reward, would be of immediate interest to the E2E-AD and VLA communities. Strengths that should be credited include the explicit residual formulation that re-uses the pretrained waypoint prior, the transparent scaling numbers (64 envs, 100 M samples, batch 16 384 / 4 096), and the public benchmarks used for evaluation.","major_comments":[{"comment":"Tables 1, 2, 4 and 5 report only single-run point estimates. No multi-seed means, standard deviations or confidence intervals are given for the longest6 or Bench2Drive numbers, nor for the control-versus-waypoint and scaling ablations. Because PPO training of residual policies is known to be seed-sensitive and the reward (Eq. 21) shares the same global optimum as the evaluation metrics, the claimed SOTA jumps (longest6 SR 0 % → 25 %, Bench2Drive DS 86.8) cannot be assessed for statistical reliability. At least three independent seeds with error bars on the primary metrics are required to support the central claim.","section":null},{"comment":"Section 3.3 freezes the vision encoder and LLM and learns only m=4 clipped residual deltas while a fixed deterministic controller (Eq. 19) maps the corrected waypoints to controls. Table 4 shows that replacing the residual waypoint head by a direct-control head drops DS from 39.89 to 33.91, yet no residual-magnitude statistics, no controller-sensitivity study, and no ablation of the clipping bounds appear. Without these diagnostics it remains unclear whether the residual formulation is genuinely recovering closed-loop competence or simply exploiting a well-aligned reward under a single lucky seed and a fixed low-level mapping.","section":null},{"comment":"The reward in Eq. 21 is taken verbatim from Jaeger et al. (2025) and is known to be metric-aligned. While the paper correctly notes that the policy is still learned from interaction, the absence of any alternative reward (or even a simple ablation that removes the soft-penalty product) leaves open the possibility that the observed gains are largely an artifact of reward–metric coincidence rather than a general residual-RL recipe. A short experiment with a deliberately misaligned or sparse reward would strengthen the claim that the residual formulation itself is the key ingredient.","section":null}],"minor_comments":[{"comment":"Figure 2 caption and surrounding text contain typos (“smooothly”, “langage”, “w.r .t.”). A careful proof-read is needed.","section":null},{"comment":"Notation for the residual action a_t (Eq. 15) and the final waypoint command u_wp_t (Eq. 18) is introduced without an explicit dimension statement; a short sentence clarifying that a_t ∈ R^{m+1} would help.","section":null},{"comment":"The heterogeneous pipeline (Section 3.4) is described at a high level; a short paragraph or appendix note on latency, failure recovery, and autossh configuration would aid reproducibility.","section":null},{"comment":"Table 3 zero-shot nuScenes results are interesting but the collision-rate numbers are not the best reported; a brief discussion of domain gap would be useful.","section":null},{"comment":"References to “InternVL2-1B (SimLingo)” versus “InternVL3-1B” in Table 1 should be made consistent with the method description in Section 3.2.","section":null}],"recommendation":"major_revision","confidential_remarks":"The systems contribution (heterogeneous CARLA/H100 pipeline) is genuine and under-emphasized relative to the residual-policy claim. If the authors can supply multi-seed statistics and a modest residual/controller diagnostic, the paper becomes a solid systems+method contribution for the journal; without them the SOTA numbers remain hard to trust. Scope is appropriate for a robotics/ML systems venue."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The one thing worth knowing is that residual waypoint PPO around a frozen InternVL3-1B prior, scaled to 64 parallel CARLA servers and 100 M samples, lifts longest6 v2 success from 0 % to 25 % and sets the highest Bench2Drive numbers among the listed non-privileged methods. That is a real empirical result inside the CARLA ecosystem.\n\nWhat is new is the concrete combination: keep the pretrained path/speed waypoints as a base plan, learn only clipped lateral and speed residuals at m=4 anchors, freeze vision encoder and LLM, and move the simulators onto separate V100 hosts so the H100 learner is not starved. The residual-vs-direct-control ablation (Table 4) and the scaling ladder (Table 5) both point the same way—the residual formulation and the large batch matter. The engineering recipe (SSH tunnels, autossh, DD-PPO) is reproducible enough that other groups can try it.\n\nSoft spots are real but proportionate. All numbers are single-run point estimates; no multi-seed variance, no residual-magnitude histograms, no controller-sensitivity study. The reward is taken from Jaeger et al. and is known to share the same global optimum as the reported metrics, so part of the jump is expected once you optimize that objective at scale. The paper itself flags Sim2Real and reward design as open. The zero-shot nuScenes L2 numbers are open-loop only and do not speak to closed-loop transfer. None of these sink the central claim inside simulation; they just keep confidence moderate.\n\nThis is for people already training VLAs or RL drivers in CARLA who need a practical post-training recipe. The math is standard PPO, the citations cover the right prior work, and the data tables are clean. I would bring it to reading group, cite the residual + heterogeneous-pipeline pattern if I am doing similar work, and send it to peer review. A serious referee can demand multi-seed stats and residual diagnostics without killing the contribution.","headline":"Solid systems paper: residual RL on a frozen VLA prior plus a 64-env heterogeneous CARLA pipeline produces large closed-loop gains; single-run numbers and metric-aligned reward keep the claim under-supported but still worth refereeing.","tokens_in":16269,"tokens_out":514,"would_cite":true,"duration_ms":5369,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Closed-loop RL at scale turns imitation-learned VLAs into far stronger autonomous drivers.","keywords":["end-to-end autonomous driving","Vision-Language-Action","closed-loop reinforcement learning","residual waypoint policy","PPO","CARLA","heterogeneous training pipeline"],"falsifier":"Train the identical residual architecture and scaling setup on the same routes but replace the residual head with a direct low-level control head; if driving score and success rate do not collapse, the residual-prior claim is false.","tokens_in":16165,"feed_emoji":"🚗","tokens_out":817,"duration_ms":7318,"temperature":0.7,"pith_summary":"Imitation learning on expert trajectories leaves Vision-Language-Action driving policies brittle once they run closed-loop: small prediction errors compound into crashes. CLEAR closes that gap by freezing a pretrained VLA, treating its waypoint plan as a prior, and training only a residual policy that issues bounded lateral and speed corrections. A heterogeneous pipeline puts dozens of CARLA simulators on one set of GPUs and the learner on another, scaling to 64 parallel environments and 100 million samples under PPO. With nothing more than a simple route-completion reward, the residual policy lifts success rates from 0 % to 25 % on the long, hard longest6 v2 routes and sets new state-of-the-art numbers on Bench2Drive. The result shows that large-scale closed-loop RL can extract substantial driving competence from an open-loop prior without retraining the entire multimodal backbone.","feed_headline":"RL at scale lifts VLA drivers from 0% to 25% success","feed_subtitle":"Residual waypoints plus 64 parallel simulators turn open-loop priors into closed-loop SOTA","key_machinery":"Residual waypoint policy: the frozen VLA supplies base path and speed waypoints at fixed longitudinal anchors; a small MLP samples clipped residual offsets that are added to those anchors and then mapped by a deterministic controller to low-level controls, so PPO optimizes only the residual distribution.","core_discovery":"A residual waypoint policy learned by large-scale PPO around a frozen pretrained VLA prior, enabled by a heterogeneous simulator-learner pipeline that supports 64 parallel CARLA environments and 100 M samples, produces state-of-the-art closed-loop driving scores on CARLA longest6 v2 and Bench2Drive using only a simple route-completion reward.","pith_inferences":["The same residual-plus-heterogeneous pattern could be applied to other large multimodal policies whose open-loop imitation performance saturates.","If residual corrections remain small on most frames, the frozen backbone may already encode most of the necessary scene geometry, suggesting lighter online adapters may suffice.","Further scaling will be gated by CARLA rendering stability rather than learner compute, pointing to simulator engineering as the next bottleneck."],"forward_implications":["Open-loop VLA pretraining plus residual closed-loop RL becomes a practical two-stage recipe for end-to-end driving.","Heterogeneous placement of simulator and learner removes the main GPU-memory bottleneck that previously limited parallel CARLA rollouts.","A simple route-completion reward is already sufficient to produce large closed-loop gains once sample volume reaches tens of millions.","Zero-shot transfer of the residual policy to real-world logs (nuScenes) becomes measurable without any real-world fine-tuning."],"fun_headline_variants":["Residual PPO around VLA prior reaches closed-loop SOTA on CARLA","CLEAR scales residual waypoint RL to 64 parallel sims and 100M samples","Heterogeneous pipeline turns open-loop VLA into closed-loop driving SOTA","Simple route reward residual policy beats prior VLA methods on Bench2Drive","Closed-loop residual waypoints lift E2E-AD success beyond open-loop priors"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The open-loop VLA waypoint plan stays good enough that small, clipped residual corrections alone can recover closed-loop competence without unfreezing the vision encoder or language model.","fun_headline_variants_meta":{"raw":{"variants":["Residual PPO around VLA prior reaches closed-loop SOTA on CARLA","CLEAR scales residual waypoint RL to 64 parallel sims and 100M samples","Heterogeneous pipeline turns open-loop VLA into closed-loop driving SOTA","Simple route reward residual policy beats prior VLA methods on Bench2Drive","Closed-loop residual waypoints lift E2E-AD success beyond open-loop priors"]},"model":"grok-4.5","effort":"low","cost_usd":0.004192,"raw_usage":{"total_tokens":1262,"prompt_tokens":845,"num_sources_used":0,"completion_tokens":86,"cost_in_usd_ticks":41920000,"prompt_tokens_details":{"text_tokens":845,"audio_tokens":0,"image_tokens":0,"cached_tokens":0},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":331,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":845,"tokens_out":86,"duration_ms":3146,"temperature":1.0,"reasoning_tokens":331,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-12T06:40:11.267535+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train the identical residual architecture and scaling setup on the same routes but replace the residual head with a direct low-level control head; if driving score and success rate do not collapse, the residual-prior claim is false.","supporting_citations":[],"review_version":1}