Pith. sign in

REVIEW 3 major objections 5 minor 72 references

Closed-loop RL at scale turns imitation-learned VLAs into far stronger autonomous drivers.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 06:40 UTC pith:EKY465FV

load-bearing objection Solid systems paper: residual RL on a frozen VLA prior plus a 64-env heterogeneous CARLA pipeline produces large closed-loop gains; single-run numbers and metric-aligned reward keep the claim under-supported but still worth refereeing. the 3 major comments →

arxiv 2607.02841 v1 pith:EKY465FV submitted 2026-07-03 cs.RO cs.CV

CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Driving

Yunxiao Shi , Hong Cai , Mohammad Ghavamzadeh , Fatih Porikli This is my paper
classification cs.RO cs.CV
keywords end-to-end autonomous drivingVision-Language-Actionclosed-loop reinforcement learningresidual waypoint policyPPOCARLAheterogeneous training pipeline
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Imitation learning on expert trajectories leaves Vision-Language-Action driving policies brittle once they run closed-loop: small prediction errors compound into crashes. CLEAR closes that gap by freezing a pretrained VLA, treating its waypoint plan as a prior, and training only a residual policy that issues bounded lateral and speed corrections. A heterogeneous pipeline puts dozens of CARLA simulators on one set of GPUs and the learner on another, scaling to 64 parallel environments and 100 million samples under PPO. With nothing more than a simple route-completion reward, the residual policy lifts success rates from 0 % to 25 % on the long, hard longest6 v2 routes and sets new state-of-the-art numbers on Bench2Drive. The result shows that large-scale closed-loop RL can extract substantial driving competence from an open-loop prior without retraining the entire multimodal backbone.

Core claim

A residual waypoint policy learned by large-scale PPO around a frozen pretrained VLA prior, enabled by a heterogeneous simulator-learner pipeline that supports 64 parallel CARLA environments and 100 M samples, produces state-of-the-art closed-loop driving scores on CARLA longest6 v2 and Bench2Drive using only a simple route-completion reward.

What carries the argument

Residual waypoint policy: the frozen VLA supplies base path and speed waypoints at fixed longitudinal anchors; a small MLP samples clipped residual offsets that are added to those anchors and then mapped by a deterministic controller to low-level controls, so PPO optimizes only the residual distribution.

Load-bearing premise

The open-loop VLA waypoint plan stays good enough that small, clipped residual corrections alone can recover closed-loop competence without unfreezing the vision encoder or language model.

What would settle it

Train the identical residual architecture and scaling setup on the same routes but replace the residual head with a direct low-level control head; if driving score and success rate do not collapse, the residual-prior claim is false.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Open-loop VLA pretraining plus residual closed-loop RL becomes a practical two-stage recipe for end-to-end driving.
  • Heterogeneous placement of simulator and learner removes the main GPU-memory bottleneck that previously limited parallel CARLA rollouts.
  • A simple route-completion reward is already sufficient to produce large closed-loop gains once sample volume reaches tens of millions.
  • Zero-shot transfer of the residual policy to real-world logs (nuScenes) becomes measurable without any real-world fine-tuning.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same residual-plus-heterogeneous pattern could be applied to other large multimodal policies whose open-loop imitation performance saturates.
  • If residual corrections remain small on most frames, the frozen backbone may already encode most of the necessary scene geometry, suggesting lighter online adapters may suffice.
  • Further scaling will be gated by CARLA rendering stability rather than learner compute, pointing to simulator engineering as the next bottleneck.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents CLEAR, a two-stage system for closed-loop RL fine-tuning of Vision-Language-Action (VLA) policies for end-to-end autonomous driving. A SimLingo-style VLA (InternVL3-1B backbone) is first pretrained with imitation learning on expert waypoints. Closed-loop fine-tuning then freezes the vision encoder and LLM and learns a residual waypoint policy (m=4 longitudinal anchors, clipped lateral offsets and speed) around the frozen prior; residuals are optimized with PPO (DD-PPO) under a simple route-completion reward taken from prior work, and a fixed controller maps the corrected waypoints to low-level controls. A heterogeneous pipeline places CARLA servers on V100 hosts and the learner on H100s, linked by SSH tunnels, enabling 64 parallel environments and 100 M samples. On CARLA longest6 v2 the method raises driving score from 18.43 (IL) to 39.89 and success rate from 0 % to 25 %; on Bench2Drive it reaches DS 86.8 / SR 69.5, claimed as new SOTA among non-privileged planners. Zero-shot nuScenes L2 is competitive. Ablations compare residual waypoints versus direct control and show gains with scale.

Significance. If the residual-plus-scale recipe is robust, the work supplies a practical, reproducible path for closed-loop post-training of large VLAs that previously relied almost exclusively on open-loop imitation. The heterogeneous pipeline is a concrete systems contribution that removes a well-known resource-contention bottleneck for vision-based RL in CARLA. Large absolute gains on two standard long-horizon and multi-ability benchmarks, obtained with a deliberately simple reward, would be of immediate interest to the E2E-AD and VLA communities. Strengths that should be credited include the explicit residual formulation that re-uses the pretrained waypoint prior, the transparent scaling numbers (64 envs, 100 M samples, batch 16 384 / 4 096), and the public benchmarks used for evaluation.

major comments (3)
  1. Tables 1, 2, 4 and 5 report only single-run point estimates. No multi-seed means, standard deviations or confidence intervals are given for the longest6 or Bench2Drive numbers, nor for the control-versus-waypoint and scaling ablations. Because PPO training of residual policies is known to be seed-sensitive and the reward (Eq. 21) shares the same global optimum as the evaluation metrics, the claimed SOTA jumps (longest6 SR 0 % → 25 %, Bench2Drive DS 86.8) cannot be assessed for statistical reliability. At least three independent seeds with error bars on the primary metrics are required to support the central claim.
  2. Section 3.3 freezes the vision encoder and LLM and learns only m=4 clipped residual deltas while a fixed deterministic controller (Eq. 19) maps the corrected waypoints to controls. Table 4 shows that replacing the residual waypoint head by a direct-control head drops DS from 39.89 to 33.91, yet no residual-magnitude statistics, no controller-sensitivity study, and no ablation of the clipping bounds appear. Without these diagnostics it remains unclear whether the residual formulation is genuinely recovering closed-loop competence or simply exploiting a well-aligned reward under a single lucky seed and a fixed low-level mapping.
  3. The reward in Eq. 21 is taken verbatim from Jaeger et al. (2025) and is known to be metric-aligned. While the paper correctly notes that the policy is still learned from interaction, the absence of any alternative reward (or even a simple ablation that removes the soft-penalty product) leaves open the possibility that the observed gains are largely an artifact of reward–metric coincidence rather than a general residual-RL recipe. A short experiment with a deliberately misaligned or sparse reward would strengthen the claim that the residual formulation itself is the key ingredient.
minor comments (5)
  1. Figure 2 caption and surrounding text contain typos (“smooothly”, “langage”, “w.r .t.”). A careful proof-read is needed.
  2. Notation for the residual action a_t (Eq. 15) and the final waypoint command u_wp_t (Eq. 18) is introduced without an explicit dimension statement; a short sentence clarifying that a_t ∈ R^{m+1} would help.
  3. The heterogeneous pipeline (Section 3.4) is described at a high level; a short paragraph or appendix note on latency, failure recovery, and autossh configuration would aid reproducibility.
  4. Table 3 zero-shot nuScenes results are interesting but the collision-rate numbers are not the best reported; a brief discussion of domain gap would be useful.
  5. References to “InternVL2-1B (SimLingo)” versus “InternVL3-1B” in Table 1 should be made consistent with the method description in Section 3.2.

Circularity Check

0 steps flagged

No derivation circularity: residual RL is optimized by interaction against a metric-aligned reward, not forced by construction or self-citation uniqueness.

full rationale

CLEAR's central claim is an empirical systems result: a residual waypoint policy (Eqs. 13–18) around a frozen open-loop VLA prior, trained with PPO on a simple route-completion reward (Eq. 21) taken from Jaeger et al., plus a heterogeneous 64-env pipeline, yields large closed-loop gains on longest6 v2 and Bench2Drive. The residual is not algebraically identical to the expert or to the metric; it is sampled and optimized from simulator interaction. The reward is known to share the same global optimum as the evaluation metrics (explicitly noted in §5), which is reward–metric alignment rather than a definitional identity that forces the reported DS/SR numbers. Ablations (Tables 4–5) and external baselines further show the gains are not tautological. The only minor self-use is adoption of the Carl reward family; that is ordinary engineering practice and not a load-bearing uniqueness theorem or fitted-input-as-prediction. Score 1 reflects that single non-circular self-use of a prior reward; the derivation chain itself is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

The central claim rests on standard RL and VLA machinery plus a handful of engineering and modeling choices (residual space, freeze decision, reward, controller, hardware split). No new physical entities are postulated; free parameters are the usual RL hyper-parameters and the residual clipping bounds.

free parameters (4)
  • number of longitudinal anchors m = 4
    Set to 4 by hand; controls residual action dimensionality and prediction horizon.
  • residual clipping bounds (y_min/max, v_min/max)
    Hand-chosen limits that keep the residual policy inside a safe neighborhood of the VLA prior.
  • PPO total-batch / mini-batch and sample count = 16384/4096, 100M
    16384 / 4096 and 100 M samples chosen for scaling experiments; performance is shown to depend strongly on these values (Table 5).
  • reward soft-penalty factors p_t and hard-penalty P
    Taken from prior Carl reward; exact numerical schedule is not re-derived.
axioms (4)
  • ad hoc to paper A frozen open-loop VLA waypoint prior plus bounded residuals is expressive enough for closed-loop recovery on long routes.
    Core modeling choice of §3.3; justified only by final performance, not by a completeness argument.
  • domain assumption The deterministic low-level controller that maps residual waypoints to (steer, throttle/brake) can be treated as part of the environment (no gradients).
    Standard in waypoint-based driving; stated after Eq. (19)–(20).
  • domain assumption CARLA simulator dynamics and the longest6/Bench2Drive scenario distributions are adequate proxies for the closed-loop driving problem of interest.
    Implicit throughout experiments; limitations section notes sim-to-real remains open.
  • domain assumption PPO with the stated large-batch DD-PPO configuration converges stably under the heterogeneous SSH-tunnel setup.
    Assumed by the scaling claims; supported only by the reported final metrics.
invented entities (2)
  • residual waypoint policy around a frozen VLA prior no independent evidence
    purpose: Allows RL finetuning without destroying pretrained knowledge and without outputting low-level controls directly.
    Defined in §3.3; the residual action space itself is the main algorithmic invention, but it is an engineering construct rather than a new physical entity.
  • heterogeneous CARLA-server / H100-learner pipeline with SSH tunnels no independent evidence
    purpose: Removes GPU memory contention and driver instability so that 64 parallel environments become feasible.
    Systems contribution of §3.4; no independent theoretical claim.

pith-pipeline@v1.1.0-grok45 · 19349 in / 3260 out tokens · 24723 ms · 2026-07-12T06:40:11.267535+00:00 · methodology

0 comments
read the original abstract

End-to-end autonomous driving (E2E-AD) aims to directly map raw sensor information to driving actions. Recently, with the rapid advancement of multi-modal large language models (MLLMs), researchers have proposed the paradigm of Vision-Language-Action (VLA) models for E2E-AD, where it seeks to integrate visual perception, language understanding and action prediction within a single policy. However, existing VLA-based policies largely adopts imitation learning, where it only learns to drive by optimizing distance-based metrics w.r.t. logged expert trajectories. Such distribution shift between open-loop training and closed-loop inference leads to suboptimal performance in closed-loop planning. To close this gap, we present CLEAR, a system that enables closed-loop training using Reinforcement Learning (RL) at scale for E2E-AD. We propose to learn a novel residual waypoint policy around the waypoint prior from pretrained VLA policies, effectively harnessing the knowledge within. On another front, one of the key challenges to scale up RL for vision-based policies is the number of parallel simulation environments since RL is data hungry. To that end, we design a heterogeneous pipeline that places the simulator and the VLA learner on distinct compute groups, which allows us to dramatically increase the number of simulation environments running in parallel while avoiding resource contention and maintaining training stability. We show that with a simple reward, CLEAR significantly outperforms previous methods and sets new state-of-the-art performance on the challenging benchmarks of CARLA longest6 v2 and Bench2Drive.

Figures

Figures reproduced from arXiv: 2607.02841 by Fatih Porikli, Hong Cai, Mohammad Ghavamzadeh, Yunxiao Shi.

Figure 1
Figure 1. Figure 1: CLEAR system pipeline. We conduct closed-loop RL finetuning on top of VLAs pretrained using imitation learning, and learn a residual policy around the pretrained waypoints prior. With our heterogeneous finetuning pipeline, we scale up the number of simulators [52] running in parallel and PPO [34, 53] updates dramatically. Observable Markov Decision Process (POMDP), M = (O, A,P, r, γ), (1) where ot = {It, v… view at source ↗
Figure 2
Figure 2. Figure 2: Qualitative predictions of CLEAR on the CARLA longest6 v2 [33, 59] benchmark. We visualize the pretrained waypoints horizon (first row) vs. our residual corrected ones (second row). One can see that pre￾trained policy gave the wrong prediction and led to a crash, whereas our policy followed the traffic smooothly. Best viewed in color and zoomed in [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

72 extracted references · 21 linked inside Pith

  1. [1]

    L. Chen, P. Wu, K. Chitta, B. Jaeger, A. Geiger, and H. Li. End-to-end autonomous driving: Challenges and frontiers.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12):10164–10183, 2024

    2024

  2. [2]

    Y . Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang, et al. Planning-oriented autonomous driving. InProceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pages 17853–17862, 2023

    2023

  3. [3]

    Jiang, S

    B. Jiang, S. Chen, Q. Xu, B. Liao, J. Chen, H. Zhou, Q. Zhang, W. Liu, C. Huang, and X. Wang. Vad: Vectorized scene representation for efficient autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 8340–8350, 2023

    2023

  4. [4]

    S. Chen, B. Jiang, H. Gao, B. Liao, Q. Xu, Q. Zhang, C. Huang, W. Liu, and X. Wang. Vadv2: End-to-end vectorized autonomous driving via probabilistic planning.arXiv preprint arXiv:2402.13243, 2024

    Pith/arXiv arXiv 2024

  5. [5]

    Li and D

    P. Li and D. Cui. Navigation-guided sparse scene representation for end-to-end autonomous driving. InInternational Conference on Learning Representations (ICLR), 2025

    2025

  6. [6]

    Hawke, R

    J. Hawke, R. Shen, C. Gurau, S. Sharma, D. Reda, N. Nikolov, P. Mazur, S. Micklethwaite, N. Griffiths, A. Shah, et al. Urban driving with conditional imitation learning. In2020 IEEE International Conference on Robotics and Automation (ICRA), pages 251–257. IEEE, 2020

    2020

  7. [7]

    Chitta, A

    K. Chitta, A. Prakash, and A. Geiger. Neat: Neural attention fields for end-to-end autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 15793–15803, 2021

    2021

  8. [8]

    Chitta, A

    K. Chitta, A. Prakash, B. Jaeger, Z. Yu, K. Renz, and A. Geiger. Transfuser: Imitation with transformer-based sensor fusion for autonomous driving.IEEE transactions on pattern analy- sis and machine intelligence, 45(11):12878–12895, 2022

    2022

  9. [9]

    X. Jia, P. Wu, L. Chen, J. Xie, C. He, J. Yan, and H. Li. Think twice before driving: To- wards scalable decoders for end-to-end autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21983–21994, 2023

    2023

  10. [10]

    H. Shao, L. Wang, R. Chen, H. Li, and Y . Liu. Safety-enhanced autonomous driving using interpretable sensor fusion transformer. InConference on Robot Learning, pages 726–737. PMLR, 2023

    2023

  11. [11]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  12. [12]

    G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Pith/arXiv arXiv 2023

  13. [13]

    J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huang, B. Hui, L. Ji, M. Li, J. Lin, R. Lin, D. Liu, G. Liu, C. Lu, K. Lu, J. Ma, R. Men, X. Ren, X. Ren, C. Tan, S. Tan, J. Tu, P. Wang, S. Wang, W. Wang, S. Wu, B. Xu, J. Xu, A. Yang, H. Yang, J. Yang, S. Yang, Y . Yao, B. Yu, H. Yuan, Z. Yuan, J. Zhang, X. Zhang, Y . Zhang, ...

    Pith/arXiv arXiv 2023

  14. [14]

    Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, M. Zhong, Q. Zhang, X. Zhu, L. Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 24185–24198, 2024. 9

    2024

  15. [15]

    Comanici, E

    G. Comanici, E. Bieber, M. Schaekermann, I. Pasupat, N. Sachdeva, I. Dhillon, M. Blistein, O. Ram, D. Zhang, E. Rosen, et al. Gemini 2.5: Pushing the frontier with advanced rea- soning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Pith/arXiv arXiv 2025

  16. [16]

    Singh, A

    A. Singh, A. Fry, A. Perelman, A. Tart, A. Ganesh, A. El-Kishky, A. McLaughlin, A. Low, A. Ostrow, A. Ananthram, et al. Openai gpt-5 system card.arXiv preprint arXiv:2601.03267, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Jiang, Z

    S. Jiang, Z. Huang, K. Qian, Z. Luo, T. Zhu, Y . Zhong, Y . Tang, M. Kong, Y . Wang, S. Jiao, et al. A survey on vision-language-action models for autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4524–4536, 2025

    2025

  18. [18]

    Caesar, V

    H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom. nuscenes: A multimodal dataset for autonomous driving. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020

    2020

  19. [19]

    Caesar, J

    H. Caesar, J. Kabzan, K. S. Tan, W. K. Fong, E. Wolff, A. Lang, L. Fletcher, O. Beijbom, and S. Omari. nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles. arXiv preprint arXiv:2106.11810, 2021

    Pith/arXiv arXiv 2021

  20. [20]

    P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V . Patnaik, P. Tsui, J. Guo, Y . Zhou, Y . Chai, B. Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2446–2454, 2020

    2020

  21. [21]

    Dauner, M

    D. Dauner, M. Hallgarten, T. Li, X. Weng, Z. Huang, Z. Yang, H. Li, I. Gilitschenski, B. Ivanovic, M. Pavone, et al. Navsim: Data-driven non-reactive autonomous vehicle sim- ulation and benchmarking.Advances in Neural Information Processing Systems, 37:28706– 28719, 2024

    2024

  22. [22]

    X. Jia, Z. Yang, Q. Li, Z. Zhang, and J. Yan. Bench2drive: Towards multi-ability benchmarking of closed-loop end-to-end autonomous driving.Advances in Neural Information Processing Systems, 37:819–844, 2024

    2024

  23. [23]

    W. Cao, M. Hallgarten, T. Li, D. Dauner, X. Gu, C. Wang, Y . Miron, M. Aiello, H. Li, I. Gilitschenski, et al. Pseudo-simulation for autonomous driving.Conference on Robot Learn- ing, 2025

    2025

  24. [24]

    Hwang, R

    J.-J. Hwang, R. Xu, H. Lin, W.-C. Hung, J. Ji, K. Choi, D. Huang, T. He, P. Covington, B. Sapp, et al. Emma: End-to-end multimodal model for autonomous driving.arXiv preprint arXiv:2410.23262, 2024

    Pith/arXiv arXiv 2024

  25. [25]

    S. Xing, C. Qian, Y . Wang, H. Hua, K. Tian, Y . Zhou, and Z. Tu. Openemma: Open-source multimodal model for end-to-end autonomous driving. InProceedings of the Winter Confer- ence on Applications of Computer Vision, pages 1001–1009, 2025

    2025

  26. [26]

    H. Fu, D. Zhang, Z. Zhao, J. Cui, D. Liang, C. Zhang, D. Zhang, H. Xie, B. Wang, and X. Bai. Orion: A holistic end-to-end autonomous driving framework by vision-language instructed action generation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 24823–24834, 2025

    2025

  27. [27]

    K. Renz, L. Chen, E. Arani, and O. Sinavski. Simlingo: Vision-only closed-loop autonomous driving with language-action alignment. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 11993–12003, 2025. 10

    2025

  28. [28]

    Y . Li, K. Xiong, X. Guo, F. Li, S. Yan, G. Xu, L. Zhou, L. Chen, H. Sun, B. Wang, et al. Recog- drive: A reinforced cognitive framework for end-to-end autonomous driving. InInternational Conference on Learning Representations (ICLR), 2026

    2026

  29. [29]

    R. S. Sutton, A. G. Barto, et al.Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998

    1998

  30. [30]

    K. Renz, K. Chitta, O.-B. Mercea, A. Koepke, Z. Akata, and A. Geiger. Plant: Explainable planning transformers via object-level representations.Conference on Robotic Learning, 2022

    2022

  31. [31]

    P. Wu, X. Jia, L. Chen, J. Yan, H. Li, and Y . Qiao. Trajectory-guided control prediction for end-to-end autonomous driving: A simple yet strong baseline.Advances in Neural Information Processing Systems, 35:6119–6132, 2022

    2022

  32. [32]

    Q. Li, X. Jia, S. Wang, and J. Yan. Think2drive: Efficient reinforcement learning by thinking with latent world model for autonomous driving (in carla-v2). InEuropean conference on computer vision, pages 142–158. Springer, 2024

    2024

  33. [33]

    Jaeger, D

    B. Jaeger, D. Dauner, J. Beißwenger, S. Gerstenecker, K. Chitta, and A. Geiger. Carl: Learning scalable planning policies with simple rewards.arXiv preprint arXiv:2504.17838, 2025

    Pith/arXiv arXiv 2025

  34. [34]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  35. [35]

    Huang, C

    Z. Huang, C. Feng, F. Yan, B. Xiao, Z. Jie, Y . Zhong, X. Liang, and L. Ma. Drivemm: All-in- one large multimodal model for autonomous driving.arXiv preprint arXiv:2412.07689, 2(3): 8, 2024

    Pith/arXiv arXiv 2024

  36. [36]

    D. Fu, X. Li, L. Wen, M. Dou, P. Cai, B. Shi, and Y . Qiao. Drive like a human: Rethinking autonomous driving with large language models. In2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW), pages 910–919. IEEE, 2024

    2024

  37. [37]

    J. Liu, P. Hang, X. Qi, J. Wang, and J. Sun. Mtd-gpt: A multi-task decision-making gpt model for autonomous driving at unsignalized intersections. In2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC), pages 5154–5161. IEEE, 2023

    2023

  38. [38]

    Y . Ma, Y . Cao, J. Sun, M. Pavone, and C. Xiao. Dolphins: Multimodal language model for driving. InEuropean Conference on Computer Vision, pages 403–420. Springer, 2024

    2024

  39. [39]

    P. Paul, A. Garg, T. Choudhary, A. K. Singh, and K. M. Krishna. Lego-drive: Language- enhanced goal-oriented closed-loop end-to-end autonomous driving. In2024 IEEE/RSJ In- ternational Conference on Intelligent Robots and Systems (IROS), pages 10020–10026. IEEE, 2024

    2024

  40. [40]

    T.-H. Wang, A. Maalouf, W. Xiao, Y . Ban, A. Amini, G. Rosman, S. Karaman, and D. Rus. Drive anywhere: Generalizable end-to-end autonomous driving with multi-modal foundation models. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 6687–6694. IEEE, 2024

    2024

  41. [41]

    W. Wang, J. Xie, C. Hu, H. Zou, J. Fan, W. Tong, Y . Wen, S. Wu, H. Deng, Z. Li, et al. Drivemlm: Aligning multi-modal large language models with behavioral planning states for autonomous driving.arXiv preprint arXiv:2312.09245, 2023

    arXiv 2023

  42. [42]

    J. Mao, Y . Qian, J. Ye, H. Zhao, and Y . Wang. Gpt-driver: Learning to drive with gpt.arXiv preprint arXiv:2310.01415, 2023

    Pith/arXiv arXiv 2023

  43. [43]

    X. Tian, J. Gu, B. Li, Y . Liu, Y . Wang, Z. Zhao, K. Zhan, P. Jia, X. Lang, and H. Zhao. Drivevlm: The convergence of autonomous driving and large vision-language models.arXiv preprint arXiv:2402.12289, 2024. 11

    Pith/arXiv arXiv 2024

  44. [44]

    Jiang, S

    B. Jiang, S. Chen, B. Liao, X. Zhang, W. Yin, Q. Zhang, C. Huang, W. Liu, and X. Wang. Senna: Bridging large vision-language models and end-to-end autonomous driving.arXiv preprint arXiv:2410.22313, 2024

    Pith/arXiv arXiv 2024

  45. [45]

    Ouyang, J

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744, 2022

    2022

  46. [46]

    H. Gao, S. Chen, B. Jiang, B. Liao, Y . Shi, X. Guo, Y . Pu, X. Li, W. Liu, Q. Zhang, et al. Rad: Training an end-to-end driving policy via large-scale 3dgs-based reinforcement learning. Advances in Neural Information Processing Systems, 38:32551–32576, 2026

    2026

  47. [47]

    D. Li, J. Ren, Y . Wang, X. Wen, P. Li, L. Xu, K. Zhan, Z. Xia, P. Jia, X. Lang, et al. Finetuning generative trajectory model with reinforcement learning from human feedback.arXiv e-prints, pages arXiv–2503, 2025

    2025

  48. [48]

    Huang, X

    Z. Huang, X. Weng, M. Igl, Y . Chen, Y . Cao, B. Ivanovic, M. Pavone, and C. Lv. Gen-drive: Enhancing diffusion generative driving policies with reward modeling and reinforcement learn- ing fine-tuning. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 3445–3451. IEEE, 2025

    2025

  49. [49]

    Jiang, S

    B. Jiang, S. Chen, Q. Zhang, W. Liu, and X. Wang. Alphadrive: Unleashing the power of vlms in autonomous driving via reinforcement learning and reasoning.arXiv preprint arXiv:2503.07608, 2025

    Pith/arXiv arXiv 2025

  50. [50]

    Z. Zhou, T. Cai, S. Zhao, Y . Zhang, Z. Huang, B. Zhou, and J. Ma. Autovla: A vision-language- action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning.Advances in Neural Information Processing Systems, 38:27920–27956, 2026

    2026

  51. [51]

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  52. [52]

    Dosovitskiy, G

    A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun. Carla: An open urban driving simulator. InConference on robot learning, pages 1–16. PMLR, 2017

    2017

  53. [53]

    Wijmans, A

    E. Wijmans, A. Kadian, A. Morcos, S. Lee, I. Essa, D. Parikh, M. Savva, and D. Batra. Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames.International Conference on Learning Representations (ICLR), 2020

    2020

  54. [54]

    J. Zhu, W. Wang, Z. Chen, Z. Liu, S. Ye, L. Gu, H. Tian, Y . Duan, W. Su, J. Shao, et al. In- ternvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479, 2025

    Pith/arXiv arXiv 2025

  55. [55]

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, D. Kai, P. Wang, W. Shijie, T. Jun, Z. Humen, Y . Zhu, M. Yang, Z. Li, J. Wan, P. Wang, W. Ding, Z. Fu, Y . Xu, J. Ye, Z. Xi, X. Tianbao, Z. Cheng, H. Zhang, Z. Yang, H. Xu, and J. Lin. Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    Pith/arXiv arXiv 2025

  56. [56]

    T. C. community. The carla issues section.https://github.com/carla-simulator/carla/issues, 2022

    2022

  57. [57]

    U. Engine. Unreal engine.Retrieved from Unreal Engine: https://www. unrealengine. com/en- US/what-is-unreal-engine-4, 2018

    2018

  58. [58]

    Ylonen and C

    T. Ylonen and C. Lonvick. The secure shell (ssh) connection protocol. RFC 4254 (Standards Track), Jan. 2006. URLhttps://www.rfc-editor.org/rfc/rfc4254

    2006

  59. [59]

    T. C. team. The carla leaderboard 2.0.https://leaderboard.carla.org, 2022. 12

    2022

  60. [60]

    Loshchilov and F

    I. Loshchilov and F. Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Pith/arXiv arXiv 2017

  61. [61]

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen, et al. Lora: Low-rank adaptation of large language models.Iclr, 1(2):3, 2022

    2022

  62. [62]

    X. Jia, Y . Gao, L. Chen, J. Yan, P. L. Liu, and H. Li. Driveadapter: Breaking the coupling barrier of perception and planning in end-to-end autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 7953–7963, 2023

    2023

  63. [63]

    Z. Song, C. Jia, L. Liu, H. Pan, Y . Zhang, J. Wang, X. Zhang, S. Xu, L. Yang, and Y . Luo. Don’t shake the wheel: Momentum-aware planning in end-to-end autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22432–22441, 2025

    2025

  64. [64]

    X. Jia, J. You, Z. Zhang, and J. Yan. Drivetransformer: Unified transformer for scalable end- to-end autonomous driving.arXiv preprint arXiv:2503.07656, 2025

    Pith/arXiv arXiv 2025

  65. [65]

    Y . Tang, Z. Xu, Z. Meng, and E. Cheng. Hip-ad: Hierarchical and multi-granularity planning with deformable attention for autonomous driving in a single decoder. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 25605–25615, 2025

    2025

  66. [66]

    Yasarla, D

    R. Yasarla, D. Hegde, S. Han, H.-P. Cheng, Y . Shi, M. Sadeghigooghari, S. Mahajan, A. Bhat- tacharyya, L. Liu, R. Garrepalli, et al. Generative scenario rollouts for end-to-end autonomous driving.arXiv preprint arXiv:2601.11475, 2026

    arXiv 2026

  67. [67]

    Z. Yang, X. Jia, Q. Li, X. Yang, M. Yao, and J. Yan. Raw2drive: Reinforcement learning with aligned world models for end-to-end autonomous driving (in carla v2).Advances in Neural Information Processing Systems, 38:134122–134147, 2026

    2026

  68. [68]

    S. Hu, L. Chen, P. Wu, H. Li, J. Yan, and D. Tao. St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning. InEuropean Conference on Computer Vision, pages 533–549. Springer, 2022

    2022

  69. [69]

    W. Tong, C. Sima, T. Wang, L. Chen, S. Wu, H. Deng, Y . Gu, L. Lu, P. Luo, D. Lin, et al. Scene as occupancy. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 8406–8415, 2023

    2023

  70. [70]

    Zheng, R

    W. Zheng, R. Song, X. Guo, C. Zhang, and L. Chen. Genad: Generative end-to-end au- tonomous driving. InEuropean Conference on Computer Vision, pages 87–104. Springer, 2024

    2024

  71. [71]

    T. Lin, K. Sachdev, L. Fan, J. Malik, and Y . Zhu. Sim-to-real reinforcement learning for vision-based dexterous manipulation on humanoids.arXiv preprint arXiv:2502.20396, 2025

    Pith/arXiv arXiv 2025

  72. [72]

    W. Zhao, J. P. Queralta, and T. Westerlund. Sim-to-real transfer in deep reinforcement learning for robotics: a survey. In2020 IEEE symposium series on computational intelligence (SSCI), pages 737–744. IEEE, 2020. 13

    2020