Pith. sign in

REVIEW 2 major objections 7 minor 3 cited by

DSDrive: Distilling Large Language Model for Lightweight End-to-End Autonomous Driving with Unified Reasoning and Planning

T0 review · 2 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 1-billion-parameter driving model out-scores a 7-billion-parameter rival in closed-loop tests after reasoning distillation.

desk verdict A genuinely novel reasoning-to-planning distillation for 1B AD models, but the closed-loop evidence is single-run and the text overstates the tables. read the letter →

arxiv 2505.05360 v1 pith:AUF2KNS6 submitted 2025-05-08 cs.RO

Wenru Liu , Pei Liu , Jun Ma This is my paper · ORCID
classification cs.RO
keywords end-to-endautonomousdrivingchain-of-thoughtdistillationvisionlanguagemodelcompactLLMwaypoint-drivendual-headcoordinationclosed-loopsimulationCARLA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a compact 1-billion-parameter language model can be taught to reason about driving scenes by a much larger vision-language model, and that this distilled reasoning improves closed-loop driving enough to match or beat a 7-billion-parameter baseline. The method, DSDrive, unifies chain-of-thought reasoning with waypoint prediction by making the predicted waypoints the final answer of the reasoning process. A dual-head design shares hidden states so one head produces a readable explanation while the other outputs the trajectory. The payoff is interpretable, efficient end-to-end driving: reasoning you can inspect alongside planning you can execute, at a fraction of the memory of larger models.

What carries the argument

The load-bearing mechanism is the waypoint-driven dual-head coordination module. It restructures the distillation dataset so each think-and-answer chain ends with the ground-truth future waypoints, making trajectory prediction the terminal step of the reasoning process. The driving model then has two heads over the same LLM hidden states: a chain-of-thought answer head built with Qwen2-style decoder layers and an MLP waypoint head, trained jointly with a weighted loss on waypoints, answer tokens, and end-state classification. This makes reasoning and planning share one objective and allows a compact 1-billion-parameter LLaMA model to absorb reasoning capabilities from Qwen2.5-VL-max.

What would settle it

Run DSDrive and LMDrive (LLaVA-7B) on the same LangAuto-Short routes for several independent CARLA episodes and compare the distribution of Driving Scores; if DSDrive's mean does not clearly exceed the 7B baseline's mean beyond the run-to-run spread, the claimed outperformance is not established.

Watch

Extended reading notes

Core claim

The central claim is that knowledge distillation from a large vision-language model into a compact language model, combined with a waypoint-driven dual-head architecture, lets a 1-billion-parameter model perform closed-loop end-to-end driving on par with or better than a 7-billion-parameter system. In CARLA LangAuto evaluations, DSDrive with LLaMA-1B achieves a Driving Score of 29.57 versus LMDrive's 28.51 on long routes and 62.05 versus 45.03 on short routes, while using roughly 8 GB of peak memory compared with about 14 GB for the larger baseline. The reasoning head and the planning head share the same hidden states and are both supervised by ground-truth waypoints, which the paper argues aligns semantic reasoning with numerical trajectory planning. Ablation results comparing CoT-only, ground-truth-only, and the full dual-task design support the conclusion that both the distilled reasoning and the waypoint anchoring contribute to the gain.

Load-bearing premise

The central claim depends on the single-run closed-loop scores in Table II being reproducible, because the paper does not report repeated runs, random seeds, or variance in the stochastic CARLA simulator.

Editorial extensions

If this is right

  • Compact LLMs can inherit reasoning skills from larger VLMs through structured chain-of-thought distillation, reducing the deployment cost of interpretable autonomous driving.
  • Reasoning and planning can be aligned by anchoring the reasoning output to the planned trajectory, so explanations are tied to actual vehicle behavior rather than being detached text.
  • DSDrive's closed-loop scores on the LangAuto short and long routes suggest that distilled reasoning transfers to navigation behavior, not just to language quality.
  • Peak memory around 8 GB, versus about 14 GB for a 7B baseline, moves LLM-based end-to-end driving closer to embedded platforms with limited computational resources.
  • The higher lane-deviation counts reported for DSDrive indicate a known trade-off: route-completion gains come with lateral-instability costs that need targeted improvement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reported short-route gap of 62.05 versus 45.03 survives repeated runs with multiple seeds, it would suggest that the reasoning head acts as a regularizer for the planner beyond simple imitation learning.
  • A natural testable extension is to apply the same distillation recipe to smaller or larger student backbones, such as 0.5B or 3B models, to see whether the reasoning gain scales with student capacity.
  • The principle of making the final plan the terminal reasoning token may generalize beyond driving to other sequential decision tasks where the output can be expressed as a trajectory or plan.
  • Running the same closed-loop benchmark with variance reporting would transform the current point estimates into a statistically grounded comparison, which the paper does not provide.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. DSDrive is an end-to-end autonomous driving system that uses a compact 1B-parameter LLM as its backbone, trained via knowledge distillation from a large VLM (Qwen2.5-VL-max) that generates chain-of-thought reasoning answers. The proposed framework also includes a waypoint-driven dual-head coordination module intended to align the reasoning and planning tasks by embedding ground-truth waypoints into the textual reasoning answers and having both heads share the same hidden states. The model processes multi-view images, LiDAR, navigation instructions, and outputs waypoints, reasoning text, and an end-state indicator. Experiments in the CARLA simulator on the LangAuto benchmark compare DSDrive against LMDrive with LLaVA-7B and LLaMA-1B backbones, as well as against two ablations. The authors report that DSDrive achieves comparable or better Driving Score and Route Completion than LMDrive LLaVA-7B on LangAuto-Long and LangAuto-Short while using less memory and similar inference time, and that the dual-head coordination design improves performance over the ablations. The paper claims that this demonstrates the feasibility of lightweight LLM-based end-to-end driving with interpretable reasoning.

Significance. If the performance claims hold, DSDrive would be a valuable contribution: it shows that reasoning distillation from a large VLM can transfer chain-of-thought capabilities to a 1B-parameter driving model, improving closed-loop planning while keeping memory and latency low, and it offers an interpretable reasoning stream. The idea of embedding waypoints into the reasoning answer to create a shared objective is a reasonable and potentially effective mechanism for aligning high-level text with low-level control. The closed-loop evaluation on the external CARLA LangAuto benchmark is a strength because it avoids circular reasoning, and the comparison with an established baseline (LMDrive) is appropriate. However, the current evidence is conditional: the central comparison rests on unrepeated point estimates, and the ablation claims are overstated, so the significance is not yet fully established.

major comments (2)
  1. [IV.B, Table II] The closed-loop evaluation in Table II is reported as single point estimates with no standard deviation, confidence interval, or number of repeated runs, and no random seeds are given. CARLA is stochastic in vehicle spawns, traffic-light timing, and pedestrian behavior; the paper also does not specify route composition (number of routes per town, episodes per route, traffic/weather variations). Given this, the headline claim in the abstract that DSDrive 'outperforms in many key metrics' is not supported by the data: DSDrive's IS is lower than LMDrive LLaVA-7B on LangAuto-Long (0.77 vs 0.80) and LangAuto-Short (0.81 vs 0.84), and its DS/RC are lower on LangAuto-Tiny (60.67/72.53 vs 61.91/73.19). Even the large LangAuto-Short margin could shrink or disappear under reseeding. Because the central distillation claim rests on this comparison, the authors should add repeated runs with multiple seeds and report variance, and adjust their conclusions accordingly.
  2. [IV.D, Table III] The text states 'our method beats the two ablatives with higher DS, RC, and IS,' but Table III contradicts this: on LangAuto-Tiny, Dual-task RC (54.50) is lower than GT-only (55.83); on LangAuto-Short, Dual-task IS (0.85) is lower than GT-only (0.86). In addition, the two ablations ('CoT only' and 'GT only') change the training dataset, not the architecture of the dual-head coordination module; therefore the experiment validates the dataset design (presence or absence of VLM answers and waypoint text), not the claimed contribution of the dual-head coordination mechanism itself. The section's conclusion overstates what the ablation shows.
minor comments (7)
  1. [I] In the Introduction, 'of of E2E AD' contains a duplicated preposition; it should read 'of E2E AD'.
  2. [IV.D] The heading and body text use 'ablatives' where the intended word is 'ablations'.
  3. [Table II] The model name 'LLaV A-7B' appears with a stray space in several places; it should be 'LLaVA-7B'.
  4. [III.A] Qwen2.5-VL-max is described as 'open-sourced,' but according to the Qwen2.5-VL technical report, the 'max' variant is a proprietary/large-scale model, not openly released; please correct or clarify the model availability.
  5. [IV.A.2] The LangAuto benchmark route composition (e.g., number of routes per town, episodes per route, traffic settings) is not specified; a reference to the original benchmark protocol or a detailed description should be added so the evaluation is reproducible.
  6. [IV.C.3] The comparison with Qwen2.5-VL-7B in Fig. 7 is based on a few handpicked examples; if retained, please provide quantitative reasoning-quality metrics over a fixed set of scenarios to support the claim that DSDrive does not fall short.
  7. [IV.E] The inference-time and memory measurements lack hardware/software specifications (GPU model, batch size, precision); please report them for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: external CARLA LangAuto benchmark supports the central driving claim; self-citations are contextual only.

full rationale

The paper's central quantitative claim is closed-loop driving performance on the CARLA LangAuto benchmark (Table II), which is an external simulator benchmark with scores computed from route completion, infraction, and driving score. The derivation chain is: Qwen2.5-VL-max generates structured CoT reasoning data; ground-truth waypoints are appended to the reasoning template; the compact LLaMA-1B model is trained with a joint loss (Eq. 3: waypoint L1, answer cross-entropy, end-state cross-entropy); the resulting policy is then deployed in CARLA and scored. No parameter is fitted to the benchmark and then reported as a prediction; the benchmark scores are not recomputed from the training targets by construction. The dual-head module does embed ground-truth waypoints into both the reasoning and planning targets, so the observed agreement between the two heads in Fig. 8 is a designed property rather than an emergent empirical discovery, but the paper's headline claim of driving performance rests on external closed-loop scores, not on this agreement. Self-citations ([7], [8], [18], [20]) appear in related-work or introductory context and do not carry the load of the closed-loop result. The lack of repeated runs, seeds, or variance in Table II is a reproducibility/statistical concern, not a circularity concern. Therefore no circular step is identified.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the quality of VLM-generated labels, the validity of distillation as a transfer mechanism, and the stability of the CARLA benchmark. The paper does not release code, data, or prompt templates and reports no variance, so these assumptions carry most of the evidential weight.

free parameters (1)
  • loss weights lambda_1, lambda_2, lambda_3 = not reported
    Eq. (3) balances waypoint, answer, and end-state losses; no values or sensitivity analysis are given, so the reported driving performance is conditional on an unstated hand choice.
assumptions (5)
  • domain assumption Qwen2.5-VL-max produces correct and complete chain-of-thought labels for driving scenes.
    Section III-A constructs the entire distillation dataset from VLM answers without human verification, consistency filtering, or quality metrics.
  • domain assumption Supervised imitation of teacher chain-of-thought tokens transfers reasoning ability to a 1B student.
    This is the core hypothesis of Section III-C; the ablations support it indirectly but do not isolate the transfer mechanism.
  • domain assumption CARLA LangAuto closed-loop scores are stable and representative with no repeated runs.
    Section IV.B reports point estimates only; the conclusions depend on this assumption.
  • domain assumption Teacher-forcing training on ground-truth answer tokens generalizes to autoregressive inference.
    Algorithm 1 trains on ground-truth tokens while Algorithm 2 generates tokens autoregressively; the paper does not address the exposure bias gap.
  • domain assumption The pretrained vision encoder and LLaMA-1B backbone provide sufficient representations for both reasoning and planning.
    These components are imported from prior work and are not validated independently in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DSDrive: Distilling Large Language Model for Lightweight End-to-End Autonomous Driving with Unified Reasoning and Planning." pith.science (2026). https://pith.science/paper/AUF2KNS6

@misc{pith2026250505360,
  author       = {Pith},
  title        = {Pith review of: DSDrive: Distilling Large Language Model for Lightweight End-to-End Autonomous Driving with Unified Reasoning and Planning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AUF2KNS6}},
  note         = {Machine review of arXiv:2505.05360}
}
read the original abstract

We present DSDrive, a streamlined end-to-end paradigm tailored for integrating the reasoning and planning of autonomous vehicles into a unified framework. DSDrive leverages a compact LLM that employs a distillation method to preserve the enhanced reasoning capabilities of a larger-sized vision language model (VLM). To effectively align the reasoning and planning tasks, a waypoint-driven dual-head coordination module is further developed, which synchronizes dataset structures, optimization objectives, and the learning process. By integrating these tasks into a unified framework, DSDrive anchors on the planning results while incorporating detailed reasoning insights, thereby enhancing the interpretability and reliability of the end-to-end pipeline. DSDrive has been thoroughly tested in closed-loop simulations, where it performs on par with benchmark models and even outperforms in many key metrics, all while being more compact in size. Additionally, the computational efficiency of DSDrive (as reflected in its time and memory requirements during inference) has been significantly enhanced. Evidently thus, this work brings promising aspects and underscores the potential of lightweight systems in delivering interpretable and efficient solutions for AD.

Figures

Figures reproduced from arXiv: 2505.05360 by the authors.

Figure 1
Figure 1. The dataset for distillation is prepared with an explicit think-and [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The proposed E2E driving model boasts advanced reasoning and planning capabilities. Its student model ensures inference efficiency and is enhanced [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The explicit think-and-answer reasoning process for representative driving scenarios from the CARLA simulator. This includes urban road and highway [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Performance at a typical urban intersection in response to traffic light [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Performance when the AV approaches an intersection and makes a left [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 7
Figure 7. Figure 7: Comparison of reasoning capabilities of our method to Qwen2.5-VL [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Visualization of the reasoning and planning alignment in CARLA. As illustrated across three distinct operational scenarios: (a) Highway with moderately [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. NetRoller: Interfacing General and Specialized Models for End-to-End Autonomous Driving

    cs.RO 2025-06 conditional novelty 6.0 of 10

    A three-stage adapter extracts early hidden-state features from a driving LLM and injects them into a VAD-based planner, reporting a 16.7% lower collision rate and 12.5% better trajectory similarity on nuScenes.

  2. Large Foundation Models for Trajectory Prediction in Autonomous Driving: A Comprehensive Survey

    cs.RO 2025-09 conditional novelty 4.0 of 10

    A structured survey of LLM-based trajectory prediction methods, organized into trajectory-language mapping, multimodal fusion, and constraint-based reasoning, with benchmarks, metrics, and future directions.

  3. VLM-UDMC: VLM-Enhanced Unified Decision-Making and Motion Control for Urban Autonomous Driving

    cs.RO 2025-07 conditional novelty 4.0 of 10

    VLM-UDMC uses a vision-language model to switch safety cost functions in a model predictive controller and a multi-kernel LSTM to predict traffic trajectories, reporting improved urban driving metrics in CARLA and cam...

Reference graph

Works this paper leans on

46 extracted references · 23 canonical work pages · cited by 3 Pith papers

  1. [1]

    Deep Learning-Based Vehicle Behavior Prediction for Autonomous Driving Applications: A Review,

    S. Mozaffari, O. Y . Al-Jarrah, M. Dianati, P. Jennings, and A. Mouzaki- tis, “Deep Learning-Based Vehicle Behavior Prediction for Autonomous Driving Applications: A Review,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 1, pp. 33–47, 2020

    work page 2020

  2. [2]

    Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys,

    L. Chen, Y . Li, C. Huang, B. Li, Y . Xing, D. Tian, L. Li, Z. Hu, X. Na, Z. Li et al., “Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys,” IEEE Transactions on Intelligent Vehicles , vol. 8, no. 2, pp. 1046–1056, 2022

    work page 2022

  3. [3]

    Is it safe to drive? an overview of factors, metrics, and datasets for driveability assessment in autonomous driving,

    J. Guo, U. Kurup, and M. Shah, “Is it safe to drive? an overview of factors, metrics, and datasets for driveability assessment in autonomous driving,” IEEE Transactions on Intelligent Transportation Systems , vol. 21, no. 8, pp. 3135–3151, 2019

    work page 2019

  4. [4]

    Deep Learning for Safe Autonomous Driving: Current Challenges and Future Directions,

    K. Muhammad, A. Ullah, J. Lloret, J. Del Ser, and V . H. C. De Al- buquerque, “Deep Learning for Safe Autonomous Driving: Current Challenges and Future Directions,” IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 7, pp. 4316–4336, 2020

    work page 2020

  5. [5]

    Claude 3.5 Sonnet,

    Anthropic, “Claude 3.5 Sonnet,” https://www.anthropic.com/news/ claude-3.5-sonnet, Jul. 2024, [Computer software]

    work page 2024

  6. [6]

    GPT-4 Technical Report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “GPT-4 Technical Report,” arXiv preprint arXiv:2303.08774 , 2023

    arXiv 2023

  7. [7]

    CoDriveVLM: VLM-Enhanced Urban Cooperative Dispatching and Motion Planning for Future Autonomous Mobility on Demand Systems,

    H. Liu, R. Yao, W. Liu, Z. Huang, S. Shen, and J. Ma, “CoDriveVLM: VLM-Enhanced Urban Cooperative Dispatching and Motion Planning for Future Autonomous Mobility on Demand Systems,” arXiv preprint arXiv:2501.06132, 2025

    arXiv 2025

  8. [8]

    VLM-E2E: Enhancing End-to-End Autonomous Driving with Multimodal Driver Attention Fusion,

    P. Liu, H. Liu, H. Liu, X. Liu, J. Ni, and J. Ma, “VLM-E2E: Enhancing End-to-End Autonomous Driving with Multimodal Driver Attention Fusion,” arXiv preprint arXiv:2502.18042 , 2025

    arXiv 2025

Show all 46 references
  1. [9]

    Multimodal End-to-End Autonomous Driving,

    Y . Xiao, F. Codevilla, A. Gurram, O. Urfalioglu, and A. M. L ´opez, “Multimodal End-to-End Autonomous Driving,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 1, pp. 537–547, 2020

    2020

  2. [10]

    Reason2Drive: Towards Interpretable and Chain-Based Reasoning for Autonomous Driving,

    M. Nie, R. Peng, C. Wang, X. Cai, J. Han, H. Xu, and L. Zhang, “Reason2Drive: Towards Interpretable and Chain-Based Reasoning for Autonomous Driving,” in European Conference on Computer Vision , 2024, pp. 292–308

    2024

  3. [11]

    Edge intelligence in intelligent transportation systems: A survey,

    T. Gong, L. Zhu, F. R. Yu, and T. Tang, “Edge intelligence in intelligent transportation systems: A survey,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 9, pp. 8919–8944, 2023

    2023

  4. [12]

    Vision Language Models in Autonomous Driving: A Survey and Outlook,

    X. Zhou, M. Liu, E. Yurtsever, B. L. Zagar, W. Zimmer, H. Cao, and A. C. Knoll, “Vision Language Models in Autonomous Driving: A Survey and Outlook,” IEEE Transactions on Intelligent Vehicles , 2024

    2024

  5. [13]

    DeepSeek-R1: Incentivizing Reason- ing Capability in LLMs via Reinforcement Learning,

    D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi et al. , “DeepSeek-R1: Incentivizing Reason- ing Capability in LLMs via Reinforcement Learning,” arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  6. [14]

    Eureka: Human-Level Reward Design via Coding Large Language Models,

    Y . J. Ma, W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Jayara- man, Y . Zhu, L. Fan, and A. Anandkumar, “Eureka: Human-Level Reward Design via Coding Large Language Models,” arXiv preprint arXiv:2310.12931, 2023

    2023 arXiv

  7. [15]

    Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving,

    L. Chen, O. Sinavski, J. H ¨unermann, A. Karnsund, A. J. Willmott, D. Birch, D. Maund, and J. Shotton, “Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving,” in 2024 IEEE International Conference on Robotics and Automation , 2024, pp. 14 0...

    2024

  8. [16]

    DriveMM: All-in-One Large Multimodal Model for Au- tonomous Driving,

    Z. Huang, C. Feng, F. Yan, B. Xiao, Z. Jie, Y . Zhong, X. Liang, and L. Ma, “DriveMM: All-in-One Large Multimodal Model for Au- tonomous Driving,” arXiv preprint arXiv:2412.07689 , 2024

    2024 arXiv

  9. [17]

    Rethinking Closed-Loop Training for Autonomous Driving,

    C. Zhang, R. Guo, W. Zeng, Y . Xiong, B. Dai, R. Hu, M. Ren, and R. Urtasun, “Rethinking Closed-Loop Training for Autonomous Driving,” in European Conference on Computer Vision , 2022, pp. 264– 282

    2022

  10. [18]

    Integrating Decision-Making Into Differentiable Optimization Guided Learning for End-to-End Planning of Autonomous Vehicles,

    W. Liu, Y . Song, C. Meng, Z. Huang, H. Liu, C. Lv, and J. Ma, “Integrating Decision-Making Into Differentiable Optimization Guided Learning for End-to-End Planning of Autonomous Vehicles,” arXiv preprint arXiv:2412.01234, 2024

    2024 arXiv

  11. [19]

    ActiveAD: Planning-Oriented Active Learning for End-to-End Autonomous Driv- ing,

    H. Lu, X. Jia, Y . Xie, W. Liao, X. Yang, and J. Yan, “ActiveAD: Planning-Oriented Active Learning for End-to-End Autonomous Driv- ing,” arXiv preprint arXiv:2403.02877 , 2024

    2024 arXiv

  12. [20]

    CALMM-Drive: Confidence-aware autonomous driving with large mul- timodal model,

    R. Yao, Y . Wang, H. Liu, R. Yang, Z. Peng, L. Zhu, and J. Ma, “CALMM-Drive: Confidence-aware autonomous driving with large mul- timodal model,” arXiv preprint arXiv:2412.04209 , 2024

    2024 arXiv

  13. [21]

    ST-P3: End-to- End Vision-Based Autonomous Driving via Spatial-Temporal Feature Learning,

    S. Hu, L. Chen, P. Wu, H. Li, J. Yan, and D. Tao, “ST-P3: End-to- End Vision-Based Autonomous Driving via Spatial-Temporal Feature Learning,” in European Conference on Computer Vision, 2022, pp. 533– 549

    2022

  14. [22]

    Planning-Oriented Autonomous Driving,

    Y . Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang et al. , “Planning-Oriented Autonomous Driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 17 853–17 862

    2023

  15. [23]

    DriveTransformer: Unified Trans- former for Scalable End-to-End Autonomous Driving,

    X. Jia, J. You, Z. Zhang, and J. Yan, “DriveTransformer: Unified Trans- former for Scalable End-to-End Autonomous Driving,” arXiv preprint arXiv:2503.07656, 2025

    2025 arXiv

  16. [24]

    Think Twice Before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving,

    X. Jia, P. Wu, L. Chen, J. Xie, C. He, J. Yan, and H. Li, “Think Twice Before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 21 983–21 994

    2023

  17. [25]

    Goal-oriented Autonomous Driving,

    Y . Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang, L. Lu, X. Jia, Q. Liu, J. Dai, Y . Qiao, and H. Li, “Goal-oriented Autonomous Driving,” 2022

    2022

  18. [26]

    COOPERNAUT: End- to-End Driving With Cooperative Perception for Networked Vehicles,

    J. Cui, H. Qiu, D. Chen, P. Stone, and Y . Zhu, “COOPERNAUT: End- to-End Driving With Cooperative Perception for Networked Vehicles,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 17 252–17 262

    2022

  19. [27]

    Rea- sonNet: End-to-End Driving With Temporal and Global Reasoning,

    H. Shao, L. Wang, R. Chen, S. L. Waslander, H. Li, and Y . Liu, “Rea- sonNet: End-to-End Driving With Temporal and Global Reasoning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 13 723–13 733

    2023

  20. [28]

    V AD: Vectorized Scene Representation for Efficient Autonomous Driving,

    B. Jiang, S. Chen, Q. Xu, B. Liao, J. Chen, H. Zhou, Q. Zhang, W. Liu, C. Huang, and X. Wang, “V AD: Vectorized Scene Representation for Efficient Autonomous Driving,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 8340–8350

    2023

  21. [29]

    V ADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning,

    S. Chen, B. Jiang, H. Gao, B. Liao, Q. Xu, Q. Zhang, C. Huang, W. Liu, and X. Wang, “V ADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning,” arXiv preprint arXiv:2402.13243 , 2024

    2024 arXiv

  22. [30]

    ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst,

    M. Bansal, A. Krizhevsky, and A. Ogale, “ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst,” arXiv preprint arXiv:1812.03079, 2018

    2018 arXiv

  23. [31]

    PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Driving,

    J. Cheng, Y . Chen, and Q. Chen, “PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Driving,” arXiv preprint arXiv:2404.14327, 2024

    2024 arXiv

  24. [32]

    Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios,

    Y . Lu, J. Fu, G. Tucker, X. Pan, E. Bronstein, R. Roelofs, B. Sapp, B. White, A. Faust, S. Whiteson et al. , “Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios,” in 2023 IEEE/RSJ International Conference on Intellige...

    2023

  25. [33]

    RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Lan- guage Model,

    J. Yuan, S. Sun, D. Omeiza, B. Zhao, P. Newman, L. Kunze, and M. Gadd, “RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Lan- guage Model,” arXiv preprint arXiv:2402.10828 , 2024

    2024

  26. [34]

    AlphaDrive: Un- leashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoning,

    B. Jiang, S. Chen, Q. Zhang, W. Liu, and X. Wang, “AlphaDrive: Un- leashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoning,” arXiv preprint arXiv:2503.07608 , 2025

    2025 arXiv

  27. [35]

    PRIMEDrive-CoT: A Precognitive Chain-of-Thought Framework for Uncertainty-Aware Object Interaction in Driving Scene Scenario,

    S. Mandalika, A. Nambiar et al. , “PRIMEDrive-CoT: A Precognitive Chain-of-Thought Framework for Uncertainty-Aware Object Interaction in Driving Scene Scenario,” arXiv preprint arXiv:2504.05908 , 2025

    2025 arXiv

  28. [36]

    GPT-Driver: Learning to Drive with GPT,

    J. Mao, Y . Qian, J. Ye, H. Zhao, and Y . Wang, “GPT-Driver: Learning to Drive with GPT,” arXiv preprint arXiv:2310.01415 , 2023

    2023 arXiv

  29. [37]

    DriveLM: Driving with Graph Visual Question Answering,

    C. Sima, K. Renz, K. Chitta, L. Chen, H. Zhang, C. Xie, J. Beißwenger, P. Luo, A. Geiger, and H. Li, “DriveLM: Driving with Graph Visual Question Answering,” in European Conference on Computer Vision , 2024, pp. 256–274

    2024

  30. [38]

    VLP: Vision Language Planning for Autonomous Driving,

    C. Pan, B. Yaman, T. Nesti, A. Mallik, A. G. Allievi, S. Velipasalar, and L. Ren, “VLP: Vision Language Planning for Autonomous Driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 14 760–14 769

    2024

  31. [39]

    WiseAD: Knowledge Augmented End-to-End Autonomous Driving with Vision-Language Model,

    S. Zhang, W. Huang, Z. Gao, H. Chen, and C. Lv, “WiseAD: Knowledge Augmented End-to-End Autonomous Driving with Vision-Language Model,” arXiv preprint arXiv:2412.09951 , 2024

    2024 arXiv

  32. [40]

    Sce2DriveX: A Generalized MLLM Framework for Scene-to-Drive Learning,

    R. Zhao, Q. Yuan, J. Li, H. Hu, Y . Li, C. Zheng, and F. Gao, “Sce2DriveX: A Generalized MLLM Framework for Scene-to-Drive Learning,” arXiv preprint arXiv:2502.14917 , 2025. 13

    2025 arXiv

  33. [41]

    LMDrive: Closed-Loop End-to-End Driving with Large Language Models,

    H. Shao, Y . Hu, L. Wang, G. Song, S. L. Waslander, Y . Liu, and H. Li, “LMDrive: Closed-Loop End-to-End Driving with Large Language Models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 15 120–15 130

    2024

  34. [42]

    Senna: Bridging Large Vision- Language Models and End-to-End Autonomous Driving,

    B. Jiang, S. Chen, B. Liao, X. Zhang, W. Yin, Q. Zhang, C. Huang, W. Liu, and X. Wang, “Senna: Bridging Large Vision- Language Models and End-to-End Autonomous Driving,” arXiv preprint arXiv:2410.22313, 2024

    2024 arXiv

  35. [43]

    Qwen2.5-VL Technical Report,

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang et al., “Qwen2.5-VL Technical Report,” arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  36. [44]

    The Llama 3 Herd of Models,

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan et al. , “The Llama 3 Herd of Models,” arXiv preprint arXiv:2407.21783 , 2024

    2024 arXiv

  37. [45]

    BLIP-2: Bootstrapping Language- Image Pre-training with Frozen Image Encoders and Large Language Models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “BLIP-2: Bootstrapping Language- Image Pre-training with Frozen Image Encoders and Large Language Models,” in International Conference on Machine Learning , 2023, pp. 19 730–19 742

    2023

  38. [46]

    CARLA: An Open Urban Driving Simulator,

    A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun, “CARLA: An Open Urban Driving Simulator,” in Conference on Robot Learning, 2017, pp. 1–16

    2017

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.