Pith. sign in

REVIEW 3 major objections 5 minor 16 cited by

A Survey on Vision-Language-Action Models for Autonomous Driving

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This survey organizes the emerging field of vision-language-action driving models into a four-stage taxonomy and defines the shared architecture behind them.

desk verdict A useful and overdue survey map of VLA4AD, but its flagship Stage-1 example contradicts its own table and the cited paper, so it needs a correction round before I'd fully trust it. read the letter →

arxiv 2506.24044 v1 pith:2ACEYQLK submitted 2025-06-30 cs.CV cs.AIcs.RO

Sicong Jiang , Zilin Huang , Kangan Qian , Ziang Luo , Tianze Zhu , Yang Zhong , Yihong Tang , Menglin Kong
show 12 more authors
Yunlong Wang Siwen Jiao Hao Ye Zihao Sheng Xin Zhao Tuopu Wen Zheng Fu Sikai Chen Kun Jiang Diange Yang Seongjin Choi Lijun Sun
This is my paper
classification cs.CVcs.AIcs.RO
keywords vision-language-actionmodelsautonomousdrivingmultimodallargelanguagefour-stagetaxonomychain-of-thoughtreasoningclosed-loopevaluationend-to-endVLA4AD
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vision-language-action (VLA) models for autonomous driving combine camera and sensor input with natural-language commands and low-level control inside one policy, promising vehicles that can follow instructions such as “yield to the ambulance” while explaining what they are doing. This paper aims to map that rapidly growing, fragmented literature: it formalizes the shared building blocks, traces the field through four stages from language-as-explainer to reasoning-augmented agents, and compares over twenty representative systems. It also consolidates the datasets and evaluation protocols that measure driving safety together with language fidelity, and it names robustness, real-time efficiency, and formal verification as the main obstacles. If the map holds, researchers gain a common vocabulary for comparing systems and a concrete agenda for moving language from a passive explanation layer into the closed-loop control path.

What carries the argument

The organizing machinery is the four-stage evolutionary taxonomy together with the architectural triad of vision encoder, language processor, and action decoder. The taxonomy is what lets the survey claim a unified field: each system is placed by how language enters the pipeline (description, intermediate representation, direct input, or reasoning memory) and by how action exits (low-level control tokens, waypoint trajectories, or hierarchical plans). The action-decoder variants, including autoregressive tokenizers, diffusion heads, and flow-based controllers, are the concrete mechanisms that close the loop between words and motion.

What would settle it

A reader could rerun a systematic search over the same period using the survey's own inclusion criteria and classify every returned system into the four stages. If a substantial number of systems fall outside the stages, or if an end-to-end VLA predates the modular stage, the taxonomy's claim to describe the field's evolution would be false.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a coherent research paradigm exists: VLA for autonomous driving (VLA4AD). The paper argues that the field has moved through four identifiable stages: pre-VLA explainers that narrate scenes without touching the controls, modular VLA systems that turn language into intermediate plans, unified end-to-end VLA networks that map sensors directly to trajectories, and reasoning-augmented VLA agents that insert chain-of-thought, memory, and tool use into the control loop. It further claims that all of these systems share a common architecture of vision encoder, language processor, and action decoder, and that the remaining bottlenecks are robustness, real-time throughput, and formal verification rather than basic feasibility.

Load-bearing premise

The survey's story depends on the selected systems being representative of the field as of mid-2025 and the four stages genuinely capturing their development; because the paper does not report a systematic search protocol, that representativeness is asserted rather than demonstrated.

Editorial extensions

If this is right

  • New work can be described by its stage and compared on fixed axes, such as input sensors, language interface, and output form.
  • Evaluation can move from measuring control and language separately to protocols that jointly score safety, instruction fidelity, and explanation quality.
  • The open-challenge list sets a concrete agenda: keeping reasoning inside a real-time control loop, formally verifying language-conditioned policies, and building a unified AI driver's test.
  • If the trend continues, future production systems will embed chain-of-thought reasoning and memory as core parts of the driving policy rather than as an external explanation layer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the four-stage ordering may be a retrospective organizing device rather than a proven chronology, because systems developed in parallel can be sorted into stages only after the fact.
  • My inference: a testable consequence of the reasoning-augmented stage is that adding explicit reasoning tokens before action output should improve closed-loop safety beyond what data scaling alone delivers; that comparison is not made in the survey.
  • My inference: the survey's camera-centric emphasis suggests that making LiDAR, radar, and map data first-class tokens in the language-action loop is a promising direction, with early evidence in the 3D-focused systems it cites.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript presents a survey of Vision-Language-Action (VLA) models for autonomous driving, coining the term VLA4AD. It proposes a four-stage evolution taxonomy (Pre-VLA language-as-explainer, Modular VLA, End-to-end VLA, Reasoning-Augmented VLA), formalizes architectural building blocks (vision encoder, language processor, action decoder), compares over 20 models in Table 1, reviews datasets and benchmarks, and discusses training/evaluation strategies, open challenges, and future directions. The authors claim this is the first comprehensive survey specifically for VLA models in autonomous driving.

Significance. If validated, this survey would provide a useful organizing reference for a rapidly growing field: it offers a common terminology, a consolidated model table, a structured comparison of training and evaluation protocols, and an explicit list of open problems. The paper's strengths include broad coverage of very recent primary sources (mostly 2024-2025 arXiv preprints), a clear architectural decomposition, and a practical GitHub repository. However, the central contributions—the four-stage taxonomy and the 'first comprehensive' claim—are not yet supported as stated: the defining example of Stage 1 contradicts the stage's own definition, and the model selection process is not reproducible from the manuscript. These issues need to be fixed before the survey can serve as a reliable reference.

major comments (3)
  1. [Section 4.1 and Table 1] Section 4.1 defines Stage 1 (Pre-VLA) as producing no direct control, stating that 'the actual vehicle control was still handled by conventional modules (PID controllers, etc.)' and that language was 'an overlay rather than integral to decision-making'. This is directly contradicted by Table 1, which lists DriveGPT-4's output as LLC (low-level control), and by the cited source (Xu et al., DriveGPT4, arXiv:2310.01412), which predicts steering, throttle, and brake tokens and evaluates them closed-loop in CARLA. Since DriveGPT-4 is the only concrete exemplar offered for this stage, the stage boundary fails at its own instance. The authors should either reclassify DriveGPT-4, revise the Stage 1 definition to state the actual distinguishing criterion (e.g., whether the action head is jointly trained with the language model), or both. In addition, Table 1 should include a stage column so that every model's assignment is auditable.
  2. [Section 4 and Table 1 (survey methodology)] The Abstract and Introduction claim that this is 'the first comprehensive survey' of VLA4AD, but no literature search methodology is reported. The paper does not state the databases searched, the date of the search, the keywords used, or the inclusion/exclusion criteria that produced the 20+ models in Table 1. Without this information, the comprehensiveness claim is untestable: a reader cannot check whether important works were omitted or whether the four-stage taxonomy was fitted to a non-representative sample. Please add a methodology paragraph describing the search protocol and explicitly describing how each model was assigned to a stage.
  3. [Sections 4.2-4.4 and Table 1] The boundaries between the four stages are not operationalized. For example, RAG-Driver is placed in Stage 2 (Modular VLA) yet outputs low-level control, while SimLingo and AutoVLA in Stages 3/4 output a mix of LLC and trajectory; conversely, ORION in Stage 4 is described as both reasoning-augmented and end-to-end. With no explicit decision rule for stage assignment, the evolutionary narrative is not checkable from within the paper. Please provide a per-stage definition in terms of architectural or training properties (e.g., whether the action head is backpropagated through the language model, whether an intermediate language representation is consumed by a separate planner) and apply that rule consistently to all entries in Table 1.
minor comments (5)
  1. [Throughout] Several typos need correction: 'V AD' in Section 2.2 should be 'VAD'; 'CarLLaV A' in Section 4.3 should be 'CarLLaVA'; 'exemplifing' (Section 2.4) should be 'exemplifying'; 'remainspassive' (Section 1) should be 'remains passive'; 'outputing' and 'explaning' (Section 4.4) should be 'outputting' and 'explaining'.
  2. [References] The bibliography contains duplicate entries: nuScenes appears as [7] and [8], UniAD as [43] and [44], ORION as [30] and [31], and LLaMA as [120] and [121]. This causes inconsistent in-text numbering (e.g., ORION is cited as [30] in Section 4.4 but as [31] in Table 1). Deduplicate and renumber the references.
  3. [Section 3.2] The text 'LLaMA2 [121]' cites the wrong reference: [121] is the original LLaMA paper, while LLaMA-2 is [122]. Correct this citation.
  4. [Section 6.2] In the evaluation metrics paragraph, 'TS-VLM cuts compute by ~90% through token pooling [167]' should cite [11] (TS-VLM) rather than [167] (DynRsl-VLM); the same confusion appears in Section 1 where [167] is used to support a claim about TS-VLM.
  5. [Figure 3 and Section 4] Stage names are inconsistent between Figure 3's caption ('VLM-as-explainer', 'Modular VLA for AD', 'Augmented VLA for AD', 'End-to-end VLA for AD') and the text in Section 4 ('Explanatory Language Models', 'Modular VLA4AD', 'End-to-end VLA4AD', 'Reasoning-Augmented VLA4AD'). Align the terminology across the figure and the text.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the survey contains no fitted-value or self-citation-reduced derivation; the four-stage taxonomy is an organizing narrative rather than a derivation, though its Stage 1 exemplar contradicts its own definition.

full rationale

This is a survey paper; there is no quantitative derivation chain, no fitted parameter renamed as prediction, and no equation reducing an output to an input. The central product is a taxonomy (Pre-VLA explainers, Modular VLA, End-to-end VLA, Reasoning-augmented VLA) and a comparative table. The taxonomy is not derived from a first-principles model; it is an interpretation imposed on the literature, so the usual circularity modes (self-definitional, fitted-input-called-prediction, uniqueness-imported-from-authors) do not apply. Self-citations appear (e.g., AgentThink [99], FasionAD++ [100], and [61] in a control-method list), but they are descriptive citations in the narrative, not load-bearing evidence for the taxonomy or for any quantitative claim, so they do not raise the circularity score above 2. The paper's own inconsistency involving DriveGPT-4 is worth flagging as a correctness risk: Section 4.1 defines Stage 1 as 'without directly outputting control' and says DriveGPT-4's 'actual vehicle control was still handled by conventional modules,' while Table 1 lists DriveGPT-4's Output as 'LLC' and the cited DriveGPT4 paper predicts steering/throttle/brake tokens. This undermines the exemplar's fit to the stage definition and makes the evolutionary narrative harder to audit, but it is a factual/classification inconsistency, not a circular reduction: the stage claim does not become true by definition, and no result is being derived from the very datum it is supposed to predict. Similarly, the 'first comprehensive survey' claim is not backed by a reported search protocol, but absence of methodology is an evidentiary/completeness limitation, not circularity. Overall the derivation chain, such as it is, is not circular; score 2 reflects the minor self-citations and the need to correct the Stage 1 exemplar.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The survey makes no quantitative predictions and fits no parameters. Its only structural assumption is the framing of VLA4AD as a distinct paradigm, justified by the architecture comparison in Section 3.

assumptions (1)
  • domain assumption The VLA4AD paradigm is a distinct, coherent research direction that warrants a separate survey.
    The Introduction and Section 2.4 assert that VLA models (with an explicit action head) form a new category beyond VLM4AD and end-to-end driving; this framing choice underlies the entire survey but is not proven.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Survey on Vision-Language-Action Models for Autonomous Driving." pith.science (2026). https://pith.science/paper/2ACEYQLK

@misc{pith2026250624044,
  author       = {Pith},
  title        = {Pith review of: A Survey on Vision-Language-Action Models for Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2ACEYQLK}},
  note         = {Machine review of arXiv:2506.24044}
}
read the original abstract

The rapid progress of multimodal large language models (MLLM) has paved the way for Vision-Language-Action (VLA) paradigms, which integrate visual perception, natural language understanding, and control within a single policy. Researchers in autonomous driving are actively adapting these methods to the vehicle domain. Such models promise autonomous vehicles that can interpret high-level instructions, reason about complex traffic scenes, and make their own decisions. However, the literature remains fragmented and is rapidly expanding. This survey offers the first comprehensive overview of VLA for Autonomous Driving (VLA4AD). We (i) formalize the architectural building blocks shared across recent work, (ii) trace the evolution from early explainer to reasoning-centric VLA models, and (iii) compare over 20 representative models according to VLA's progress in the autonomous driving domain. We also consolidate existing datasets and benchmarks, highlighting protocols that jointly measure driving safety, accuracy, and explanation quality. Finally, we detail open challenges - robustness, real-time efficiency, and formal verification - and outline future directions of VLA4AD. This survey provides a concise yet complete reference for advancing interpretable socially aligned autonomous vehicles. Github repo is available at \href{https://github.com/JohnsonJiang1996/Awesome-VLA4AD}{SicongJiang/Awesome-VLA4AD}.

Figures

Figures reproduced from arXiv: 2506.24044 by the authors.

Figure 1
Figure 1. Comparisons of autonomous driving paradigms. (a) End-to-end driving offers direct perception-to-control mapping but lacks [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the VLA4AD Architecture. ing traffic rules, parsing high-level goals, or understanding map-based constraints expressed in natural language [33]. More recent efforts have pushed toward multi-turn dialogs, reasoning chains (e.g., Chain-of-Thought prompting)[50, 119], and tool-augmented language interfaces[41, 88, 99], which support richer forms of reasoning and alignment with human decision-making processe… view at source ↗
Figure 3
Figure 3. Evolution of VLA models for AD. From left to right: (1) VLM-as-explainer: a frozen LLM narrates the driving scene but [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 16 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Steadily moving semi-infinite fracture in plane poroelasticity

    physics.geo-ph 2026-04 unverdicted novelty 7.0 of 10

    A new coupled boundary integral method models steadily moving semi-infinite fractures in plane poroelasticity, solving for mechanical deformation and fluid exchange with verification on analytical test cases.

  2. MoRAL: Sensor-Grounded BEV Reasoning for Compact VLMs toward Edge-Oriented Autonomous Driving

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A 2B VLM fine-tuned in two stages on a physics-encoded bird's-eye view image outperforms a zero-shot 8B VLM on eight driving-reasoning question types and raises emergency-braking recall from 10.8% to 47.8%.

  3. MVPruner: Dynamic Token Pruning for Accelerating Multi-view Vision-Language Models in Autonomous Driving

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    MVPruner is a two-stage dynamic token pruning technique that uses view diversity for initial budget allocation and instruction text for task-aligned selection, delivering 87.3% FLOPs reduction and 4.97x prefilling spe...

  4. EgoDyn-Bench: Evaluating Ego-Motion Understanding in Vision-Centric Foundation Models for Autonomous Driving

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    EgoDyn-Bench finds a Perception Bottleneck: foundation models hold ego-motion logic in language but misalign it with vision, underperforming geometric baselines until given explicit trajectories.

  5. EgoDyn-Bench: Evaluating Ego-Motion Understanding in Vision-Centric Foundation Models for Autonomous Driving

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    EgoDyn-Bench reveals a perception bottleneck in vision-centric foundation models: ego-motion logic derives from language while visual input adds negligible signal, with explicit trajectories restoring consistency.

  6. From Representational Complementarity to Dual Systems: Synergizing VLM and Vision-Only Backbones for End-to-End Driving

    cs.RO 2026-02 conditional novelty 6.0 of 10

    A VLM-based and a vision-only end-to-end planner are behaviorally complementary in a long tail of driving scenarios; selecting the better trajectory lifts NAVSIM PDMS from 90.80 to 92.10 at modest compute.

  7. LinMU: Multimodal Understanding Made Linear

    cs.CV 2026-01 conditional novelty 6.0 of 10

    LinMU achieves linear-complexity multimodal understanding by swapping self-attention for an M-MATE dual-branch block and distilling from a frozen teacher VLM, matching accuracy with up to 2.7x faster TTFT and 9x highe...

  8. VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layer

    cs.RO 2025-12 conditional novelty 6.0 of 10

    AEGIS wraps VLA robot policies in a CBF-based safety layer that uses VLM obstacle identification, raising collision avoidance from 18.69% to 77.85% and task success by 17.25 points on the new SafeLIBERO benchmark.

  9. E3AD: An Emotion-Aware Vision-Language-Action Model for Human-Centric End-to-End Autonomous Driving

    cs.CV 2025-12 conditional novelty 6.0 of 10

    An emotion-aware vision-language-action driving model estimates VAD emotion from commands and uses it to improve grounding and waypoint planning.

  10. PriorEye: Geospatial Visual Priors for End-to-End Autonomous Driving

    cs.CV 2026-06 unverdicted novelty 5.0 of 10

    PriorEye augments end-to-end driving models with a dual-memory architecture that stores and gates geospatial visual priors to improve performance and robustness to sensor corruption on NAVSIM-v2.

  11. MVPruner: Dynamic Token Pruning for Accelerating Multi-view Vision-Language Models in Autonomous Driving

    cs.CV 2026-06 unverdicted novelty 5.0 of 10

    MVPruner is a two-stage adaptive token pruning technique for multi-view VLMs that achieves 87.3% FLOPs reduction and 4.97x prefilling speedup while retaining 98.5% accuracy on DriveLM.

  12. Steadily moving semi-infinite fracture in plane poroelasticity

    physics.geo-ph 2026-04 conditional novelty 5.0 of 10

    XEmbodied achieves SOTA on 18 embodied VQA benchmarks by fusing 3D geometric tokens and distilled physical cues into a 30B VLM with progressive curriculum training.

  13. XEmbodied: A Foundation Model with Enhanced Geometric and Physical Cues for Large-Scale Embodied Environments

    cs.CV 2026-04 unverdicted novelty 4.0 of 10

    XEmbodied is a foundation model that integrates 3D geometric and physical signals into VLMs using a 3D Adapter and Efficient Image-Embodied Adapter, plus progressive curriculum and RL post-training, to improve spatial...

  14. Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Survey

    cs.RO 2025-10 conditional novelty 4.0 of 10

    A survey that groups VLA efficiency techniques into four categories: model architecture, perception features, action generation, and training/inference strategies.

  15. A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approach

    cs.RO 2025-12 conditional novelty 3.0 of 10

    A position/review paper argues data-driven model predictive control is the best route to safe, adaptive, human-like autonomous-driving motion planning, but provides no new derivation or experiment.

  16. Research Challenges and Progress in the End-to-End V2X Cooperative Autonomous Driving Competition

    cs.RO 2025-07 conditional novelty 3.0 of 10

    This paper summarizes the CVPR 2025 V2X cooperative driving challenge, its winning solutions, and the open research problems it reveals.

Reference graph

Works this paper leans on

166 extracted references · 3 canonical work pages · cited by 13 Pith papers

  1. [1]

    Flamingo: a visual language model for few-shot learn- ing

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, An- toine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learn- ing. Advances in neural information processing systems , 35:23716–23736, 2022. 3

    2022

  2. [2]

    An lstm net- work for highway trajectory prediction

    Florent Altch ´e and Arnaud de La Fortelle. An lstm net- work for highway trajectory prediction. In 2017 IEEE 20th international conference on intelligent transportation sys- tems (ITSC), pages 353–359. IEEE, 2017. 2

    2017

  3. [3]

    Vavim and vavam: Autonomous driving through video generative modeling

    Florent Bartoccioni, Elias Ramzi, Victor Besnier, Shashanka Venkataramanan, Tuan-Hung Vu, Yihong Xu, Loick Chambon, Spyros Gidaris, Serkan Odabas, David Hurych, et al. Vavim and vavam: Autonomous driving through video generative modeling. arXiv preprint arXiv:2502.15672, 2025. 5, 8

    arXiv 2025

  4. [4]

    pi0: A vision-language- action flow model for general robot control

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. pi0: A vision-language- action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024. 4

    arXiv

  5. [5]

    Fine-grained affective processing capabilities emerging from large lan- guage models

    Joost Broekens, Bernhard Hilpert, Suzan Verberne, Kim Baraka, Patrick Gebhard, and Aske Plaat. Fine-grained affective processing capabilities emerging from large lan- guage models. In 2023 11th international conference on af- fective computing and intelligent interaction (ACII) , pages 1–8. IEEE, 2023. 4

    2023

  6. [6]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neu- ral information processing systems , 33:1877–1901, 2020. 5

    1901

  7. [7]

    nuscenes: A multimodal dataset for autonomous driving

    Holger Caesar, Alex Bankiti, Orien Lang, et al. nuscenes: A multimodal dataset for autonomous driving. In CVPR,

  8. [8]

    nuscenes: A mul- timodal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A mul- timodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020. 4

    2020

Show all 166 references
  1. [9]

    Learning from all vehi- cles

    Dian Chen and Philipp Kr ¨ahenb¨uhl. Learning from all vehi- cles. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 17222–17231,

  2. [10]

    Insight: Enhancing autonomous driving safety through vision-language models on context-aware hazard detection and edge case evaluation

    Dianwei Chen, Zifan Zhang, Yuchen Liu, and Xian- feng Terry Yang. Insight: Enhancing autonomous driving safety through vision-language models on context-aware hazard detection and edge case evaluation. arXiv e-prints, pages arXiv–2502, 2025. 4

    2025

  3. [11]

    Ts- vlm: Text-guided softsort pooling for vision-language models in multi-view driving reasoning

    Lihong Chen, Hossein Hassani, and Soodeh Nikan. Ts- vlm: Text-guided softsort pooling for vision-language models in multi-view driving reasoning. arXiv preprint arXiv:2505.12670, 2025. 2, 6, 8, 10

    2025 arXiv

  4. [12]

    What data do we need for training an av motion planner? In 2021 IEEE Inter- national Conference on Robotics and Automation (ICRA) , pages 1066–1072

    Long Chen, Lukas Platinsky, Stefanie Speichert, Bła ˙zej Osi´nski, Oliver Scheel, Yawei Ye, Hugo Grimmett, Luca Del Pero, and Peter Ondruska. What data do we need for training an av motion planner? In 2021 IEEE Inter- national Conference on Robotics and Automation (ICRA) , pag...

    2021

  5. [13]

    End-to-end autonomous driving: Challenges and frontiers

    Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, and Hongyang Li. End-to-end autonomous driving: Challenges and frontiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 3

    2024

  6. [14]

    Vadv2: End-to-end vectorized au- tonomous driving via probabilistic planning

    Shaoyu Chen, Bo Jiang, Hao Gao, Bencheng Liao, Qing Xu, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Vadv2: End-to-end vectorized au- tonomous driving via probabilistic planning. arXiv preprint arXiv:2402.13243, 2024. 3

    2024 arXiv

  7. [15]

    Asynchronous large language model en- hanced planner for autonomous driving

    Yuan Chen, Zi-han Ding, Ziqin Wang, Yan Wang, Lijun Zhang, and Si Liu. Asynchronous large language model en- hanced planner for autonomous driving. In European Con- ference on Computer Vision, pages 22–38. Springer, 2024. 3

    2024

  8. [16]

    Ppad: Iterative interactions of prediction and planning for end-to-end autonomous driving

    Zhili Chen, Maosheng Ye, Shuangjie Xu, Tongyi Cao, and Qifeng Chen. Ppad: Iterative interactions of prediction and planning for end-to-end autonomous driving. In European Conference on Computer Vision, pages 239–256. Springer,

  9. [17]

    CoVLA: Comprehensive vision-language-action dataset for au- tonomous driving

    Haohan Chi, Huan-ang Gao, Ziming Liu, et al. CoVLA: Comprehensive vision-language-action dataset for au- tonomous driving. In WACV, 2025. 1, 4, 6, 7, 8, 9

    2025

  10. [18]

    Impromptu vla: Open weights and open data for driving vision- language-action models

    Haohan Chi, Huan-ang Gao, Ziming Liu, et al. Impromptu vla: Open weights and open data for driving vision- language-action models. arXiv preprint arXiv:2505.23757,

    arXiv

  11. [19]

    Neat: Neural attention fields for end-to-end autonomous driving

    Kashyap Chitta, Aditya Prakash, and Andreas Geiger. Neat: Neural attention fields for end-to-end autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15793–15803, 2021. 2

    2021

  12. [20]

    Transfuser: Imita- tion with transformer-based sensor fusion for autonomous driving

    Kashyap Chitta, Aditya Prakash, Bernhard Jaeger, Zehao Yu, Katrin Renz, and Andreas Geiger. Transfuser: Imita- tion with transformer-based sensor fusion for autonomous driving. IEEE transactions on pattern analysis and machine intelligence, 45(11):12878–12895, 2022. 2, 3, 4

    2022

  13. [21]

    Talk2bev: Language-enhanced bird’s- eye view maps for autonomous driving

    Tushar Choudhary, Vikrant Dewangan, Shivam Chand- hok, Shubham Priyadarshan, Anushka Jain, Arun K Singh, Siddharth Srivastava, Krishna Murthy Jatavallabhula, and K Madhava Krishna. Talk2bev: Language-enhanced bird’s- eye view maps for autonomous driving. In2024 IEEE Inter- nat...

    2024

  14. [22]

    A survey on multimodal large lan- guage models for autonomous driving

    Can Cui, Yunsheng Ma, Xu Cao, Wenqian Ye, Yang Zhou, Kaizhao Liang, Jintai Chen, Juanwu Lu, Zichong Yang, Kuei-Da Liao, et al. A survey on multimodal large lan- guage models for autonomous driving. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter ...

    2024

  15. [23]

    Chain-of-thought for autonomous driving: A comprehensive survey and future prospects

    Yixin Cui, Haotian Lin, Shuo Yang, Yixiao Wang, Yanjun Huang, and Hong Chen. Chain-of-thought for autonomous driving: A comprehensive survey and future prospects. arXiv preprint arXiv:2505.20223, 2025. 2, 4

    2025 arXiv

  16. [24]

    Hint-ad: Holistically aligned interpretability in end-to-end autonomous driving

    Kairui Ding, Boyuan Chen, Yuchen Su, Huan-ang Gao, Bu Jin, Chonghao Sima, Wuqiang Zhang, Xiaohui Li, Paul Barsch, Hongyang Li, et al. Hint-ad: Holistically aligned interpretability in end-to-end autonomous driving. arXiv preprint arXiv:2409.06702, 2024. 3

    2024 arXiv

  17. [25]

    Dualad: Disentangling the dynamic and static world for end-to-end driving

    Simon Doll, Niklas Hanselmann, Lukas Schneider, Richard Schulz, Marius Cordts, Markus Enzweiler, and Hendrik Lensch. Dualad: Disentangling the dynamic and static world for end-to-end driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,...

    2024

  18. [26]

    Carla: An open urban driv- ing simulator

    Alexey Dosovitskiy, German Ros, Felipe Codevilla, Anto- nio Lopez, and Vladlen Koltun. Carla: An open urban driv- ing simulator. InConference on robot learning, pages 1–16. PMLR, 2017. 9

    2017

  19. [27]

    On the road to portability: Compressing end-to-end motion planner for autonomous driving

    Kaituo Feng, Changsheng Li, Dongchun Ren, Ye Yuan, and Guoren Wang. On the road to portability: Compressing end-to-end motion planner for autonomous driving. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15099–15108, 2024. 2

    2024

  20. [28]

    Polarpoint-bev: Bird-eye- view perception in polar points for explainable end-to-end autonomous driving

    Yuchao Feng and Yuxiang Sun. Polarpoint-bev: Bird-eye- view perception in polar points for explainable end-to-end autonomous driving. IEEE Transactions on Intelligent Ve- hicles, 2024. 3

    2024

  21. [29]

    Drive like a human: Rethinking autonomous driving with large language models

    Daocheng Fu, Xin Li, Licheng Wen, Min Dou, Pinlong Cai, Botian Shi, and Yu Qiao. Drive like a human: Rethinking autonomous driving with large language models. In 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW), pages 910–919. IEEE, 2024. 3

    2024

  22. [31]

    Orion: A holis- tic end-to-end autonomous driving framework by vision- language instructed action generation

    Haoyu Fu, Diankun Zhang, Zongchuang Zhao, Jianfeng Cui, Dingkang Liang, Chong Zhang, Dingyuan Zhang, Hongwei Xie, Bing Wang, and Xiang Bai. Orion: A holis- tic end-to-end autonomous driving framework by vision- language instructed action generation. arXiv preprint arXiv:2503.1...

    2025 arXiv

  23. [32]

    A survey for foun- dation models in autonomous driving

    Haoxiang Gao, Zhongruo Wang, Yaqian Li, Kaiwen Long, Ming Yang, and Yiqing Shen. A survey for foun- dation models in autonomous driving. arXiv preprint arXiv:2402.01105, 2024. 2

    2024 arXiv

  24. [33]

    Lang- coop: Collaborative driving with language

    Xiangbo Gao, Yuheng Wu, Rujia Wang, et al. Lang- coop: Collaborative driving with language. arXiv preprint arXiv:2504.13406, 2025. 5, 6, 8, 9

    2025 arXiv

  25. [34]

    A review of motion planning techniques for au- tomated vehicles

    David Gonz ´alez, Joshu´e P´erez, Vicente Milan´es, and Fawzi Nashashibi. A review of motion planning techniques for au- tomated vehicles. IEEE Transactions on intelligent trans- portation systems, 17(4):1135–1145, 2015. 2

    2015

  26. [35]

    ipad: Iterative proposal-centric end-to-end autonomous driving

    Ke Guo, Haochen Liu, Xiaojun Wu, Jia Pan, and Chen Lv. ipad: Iterative proposal-centric end-to-end autonomous driving. arXiv preprint arXiv:2505.15111, 2025. 3

    2025 arXiv

  27. [36]

    End-to-end autonomous driving without costly modularization and 3d manual annotation

    Mingzhe Guo, Zhipeng Zhang, Yuan He, Ke Wang, and Liping Jing. End-to-end autonomous driving without costly modularization and 3d manual annotation. arXiv preprint arXiv:2406.17680, 2024. 3

    2024 arXiv

  28. [37]

    Drivem- llm: A benchmark for spatial understanding with multi- modal large language models in autonomous driving.arXiv preprint arXiv:2411.13112, 2024

    Xianda Guo, Ruijun Zhang, Yiqun Duan, Yuhang He, Chenming Zhang, Shuai Liu, and Long Chen. Drivem- llm: A benchmark for spatial understanding with multi- modal large language models in autonomous driving.arXiv preprint arXiv:2411.13112, 2024. 3

    2024 arXiv

  29. [38]

    Dme-driver: Integrating human decision logic and 3d scene perception in autonomous driving

    Wencheng Han, Dongqian Guo, Cheng-Zhong Xu, and Jianbing Shen. Dme-driver: Integrating human decision logic and 3d scene perception in autonomous driving. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 39, pages 3347–3355, 2025. 4

    2025

  30. [39]

    Driveaction: A benchmark for exploring human-like driving decisions in vla models

    Yuhan Hao, Zhengning Li, Lei Sun, Weilong Wang, Naixin Yi, Sheng Song, Caihong Qin, Mofan Zhou, Yifei Zhan, Peng Jia, et al. Driveaction: A benchmark for exploring human-like driving decisions in vla models. arXiv preprint arXiv:2506.05667, 2025. 9

    2025

  31. [40]

    Urban driving with conditional imitation learning

    Jeffrey Hawke, Richard Shen, Corina Gurau, Siddharth Sharma, Daniele Reda, Nikolay Nikolov, Przemysław Mazur, Sean Micklethwaite, Nicolas Griffiths, Amar Shah, et al. Urban driving with conditional imitation learning. In 2020 IEEE International Conference on Robotics and Au- t...

    2020

  32. [41]

    Driveagent: Multi-agent structured reasoning with llm and multimodal sensor fusion for autonomous driving

    Xinmeng Hou, Wuqi Wang, Long Yang, Hao Lin, Jinglun Feng, Haigen Min, and Xiangmo Zhao. Driveagent: Multi-agent structured reasoning with llm and multimodal sensor fusion for autonomous driving. arXiv preprint arXiv:2505.02123, 2025. 5

    2025 arXiv

  33. [42]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022. 5

    2022

  34. [43]

    Planning-oriented autonomous driving

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17853–17862, 2023. 2

    2023

  35. [44]

    Planning-oriented autonomous driving

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17853–17862, 2023. 3, 5

    2023

  36. [45]

    A survey on trajectory-prediction methods for autonomous driving

    Yanjun Huang, Jiatong Du, Ziru Yang, Zewei Zhou, Lin Zhang, and Hong Chen. A survey on trajectory-prediction methods for autonomous driving. IEEE Transactions on Intelligent Vehicles, 7(3):652–674, 2022. 2

    2022

  37. [46]

    Drivemm: All-in-one large multimodal model for au- tonomous driving

    Zhijian Huang, Chengjian Feng, Feng Yan, Baihui Xiao, Zequn Jie, Yujie Zhong, Xiaodan Liang, and Lin Ma. Drivemm: All-in-one large multimodal model for au- tonomous driving. arXiv preprint arXiv:2412.07689, 2024. 3

    2024 arXiv

  38. [47]

    Vlm-rl: A unified vision language models and reinforcement learning framework for safe autonomous driving

    Zilin Huang, Zihao Sheng, Yansong Qu, Junwei You, and Sikai Chen. Vlm-rl: A unified vision language models and reinforcement learning framework for safe autonomous driving. arXiv preprint arXiv:2412.15544, 2024. 3

    2024 arXiv

  39. [48]

    Nora: A small open-sourced generalist vision lan- guage action model for embodied tasks

    Chia-Yu Hung, Qi Sun, Pengfei Hong, Amir Zadeh, Chuan Li, U Tan, Navonil Majumder, Soujanya Poria, et al. Nora: A small open-sourced generalist vision lan- guage action model for embodied tasks. arXiv preprint arXiv:2504.19854, 2025. 5

    2025 arXiv

  40. [49]

    Yolo-v1 to yolo-v8, the rise of yolo and its complementary nature toward digital manufacturing and industrial defect detection

    Muhammad Hussain. Yolo-v1 to yolo-v8, the rise of yolo and its complementary nature toward digital manufacturing and industrial defect detection. Machines, 11(7):677, 2023. 2

    2023

  41. [50]

    Emma: End-to-end mul- timodal model for autonomous driving

    Jyh-Jing Hwang, Runsheng Xu, Hubert Lin, Wei-Chih Hung, Jingwei Ji, Kristy Choi, Di Huang, Tong He, Paul Covington, Benjamin Sapp, et al. Emma: End-to-end mul- timodal model for autonomous driving. arXiv preprint arXiv:2410.23262, 2024. 5, 6, 8

    arXiv

  42. [51]

    Drivelmm-o1: A step-by-step reasoning dataset and large multimodal model for driving scenario understanding

    Ayesha Ishaq, Jean Lahoud, Ketan More, Omkar Thawakar, Ritesh Thawkar, Dinura Dissanayake, Noor Ahsan, Yuhao Li, Fahad Shahbaz Khan, Hisham Cholakkal, et al. Drivelmm-o1: A step-by-step reasoning dataset and large multimodal model for driving scenario understanding. arXiv prep...

    2025 arXiv

  43. [52]

    Narrate: Versatile lan- guage architecture for optimal control in robotics

    Seif Ismail, Antonio Arbues, Ryan Cotterell, Ren ´e Zurbr¨ugg, and Carmen Amo Alonso. Narrate: Versatile lan- guage architecture for optimal control in robotics. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 9628–9635. IEEE, 2024. 2

    2024

  44. [53]

    Adriver-i: A general world model for autonomous driving

    Fan Jia, Weixin Mao, Yingfei Liu, Yucheng Zhao, Yuqing Wen, Chi Zhang, Xiangyu Zhang, and Tiancai Wang. Adriver-i: A general world model for autonomous driving. arXiv preprint arXiv:2311.13549, 2023. 1, 6, 8

    2023 arXiv

  45. [54]

    Think twice be- fore driving: Towards scalable decoders for end-to-end au- tonomous driving

    Xiaosong Jia, Penghao Wu, Li Chen, Jiangwei Xie, Con- ghui He, Junchi Yan, and Hongyang Li. Think twice be- fore driving: Towards scalable decoders for end-to-end au- tonomous driving. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, page...

    2023

  46. [55]

    Bench2drive: Towards multi-ability bench- marking of closed-loop end-to-end autonomous driving

    Xiaosong Jia, Zhenjie Yang, Qifeng Li, Zhiyuan Zhang, and Junchi Yan. Bench2drive: Towards multi-ability bench- marking of closed-loop end-to-end autonomous driving. arXiv preprint arXiv:2406.03877, 2024. 4, 7, 9, 10

    2024 arXiv

  47. [56]

    Drivetransformer: Unified transformer for scalable end-to- end autonomous driving

    Xiaosong Jia, Junqi You, Zhiyuan Zhang, and Junchi Yan. Drivetransformer: Unified transformer for scalable end-to- end autonomous driving. arXiv preprint arXiv:2503.07656,

    arXiv

  48. [57]

    Diffvla: Vision- language guided diffusion planning for autonomous driv- ing

    Anqing Jiang, Yu Gao, Zhigang Sun, et al. Diffvla: Vision- language guided diffusion planning for autonomous driv- ing. arXiv preprint arXiv:2505.19381, 2025. 1, 4, 5, 6, 8, 10

    2025 arXiv

  49. [58]

    Senna: Bridging large vision-language mod- els and end-to-end autonomous driving

    Bo Jiang, Shaoyu Chen, Bencheng Liao, Xingyu Zhang, Wei Yin, Qian Zhang, Chang Huang, Wenyu Liu, and Xing- gang Wang. Senna: Bridging large vision-language mod- els and end-to-end autonomous driving. arXiv preprint arXiv:2410.22313, 2024. 3, 6

    arXiv

  50. [59]

    Vad: Vectorized scene rep- resentation for efficient autonomous driving

    Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jia- jie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene rep- resentation for efficient autonomous driving. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vi...

    2023

  51. [60]

    Koma: Knowledge-driven multi- agent framework for autonomous driving with large lan- guage models

    Kemou Jiang, Xuan Cai, Zhiyong Cui, Aoyong Li, Yi- long Ren, Haiyang Yu, Hao Yang, Daocheng Fu, Licheng Wen, and Pinlong Cai. Koma: Knowledge-driven multi- agent framework for autonomous driving with large lan- guage models. IEEE Transactions on Intelligent Vehicles,

  52. [61]

    Communication-aware reinforcement learning for cooperative adaptive cruise control

    Sicong Jiang, Seongjin Choi, and Lijun Sun. Communication-aware reinforcement learning for cooperative adaptive cruise control. arXiv preprint arXiv:2407.08964, 2024. 2

    2024 arXiv

  53. [62]

    Learning to drive in a day

    Alex Kendall, Jeffrey Hawke, David Janz, Przemyslaw Mazur, Daniele Reda, John-Mark Allen, Vinh-Dieu Lam, Alex Bewley, and Amar Shah. Learning to drive in a day. In 2019 international conference on robotics and automa- tion (ICRA), pages 8248–8254. IEEE, 2019. 2

    2019

  54. [63]

    Jinkyu Kim, Z. Li, B. Floyd, et al. Textual explanations for self-driving vehicles. In ECCV, 2018. 4, 7, 9

    2018

  55. [64]

    Fine-tuning vision-language-action models: Optimizing speed and suc- cess

    Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and suc- cess. arXiv preprint arXiv:2502.19645, 2025. 5

    2025 arXiv

  56. [65]

    Open- vla: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Open- vla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246, 2024. 4

    2024 arXiv

  57. [66]

    A survey on motion prediction and risk assessment for in- telligent vehicles

    St ´ephanie Lef`evre, Dizan Vasquez, and Christian Laugier. A survey on motion prediction and risk assessment for in- telligent vehicles. ROBOMECH journal, 1:1–14, 2014. 2

    2014

  58. [67]

    Pointvla: Injecting the 3d world into vision-language-action models

    Chengmeng Li, Junjie Wen, Yan Peng, Yaxin Peng, Feifei Feng, and Yichen Zhu. Pointvla: Injecting the 3d world into vision-language-action models. arXiv preprint arXiv:2503.07511, 2025. 4, 5

    2025 arXiv

  59. [68]

    Does end-to-end autonomous driving really need perception tasks? arXiv preprint arXiv:2409.18341, 2024

    Peidong Li and Dixiao Cui. Does end-to-end autonomous driving really need perception tasks? arXiv preprint arXiv:2409.18341, 2024. 3

    2024 arXiv

  60. [69]

    Enhancing end-to- end autonomous driving with latent world model

    Yingyan Li, Lue Fan, Jiawei He, Yuqi Wang, Yuntao Chen, Zhaoxiang Zhang, and Tieniu Tan. Enhancing end-to- end autonomous driving with latent world model. arXiv preprint arXiv:2406.08481, 2024. 3

    2024 arXiv

  61. [70]

    Recogdrive: A reinforced cognitive framework for end-to-end autonomous driving, 2025

    Yongkang Li, Kaixin Xiong, Xiangyu Guo, Fang Li, Sixu Yan, Gangwei Xu, Lijun Zhou, Long Chen, Haiyang Sun, Bing Wang, Guang Chen, Hangjun Ye, Wenyu Liu, and Xinggang Wang. Recogdrive: A reinforced cognitive framework for end-to-end autonomous driving, 2025. 5

    2025

  62. [71]

    Hydra-mdp: End-to-end multimodal plan- ning with multi-target hydra-distillation

    Zhenxin Li, Kailin Li, Shihao Wang, Shiyi Lan, Zhid- ing Yu, Yishen Ji, Zhiqi Li, Ziyue Zhu, Jan Kautz, Zux- uan Wu, et al. Hydra-mdp: End-to-end multimodal plan- ning with multi-target hydra-distillation. arXiv preprint arXiv:2406.06978, 2024. 3

    2024 arXiv

  63. [72]

    Generalized trajectory scor- ing for end-to-end multimodal planning

    Zhenxin Li, Wenhao Yao, Zi Wang, Xinglong Sun, Joshua Chen, Nadine Chang, Maying Shen, Zuxuan Wu, Shiyi Lan, and Jose M Alvarez. Generalized trajectory scor- ing for end-to-end multimodal planning. arXiv preprint arXiv:2506.06664, 2025. 3

    2025 arXiv

  64. [73]

    Pnpnet: End-to-end per- ception and prediction with tracking in the loop

    Ming Liang, Bin Yang, Wenyuan Zeng, Yun Chen, Rui Hu, Sergio Casas, and Raquel Urtasun. Pnpnet: End-to-end per- ception and prediction with tracking in the loop. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11553–11562, 2020. 2

    2020

  65. [74]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural infor- mation processing systems , 36:34892–34916, 2023. 1, 3, 5

    2023

  66. [75]

    Mtd-gpt: A multi-task decision-making gpt model for autonomous driving at unsignalized intersections

    Jiaqi Liu, Peng Hang, Xiao Qi, Jianqiang Wang, and Jian Sun. Mtd-gpt: A multi-task decision-making gpt model for autonomous driving at unsignalized intersections. In 2023 IEEE 26th International Conference on Intelligent Trans- portation Systems (ITSC) , pages 5154–5161. IEEE, 2023. 3

    2023

  67. [76]

    Robomamba: Ef- ficient vision-language-action model for robotic reasoning and manipulation

    Jiaming Liu, Mengzhen Liu, Zhenyu Wang, Pengju An, Xiaoqi Li, Kaichen Zhou, Senqiao Yang, Renrui Zhang, Yandong Guo, and Shanghang Zhang. Robomamba: Ef- ficient vision-language-action model for robotic reasoning and manipulation. Advances in Neural Information Pro- cessing Sys...

    2024

  68. [77]

    Two tasks, one goal: Uniting motion and planning for excellent end to end autonomous driving performance

    Lin Liu, Ziying Song, Hongyu Pan, Lei Yang, and Caiyan Jia. Two tasks, one goal: Uniting motion and planning for excellent end to end autonomous driving performance. arXiv preprint arXiv:2504.12667, 2025. 3

    2025 arXiv

  69. [78]

    Vlm-e2e: Enhancing end-to-end autonomous driv- ing with multimodal driver attention fusion

    Pei Liu, Haipeng Liu, Haichao Liu, Xin Liu, Jinxin Ni, and Jun Ma. Vlm-e2e: Enhancing end-to-end autonomous driv- ing with multimodal driver attention fusion. arXiv preprint arXiv:2502.18042, 2025. 4

    2025

  70. [79]

    Reasonplan: Unified scene prediction and decision reasoning for closed-loop au- tonomous driving

    Xueyi Liu, Zuodong Zhong, Yuxin Guo, Yun-Fu Liu, Zhiguo Su, Qichao Zhang, Junli Wang, Yinfeng Gao, Yupeng Zheng, Qiao Lin, et al. Reasonplan: Unified scene prediction and decision reasoning for closed-loop au- tonomous driving. arXiv preprint arXiv:2505.20024, 2025. 4

    2025

  71. [80]

    Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation

    Zhijian Liu, Haotian Tang, Alexander Amini, Xinyu Yang, Huizi Mao, Daniela L Rus, and Song Han. Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation. In 2023 IEEE international conference on robotics and automation (ICRA), pages 2774–2781. IEEE,

    2023

  72. [81]

    Vlm-mpc: Vision language foundation model (vlm)-guided model predictive controller (mpc) for autonomous driving

    Keke Long, Haotian Shi, Jiaxi Liu, and Xiaopeng Li. Vlm-mpc: Vision language foundation model (vlm)-guided model predictive controller (mpc) for autonomous driving. arXiv preprint arXiv:2408.04821, 2024. 3

    2024 arXiv

  73. [82]

    Activead: Planning-oriented active learning for end-to-end autonomous driving

    Han Lu, Xiaosong Jia, Yichen Xie, Wenlong Liao, Xi- aokang Yang, and Junchi Yan. Activead: Planning-oriented active learning for end-to-end autonomous driving. arXiv preprint arXiv:2403.02877, 2024. 3

    2024 arXiv

  74. [83]

    Fast and fu- rious: Real time end-to-end 3d detection, tracking and mo- tion forecasting with a single convolutional net

    Wenjie Luo, Bin Yang, and Raquel Urtasun. Fast and fu- rious: Real time end-to-end 3d detection, tracking and mo- tion forecasting with a single convolutional net. In Pro- ceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 3569–3577, 2018. 2

    2018

  75. [84]

    Dolphins: Multimodal language model for driving

    Yingzi Ma, Yulong Cao, Jiachen Sun, Marco Pavone, and Chaowei Xiao. Dolphins: Multimodal language model for driving. In European Conference on Computer Vision, pages 403–420. Springer, 2024. 3

    2024

  76. [85]

    A survey on vision-language-action models for embodied ai

    Yueen Ma, Zixing Song, Yuzheng Zhuang, Jianye Hao, and Irwin King. A survey on vision-language-action models for embodied ai. arXiv preprint arXiv:2405.14093, 2024. 4

    2024 arXiv

  77. [86]

    Leapvad: A leap in autonomous driving via cogni- tive perception and dual-process thinking

    Yukai Ma, Tiantian Wei, Naiting Zhong, Jianbiao Mei, Tao Hu, Licheng Wen, Xuemeng Yang, Botian Shi, and Yong Liu. Leapvad: A leap in autonomous driving via cogni- tive perception and dual-process thinking. arXiv preprint arXiv:2501.08168, 2025. 3

    2025 arXiv

  78. [87]

    Gpt-driver: Learning to drive with gpt

    Jiageng Mao, Yuxi Qian, Junjie Ye, Hang Zhao, and Yue Wang. Gpt-driver: Learning to drive with gpt. arXiv preprint arXiv:2310.01415, 2023. 3

    2023 arXiv

  79. [88]

    A language agent for autonomous driving

    Jiageng Mao, Junjie Ye, Yuxi Qian, Marco Pavone, and Yue Wang. A language agent for autonomous driving. arXiv preprint arXiv:2311.10813, 2023. 4, 5

    2023 arXiv

  80. [89]

    Lingoqa: Visual question answering for autonomous driving

    Ana-Maria Marcu, Long Chen, Jan H ¨unermann, Alice Karnsund, Benoit Hanotte, Prajwal Chidananda, Saurabh Nair, Vijay Badrinarayanan, Alex Kendall, Jamie Shotton, et al. Lingoqa: Visual question answering for autonomous driving. In European Conference on Computer Vision , pages...

    2024

  81. [90]

    Continuously learning, adapting, and im- proving: A dual-process approach to autonomous driving

    Jianbiao Mei, Yukai Ma, Xuemeng Yang, Licheng Wen, Xinyu Cai, Xin Li, Daocheng Fu, Bo Zhang, Pinlong Cai, Min Dou, et al. Continuously learning, adapting, and im- proving: A dual-process approach to autonomous driving. arXiv preprint arXiv:2405.15324, 2024. 3

    2024 arXiv

  82. [91]

    Chatmpc: Natural language based mpc personalization

    Yuya Miyaoka, Masaki Inoue, and Tomotaka Nii. Chatmpc: Natural language based mpc personalization. In 2024 American Control Conference (ACC) , pages 3598–3603. IEEE, 2024. 3

    2024

  83. [92]

    Data scaling laws for end-to-end autonomous driving

    Alexander Naumann, Xunjiang Gu, Tolga Dimlioglu, Mar- iusz Bojarski, Alperen Degirmenci, Alexander Popov, De- vansh Bisla, Marco Pavone, Urs M¨uller, and Boris Ivanovic. Data scaling laws for end-to-end autonomous driving. arXiv preprint arXiv:2504.04338, 2025. 3

    2025 arXiv

  84. [93]

    Rea- son2drive: Towards interpretable and chain-based reason- ing for autonomous driving

    Ming Nie, Renyuan Peng, Chunwei Wang, et al. Rea- son2drive: Towards interpretable and chain-based reason- ing for autonomous driving. ECCV, 2024. 4, 8, 9, 10

    2024

  85. [94]

    Dinov2: Learning robust visual features without supervi- sion

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervi- sion. arXiv preprint arXiv:2304.07193, 2023. 4, 5

    2023 arXiv

  86. [95]

    A survey of motion planning and control techniques for self-driving urban vehicles

    Brian Paden, Michal ˇC´ap, Sze Zheng Yong, Dmitry Yer- shov, and Emilio Frazzoli. A survey of motion planning and control techniques for self-driving urban vehicles. IEEE Transactions on intelligent vehicles, 1(1):33–55, 2016. 2

    2016

  87. [96]

    Vlp: Vision language planning for autonomous driving

    Chenbin Pan, Burhaneddin Yaman, Tommaso Nesti, Ab- hirup Mallik, Alessandro G Allievi, Senem Velipasalar, and Liu Ren. Vlp: Vision language planning for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14760– 1476...

    2024

  88. [97]

    Lego-drive: Language-enhanced goal-oriented closed-loop end-to-end autonomous driving

    Pranjal Paul, Anant Garg, Tushar Choudhary, Arun Ku- mar Singh, and K Madhava Krishna. Lego-drive: Language-enhanced goal-oriented closed-loop end-to-end autonomous driving. In 2024 IEEE/RSJ International Con- ference on Intelligent Robots and Systems (IROS) , pages 10020–1002...

    2024

  89. [98]

    Fast: Efficient action tokeniza- tion for vision-language-action models

    Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokeniza- tion for vision-language-action models. arXiv preprint arXiv:2501.09747, 2025. 5

    2025 arXiv

  90. [99]

    Agentthink: A unified framework for tool-augmented chain-of-thought reasoning in vision- language models for autonomous driving

    Kangan Qian, Sicong Jiang, Yang Zhong, Ziang Luo, Zilin Huang, Tianze Zhu, Kun Jiang, Mengmeng Yang, Zheng Fu, Jinyu Miao, et al. Agentthink: A unified framework for tool-augmented chain-of-thought reasoning in vision- language models for autonomous driving. arXiv preprint arX...

    2025

  91. [100]

    Fasionad++: Integrating high-level instruc- tion and information bottleneck in fat-slow fusion systems for enhanced safety in autonomous driving with adaptive feedback

    Kangan Qian, Ziang Luo, Sicong Jiang, Zilin Huang, Jinyu Miao, Zhikun Ma, Tianze Zhu, Jiayin Li, Yangfan He, Zheng Fu, et al. Fasionad++: Integrating high-level instruc- tion and information bottleneck in fat-slow fusion systems for enhanced safety in autonomous driving with a...

    2025 arXiv

  92. [101]

    Spatialvla: Exploring spatial represen- tations for visual-language-action model

    Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang, JiaYuan Gu, Bin Zhao, Dong Wang, et al. Spatialvla: Exploring spatial represen- tations for visual-language-action model. arXiv preprint arXiv:2501.15830, 2025. 5

    2025 arXiv

  93. [102]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International conference on machine learning...

    2021

  94. [103]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023. 5

    2023

  95. [104]

    Simlingo: Vision-only closed-loop autonomous driving with language-action alignment

    Katrin Renz, Long Chen, Elahe Arani, and Oleg Sinavski. Simlingo: Vision-only closed-loop autonomous driving with language-action alignment. In Proceedings of the Computer Vision and Pattern Recognition Conference , pages 11993–12003, 2025. 6

    2025

  96. [105]

    Carllava: Vision language models for camera- only closed-loop driving

    Katrin Renz, Long Chen, Ana-Maria Marcu, Jamie Shot- ton, et al. Carllava: Vision language models for camera- only closed-loop driving. In CVPR, 2025. 1, 2, 4, 6, 8, 9, 10, 11

    2025

  97. [106]

    Plant: Ex- plainable planning transformers via object-level represen- tations

    Katrin Renz, Kashyap Chitta, Otniel-Bogdan Mercea, A Koepke, Zeynep Akata, and Andreas Geiger. Plant: Ex- plainable planning transformers via object-level represen- tations. arXiv preprint arXiv:2210.14222, 2022. 2

    2022 arXiv

  98. [107]

    Perceive, predict, and plan: Safe motion planning through interpretable se- mantic representations

    Abbas Sadat, Sergio Casas, Mengye Ren, Xinyu Wu, Pranaab Dhawan, and Raquel Urtasun. Perceive, predict, and plan: Safe motion planning through interpretable se- mantic representations. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Pr...

    2020

  99. [108]

    Vision-language-action models: Con- cepts, progress, applications and challenges

    Ranjan Sapkota, Yang Cao, Konstantinos I Roumeliotis, and Manoj Karkee. Vision-language-action models: Con- cepts, progress, applications and challenges. arXiv preprint arXiv:2505.04769, 2025. 4

    2025

  100. [109]

    Planning and decision-making for autonomous vehicles

    Wilko Schwarting, Javier Alonso-Mora, and Daniela Rus. Planning and decision-making for autonomous vehicles. Annual Review of Control, Robotics, and Autonomous Sys- tems, 1(1):187–210, 2018. 2

    2018

  101. [110]

    Lmdrive: Closed-loop end-to-end driving with large language mod- els

    Hao Shao, Yuxuan Hu, Letian Wang, Guanglu Song, Steven L Waslander, Yu Liu, and Hongsheng Li. Lmdrive: Closed-loop end-to-end driving with large language mod- els. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 15120–15130,

  102. [111]

    Safety-enhanced autonomous driving using inter- pretable sensor fusion transformer

    Hao Shao, Letian Wang, Ruobing Chen, Hongsheng Li, and Yu Liu. Safety-enhanced autonomous driving using inter- pretable sensor fusion transformer. In Conference on Robot Learning, pages 726–737. PMLR, 2023. 2, 3

    2023

  103. [112]

    Reasonnet: End-to-end driving with temporal and global reasoning

    Hao Shao, Letian Wang, Ruobing Chen, Steven L Waslan- der, Hongsheng Li, and Yu Liu. Reasonnet: End-to-end driving with temporal and global reasoning. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13723–13733, 2023. 2

    2023

  104. [113]

    Deepseekmath: Pushing the limits of math- ematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junx- iao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of math- ematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024. 5

    2024 arXiv

  105. [114]

    Utilizing navigation paths to generate target points for enhanced end-to-end autonomous driving planning

    Yuanhua Shen and Jun Li. Utilizing navigation paths to generate target points for enhanced end-to-end autonomous driving planning. arXiv preprint arXiv:2406.08349, 2024. 3

    2024 arXiv

  106. [115]

    Driv- elm: Driving with graph visual question answering

    Chonghao Sima, Katrin Renz, Kashyap Chitta, et al. Driv- elm: Driving with graph visual question answering. ECCV,

  107. [116]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aure- lien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF conference on computer ...

    2020

  108. [117]

    Sparsedrive: End-to-end au- tonomous driving via sparse scene representation

    Wenchao Sun, Xuewu Lin, Yining Shi, Chuang Zhang, Haoran Wu, and Sifa Zheng. Sparsedrive: End-to-end au- tonomous driving via sparse scene representation. arXiv preprint arXiv:2405.19620, 2024. 3

    2024 arXiv

  109. [118]

    Tokenize the world into object-level knowledge to address long-tail events in autonomous driving

    Ran Tian, Boyi Li, Xinshuo Weng, Yuxiao Chen, Edward Schmerling, Yue Wang, Boris Ivanovic, and Marco Pavone. Tokenize the world into object-level knowledge to address long-tail events in autonomous driving. arXiv preprint arXiv:2407.00959, 2024. 3

    2024 arXiv

  110. [119]

    Drivevlm: The convergence of autonomous driving and large vision-language models

    Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, and Hang Zhao. Drivevlm: The convergence of autonomous driving and large vision-language models. arXiv preprint arXiv:2402.12289, 2024. 3, 5

    2024 arXiv

  111. [121]

    Llama: Open and efficient foundation language mod- els

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth ´ee Lacroix, Bap- tiste Rozi `ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language mod- els. arXiv preprint arXiv:2302.13971, 2023. 5

    2023 arXiv

  112. [122]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023. 3

    2023 arXiv

  113. [123]

    Autonomous driving in traffic: Boss and the urban challenge

    Chris Urmson, Chris Baker, John Dolan, Paul Rybski, Bryan Salesky, William “Red” Whittaker, Dave Ferguson, and Michael Darms. Autonomous driving in traffic: Boss and the urban challenge. AI Magazine, 30(2):17–28, 2009. 2

    2009

  114. [124]

    Drivecot: Integrating chain-of-thought reasoning with end-to-end driving

    Tianqi Wang, Enze Xie, Ruihang Chu, Zhenguo Li, and Ping Luo. Drivecot: Integrating chain-of-thought reasoning with end-to-end driving. arXiv preprint arXiv:2403.16996,

    arXiv

  115. [125]

    Drive anywhere: Generalizable end-to-end autonomous driving with multi-modal foundation models

    Tsun-Hsuan Wang, Alaa Maalouf, Wei Xiao, Yutong Ban, Alexander Amini, Guy Rosman, Sertac Karaman, and Daniela Rus. Drive anywhere: Generalizable end-to-end autonomous driving with multi-modal foundation models. In 2024 IEEE International Conference on Robotics and Automation (...

    2024

  116. [126]

    Drivemlm: Align- ing multi-modal large language models with behavioral planning states for autonomous driving

    Wenhai Wang, Jiangwei Xie, ChuanYang Hu, Haoming Zou, Jianan Fan, Wenwen Tong, Yang Wen, Silei Wu, Hanming Deng, Zhiqi Li, et al. Drivemlm: Align- ing multi-modal large language models with behavioral planning states for autonomous driving. arXiv preprint arXiv:2312.09245, 2023. 3

    2023

  117. [127]

    S4tp: Social- suitable and safety-sensitive trajectory planning for au- tonomous vehicles

    Xiao Wang, Ke Tang, Xingyuan Dai, Jintao Xu, Quancheng Du, Rui Ai, Yuxiao Wang, and Weihao Gu. S4tp: Social- suitable and safety-sensitive trajectory planning for au- tonomous vehicles. IEEE Transactions on Intelligent Ve- hicles, 9(2):3220–3231, 2023. 3

    2023

  118. [128]

    Driv- ingdojo dataset: Advancing interactive and knowledge- enriched driving world model

    Yuqi Wang, Ke Cheng, Jiawei He, Qitai Wang, Hengchen Dai, Yuntao Chen, Fei Xia, and Zhao-Xiang Zhang. Driv- ingdojo dataset: Advancing interactive and knowledge- enriched driving world model. Advances in Neural Infor- mation Processing Systems, 37:13020–13034, 2024. 4

    2024

  119. [129]

    Occllama: An occupancy-language-action generative world model for au- tonomous driving

    Julong Wei, Shanshuai Yuan, Pengfei Li, Qingda Hu, Zhongxue Gan, and Wenchao Ding. Occllama: An occupancy-language-action generative world model for au- tonomous driving. arXiv preprint arXiv:2409.03272, 2024. 4

    2024 arXiv

  120. [130]

    Diffusion-vla: Scaling robot founda- tion models via unified diffusion and autoregression

    Junjie Wen, Minjie Zhu, Yichen Zhu, Zhibin Tang, Jinming Li, Zhongyi Zhou, Chengmeng Li, Xiaoyu Liu, Yaxin Peng, Chaomin Shen, et al. Diffusion-vla: Scaling robot founda- tion models via unified diffusion and autoregression. arXiv preprint arXiv:2412.03293, 2024. 5

    2024 arXiv

  121. [131]

    Dilu: A knowledge-driven approach to au- tonomous driving with large language models

    Licheng Wen, Daocheng Fu, Xin Li, Xinyu Cai, Tao Ma, Pinlong Cai, Min Dou, Botian Shi, Liang He, and Yu Qiao. Dilu: A knowledge-driven approach to au- tonomous driving with large language models. arXiv preprint arXiv:2309.16292, 2023. 4

    2023 arXiv

  122. [132]

    Para-drive: Parallelized architecture for real-time autonomous driving

    Xinshuo Weng, Boris Ivanovic, Yan Wang, Yue Wang, and Marco Pavone. Para-drive: Parallelized architecture for real-time autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15449–15458, 2024. 3

    2024

  123. [133]

    Argoverse 2: Next generation datasets for self-driving perception and forecasting

    Benjamin Wilson, William Qi, Tanmay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, et al. Argoverse 2: Next generation datasets for self-driving perception and forecasting. arXiv preprint arXiv:230...

    2023 arXiv

  124. [134]

    Bev- driver: Leveraging bev maps in llms for robust closed-loop driving

    Katharina Winter, Mark Azer, and Fabian B Flohr. Bev- driver: Leveraging bev maps in llms for robust closed-loop driving. arXiv preprint arXiv:2503.03074, 2025. 3, 4

    2025 arXiv

  125. [135]

    Con- vnext v2: Co-designing and scaling convnets with masked autoencoders

    Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. Con- vnext v2: Co-designing and scaling convnets with masked autoencoders. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16133– 161...

    2023

  126. [136]

    Are vlms ready for autonomous driving? an empirical study from the reliability, data, and metric per- spectives

    Shaoyuan Xie, Lingdong Kong, Yuhao Dong, Chonghao Sima, Wenwei Zhang, Qi Alfred Chen, Ziwei Liu, and Liang Pan. Are vlms ready for autonomous driving? an empirical study from the reliability, data, and metric per- spectives. arXiv preprint arXiv:2501.04003, 2025. 3

    2025 arXiv

  127. [137]

    Autoregressive models in vision: A survey

    Jing Xiong, Gongye Liu, Lun Huang, Chengyue Wu, Taiqiang Wu, Yao Mu, Yuan Yao, Hui Shen, Zhongwei Wan, Jinfa Huang, et al. Autoregressive models in vision: A survey. arXiv preprint arXiv:2411.05902, 2024. 5

    2024 arXiv

  128. [138]

    M2da: multi-modal fusion transformer incorporating driver attention for autonomous driving

    Dongyang Xu, Haokun Li, Qingfan Wang, Ziying Song, Lei Chen, and Hanming Deng. M2da: multi-modal fusion transformer incorporating driver attention for autonomous driving. arXiv preprint arXiv:2403.12552, 2024. 2

    2024 arXiv

  129. [139]

    Chatbev: A visual language model that understands bev maps

    Qingyao Xu, Siheng Chen, Guang Chen, Yanfeng Wang, and Ya Zhang. Chatbev: A visual language model that understands bev maps. arXiv preprint arXiv:2503.13938 ,

    arXiv

  130. [140]

    Vlm-ad: End-to-end autonomous driving through vision-language model supervision

    Yi Xu, Yuxin Hu, Zaiwei Zhang, Gregory P Meyer, Siva Karthik Mustikovela, Siddhartha Srinivasa, Eric M Wolff, and Xin Huang. Vlm-ad: End-to-end autonomous driving through vision-language model supervision. arXiv preprint arXiv:2412.14446, 2024. 3, 4

    2024 arXiv

  131. [141]

    Drivegpt4: Interpretable end-to-end autonomous driving via large language model

    Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee K Wong, Zhenguo Li, and Hengshuang Zhao. Drivegpt4: Interpretable end-to-end autonomous driving via large language model. IEEE Robotics and Au- tomation Letters, 2024. 4, 6, 8

    2024

  132. [142]

    Lidar-llm: Exploring the potential of large language models for 3d lidar understanding

    Senqiao Yang, Jiaming Liu, Renrui Zhang, Mingjie Pan, Ziyu Guo, Xiaoqi Li, Zehui Chen, Peng Gao, Hongsheng Li, Yandong Guo, et al. Lidar-llm: Exploring the potential of large language models for 3d lidar understanding. InPro- ceedings of the AAAI Conference on Artificial Intel...

    2025

  133. [143]

    Drivemoe: Mixture-of-experts for vision-language-action model in end-to-end autonomous driving

    Zhenjie Yang, Yilin Chai, Xiaosong Jia, Yuqian Shao, et al. Drivemoe: Mixture-of-experts for vision-language-action model in end-to-end autonomous driving. arXiv preprint arXiv:2505.16278, 2025. 5, 6, 7, 8, 9, 10, 11

    2025 arXiv

  134. [144]

    Drivesuprim: To- wards precise trajectory selection for end-to-end planning

    Wenhao Yao, Zhenxin Li, Shiyi Lan, Zi Wang, Xinglong Sun, Jose M Alvarez, and Zuxuan Wu. Drivesuprim: To- wards precise trajectory selection for end-to-end planning. arXiv preprint arXiv:2506.06659, 2025. 3

    2025

  135. [145]

    Fusionad: Multi-modality fusion for prediction and planning tasks of autonomous driving

    Tengju Ye, Wei Jing, Chunyong Hu, Shikun Huang, Ling- ping Gao, Fangzhen Li, Jingke Wang, Ke Guo, Wencong Xiao, Weibo Mao, et al. Fusionad: Multi-modality fusion for prediction and planning tasks of autonomous driving. arXiv preprint arXiv:2308.01006, 2023. 2

    2023 arXiv

  136. [146]

    Visual object recognition with 3d-aware features in kitti urban scenes

    J Javier Yebes, Luis M Bergasa, and Miguel ´Angel Garc´ıa- Garrido. Visual object recognition with 3d-aware features in kitti urban scenes. Sensors, 15(4):9228–9250, 2015. 4

    2015

  137. [147]

    Bdd100k: A diverse driving video database with scal- able annotation tooling

    Fisher Yu, Wenqi Xian, Yingying Chen, Fangchen Liu, Mike Liao, Vashisht Madhavan, Trevor Darrell, et al. Bdd100k: A diverse driving video database with scal- able annotation tooling. arXiv preprint arXiv:1805.04687, 2(5):6, 2018. 7, 9

    2018 arXiv

  138. [148]

    Rag- driver: Generalisable driving explanations with retrieval- augmented in-context learning in multi-modal large lan- guage model

    Jianhao Yuan, Shuyang Sun, Daniel Omeiza, Bo Zhao, Paul Newman, Lars Kunze, and Matthew Gadd. Rag- driver: Generalisable driving explanations with retrieval- augmented in-context learning in multi-modal large lan- guage model. arXiv preprint arXiv:2402.10828, 2024. 4, 5, 6, 8, 11

    2024

  139. [149]

    End-to- end interpretable neural motion planner

    Wenyuan Zeng, Wenjie Luo, Simon Suo, Abbas Sadat, Bin Yang, Sergio Casas, and Raquel Urtasun. End-to- end interpretable neural motion planner. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8660–8669, 2019. 2

    2019

  140. [150]

    World knowledge-enhanced reasoning using instruction-guided interactor in autonomous driving

    Mingliang Zhai, Cheng Li, Zengyuan Guo, Ningrui Yang, Xiameng Qin, Sanyuan Zhao, Junyu Han, Ji Tao, Yuwei Wu, and Yunde Jia. World knowledge-enhanced reasoning using instruction-guided interactor in autonomous driving. In Proceedings of the AAAI Conference on Artificial Intel-...

    2025

  141. [151]

    Sparsead: Sparse query-centric paradigm for efficient end-to-end au- tonomous driving

    Diankun Zhang, Guoan Wang, Runwen Zhu, Jianbo Zhao, Xiwu Chen, Siyu Zhang, Jiahao Gong, Qibin Zhou, Wenyuan Zhang, Ningzi Wang, et al. Sparsead: Sparse query-centric paradigm for efficient end-to-end au- tonomous driving. arXiv preprint arXiv:2404.06892, 2024. 3

    2024 arXiv

  142. [152]

    Minidrive: More efficient vision-language models with multi-level 2d features as text tokens for autonomous driving

    Enming Zhang, Xingyuan Dai, Yisheng Lv, and Qinghai Miao. Minidrive: More efficient vision-language models with multi-level 2d features as text tokens for autonomous driving. arXiv preprint arXiv:2409.07267, 2024. 5

    2024 arXiv

  143. [153]

    Safeauto: Knowledge-enhanced safe autonomous driving with multimodal foundation models

    Jiawei Zhang, Xuan Yang, Taiqi Wang, Yu Yao, Aleksandr Petiushko, and Bo Li. Safeauto: Knowledge-enhanced safe autonomous driving with multimodal foundation models. arXiv preprint arXiv:2503.00211, 2025. 4, 6, 7, 8, 9, 10, 11

    2025 arXiv

  144. [154]

    Instruct large language models to drive like humans

    Ruijun Zhang, Xianda Guo, Wenzhao Zheng, Chenming Zhang, Kurt Keutzer, and Long Chen. Instruct large language models to drive like humans. arXiv preprint arXiv:2406.07296, 2024. 3

    2024 arXiv

  145. [155]

    Analysis of a modu- lar autonomous driving architecture: The top submis- sion to carla leaderboard 2.0 challenge

    Weize Zhang, Mohammed Elmahgiubi, Kasra Rezaee, Behzad Khamidehi, Hamidreza Mirkhani, Fazel Arasteh, Chunlin Li, Muhammad Ahsan Kaleem, Eduardo R Corral- Soto, Dhruv Sharma, et al. Analysis of a modu- lar autonomous driving architecture: The top submis- sion to carla leaderboa...

    2024 arXiv

  146. [156]

    Graphad: Interaction scene graph for end-to-end autonomous driving

    Yunpeng Zhang, Deheng Qian, Ding Li, Yifeng Pan, Yong Chen, Zhenbao Liang, Zhiyao Zhang, Shurui Zhang, Hongxu Li, Maolei Fu, et al. Graphad: Interaction scene graph for end-to-end autonomous driving. arXiv preprint arXiv:2403.19098, 2024. 3

    2024 arXiv

  147. [157]

    Bev- erse: Unified perception and prediction in birds-eye-view for vision-centric autonomous driving

    Yunpeng Zhang, Zheng Zhu, Wenzhao Zheng, Junjie Huang, Guan Huang, Jie Zhou, and Jiwen Lu. Bev- erse: Unified perception and prediction in birds-eye-view for vision-centric autonomous driving. arXiv preprint arXiv:2205.09743, 2022. 2

    2022 arXiv

  148. [158]

    Mp- drive: Improving spatial understanding with marker-based prompt learning for autonomous driving

    Zhiyuan Zhang, Xiaofan Li, Zhihao Xu, Wenjie Peng, Zi- jian Zhou, Miaojing Shi, and Shuangping Huang. Mp- drive: Improving spatial understanding with marker-based prompt learning for autonomous driving. In Proceedings of the Computer Vision and Pattern Recognition Conference ,...

    2025

  149. [159]

    Sce2drivex: A generalized mllm framework for scene-to-drive learning.arXiv preprint arXiv:2502.14917, 2025

    Rui Zhao, Qirui Yuan, Jinyu Li, Haofeng Hu, Yun Li, Chengyuan Zheng, and Fei Gao. Sce2drivex: A generalized mllm framework for scene-to-drive learning.arXiv preprint arXiv:2502.14917, 2025. 3

    2025 arXiv

  150. [160]

    Extending large vision- language model for diverse interactive tasks in autonomous driving

    Zongcai Zhao, Yue Zhao, et al. Extending large vision- language model for diverse interactive tasks in autonomous driving. arXiv preprint arXiv:2505.08725, 2025. 2, 8, 9, 10

    2025 arXiv

  151. [161]

    3d-vla: A 3d vision-language-action generative world model.arXiv preprint arXiv:2403.09631, 2024

    Haoyu Zhen, Xiaowen Qiu, Peihao Chen, Jincheng Yang, Xin Yan, Yilun Du, Yining Hong, and Chuang Gan. 3d-vla: A 3d vision-language-action generative world model.arXiv preprint arXiv:2403.09631, 2024. 5

    2024 arXiv

  152. [162]

    Genad: Generative end-to-end au- tonomous driving

    Wenzhao Zheng, Ruiqi Song, Xianda Guo, Chenming Zhang, and Long Chen. Genad: Generative end-to-end au- tonomous driving. In European Conference on Computer Vision, pages 87–104. Springer, 2024. 3

    2024

  153. [163]

    Preliminary investigation into data scaling laws for imitation learning-based end-to-end autonomous driving

    Yupeng Zheng, Zhongpu Xia, Qichao Zhang, Teng Zhang, Ben Lu, Xiaochuang Huo, Chao Han, Yixian Li, Mengjie Yu, Bu Jin, et al. Preliminary investigation into data scaling laws for imitation learning-based end-to-end autonomous driving. arXiv preprint arXiv:2412.02689, 2024. 3

    2024

  154. [164]

    Enhance planning with physics-informed safety controller for end-to-end autonomous driving

    Hang Zhou, Haichao Liu, Hongliang Lu, Jun Ma, and Yid- ing Ji. Enhance planning with physics-informed safety controller for end-to-end autonomous driving. In 2024 IEEE International Conference on Robotics and Biomimet- ics (ROBIO), pages 1775–1782. IEEE, 2024. 3, 4

    2024

  155. [165]

    Opendrivevla: Towards end-to-end au- tonomous driving with large vision language action model

    Xingcheng Zhou, Xuyuan Han, Feng Yang, Yunpu Ma, and Alois C Knoll. Opendrivevla: Towards end-to-end au- tonomous driving with large vision language action model. arXiv preprint arXiv:2503.23463, 2025. 1, 5, 6, 8

    2025

  156. [166]

    Vision language models in autonomous driving: A survey and out- look

    Xingcheng Zhou, Mingyu Liu, Ekim Yurtsever, Bare Luka Zagar, Walter Zimmer, Hu Cao, and Alois C Knoll. Vision language models in autonomous driving: A survey and out- look. IEEE Transactions on Intelligent Vehicles, 2024. 2, 4

    2024

  157. [167]

    Dynrsl- vlm: Enhancing autonomous driving perception with dy- namic resolution vision-language models

    Xirui Zhou, Lianlei Shan, and Xiaolin Gui. Dynrsl- vlm: Enhancing autonomous driving perception with dy- namic resolution vision-language models. arXiv preprint arXiv:2503.11265, 2025. 2, 4, 6, 10, 11

    2025 arXiv

  158. [168]

    Zhao, Yun Zhang, Zhiyu Huang, Bolei Zhou, and Jiaqi Ma

    Zewei Zhou, Tianhui Cai, Seth Z. Zhao, Yun Zhang, Zhiyu Huang, Bolei Zhou, and Jiaqi Ma. Autovla: A vision- language-action model for end-to-end autonomous driv- ing with adaptive reasoning and reinforcement fine-tuning,

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.