Pith. sign in

REVIEW 3 major objections 6 minor 33 references

Trajectory-pattern anchors let language-vision models pick driving behaviors while residual flow fills in continuous paths, raising closed-loop success to 77.28 on Bench2Drive.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 04:17 UTC pith:TRDCW4E7

load-bearing objection Clean hierarchical VLA interface (pattern anchors + residual flow) that actually lifts Bench2Drive SR, with a real but well-documented failure mode when the fixed codebook is wrong. the 3 major comments →

arxiv 2607.03182 v1 pith:TRDCW4E7 submitted 2026-07-03 cs.RO cs.AI

AnchorVLA: Bridging Discrete Decisions and Continuous Trajectories for Vision-Language-Action Planning

Qi Liu , Yabei Li , Hongsong Wang , Heng Zhang , Lei He This is my paper
classification cs.RO cs.AI
keywords vision-language-actionautonomous drivingtrajectory planningtrajectory-pattern anchorsresidual flow matchingclosed-loop evaluationBench2Drivehierarchical decision making
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Existing vision-language-action planners for driving either treat the language model as a loose conditioner for a separate trajectory head or force it to emit long sequences of low-meaning coordinate tokens. This paper claims that gap can be closed by raising the action abstraction: the model first selects compact trajectory-pattern anchors—each a whole local motion pattern such as lane-keeping, yielding, or overtaking—then generates the executable path as a continuous residual inside the chosen pattern. Decision-as-Anchor Representation maps multimodal context onto that codebook; Decision-Anchored Residual Flow models multi-modal refinements by flow matching in the anchor-centered residual space. On the Bench2Drive closed-loop benchmark the resulting hierarchical planner reaches a state-of-the-art Success Rate of 77.28 and a competitive Driving Score of 89.92, while keeping autoregressive work short and semantic. A sympathetic reader cares because the design keeps the language model’s strength in discrete behavior-level decisions without sacrificing continuous control flexibility or inference efficiency.

Core claim

AnchorVLA shows that high-level VLA reasoning and continuous trajectory execution can be bridged by treating clustered full-motion patterns as explicit decision anchors. The language-vision model selects among these anchors as behavior-level decisions; continuous trajectories are then formed as the selected anchor plus a residual generated by flow matching in that residual space. This hierarchical interface yields state-of-the-art closed-loop Success Rate 77.28 and Driving Score 89.92 on Bench2Drive, outperforming both weak planning-head VLAs and full-trajectory autoregressive token generation.

What carries the argument

Trajectory-pattern anchors under Decision-as-Anchor Representation (DAAR), each a complete local motion pattern from a fixed codebook, together with Decision-Anchored Residual Flow (DARF): the final path is the chosen anchor plus a continuous residual obtained by flow matching inside the anchor-defined residual coordinate system, preserving discrete semantic decisions while restoring multi-modal continuous execution.

Load-bearing premise

A fixed set of roughly one hundred trajectory patterns clustered from training data is assumed to cover the decisions that matter, so refining residuals inside the chosen pattern is enough to produce a safe path.

What would settle it

On held-out rare maneuvers whose nearest training-derived anchors are systematically wrong, measure whether residual flow still recovers a safe closed-loop trajectory; if success collapses relative to methods free to replan the full path, the anchor-as-decision claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Autoregressive work can be limited to a few high-level anchor tokens instead of long waypoint sequences, cutting latency while raising closed-loop reliability.
  • Explicit behavior-level decisions tighten alignment between language instructions and executed trajectories.
  • Flow matching inside an anchor residual space outperforms both deterministic residual regression and full-trajectory flow under the same step budget.
  • The same anchor interface supports query-based or autoregressive decision heads, with the latter better exploiting the language-model token space.
  • Closed-loop task completion benefits more from the structured decision interface than raw aggregate driving score alone.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • When the top anchor is off-route, residual refinement cannot override it; a reflection step that re-scores anchors against language and navigation before residual generation would be a direct safeguard.
  • The discrete-pattern-then-continuous-residual pattern may transfer to other continuous-control VLA settings such as robot manipulation or aerial navigation.
  • Comfort lags some baselines; adding smoothness or comfort terms to the residual objective could close that gap without undoing the success-rate gain.
  • A static 100-anchor codebook is load-bearing; hierarchical or online codebooks would test how far the idea generalizes beyond the training distribution.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. AnchorVLA proposes a hierarchical Vision-Language-Action planner that inserts trajectory-pattern anchors as an explicit interface between high-level VLA reasoning and continuous trajectory generation. Decision-as-Anchor Representation (DAAR) maps multimodal context to a compact codebook of K=100 k-means trajectory patterns (via query-based or autoregressive heads with soft nearest-anchor targets, Eqs. 3–8). Decision-Anchored Residual Flow (DARF) then generates multi-modal continuous residuals in the selected anchor-centered space via flow matching (Eqs. 1–2, 9–13) and ranks candidates by confidence. On the Bench2Drive closed-loop benchmark the method reports Success Rate 77.28 (best among compared methods) and Driving Score 89.92, with ablations on decision modeling, flow formulation, and navigation modality, plus multi-ability scores and qualitative failure analysis.

Significance. If the reported closed-loop gains hold under broader evaluation, the paper offers a useful middle ground between weakly constrained planning heads and long coordinate-token autoregression: behavior-level tokens that keep LLM decision making while residual flow restores continuous flexibility and lower latency (Table 3). The two-stage DAAR/DARF decomposition, soft anchor supervision, matched-anchor flow loss, and decoupled confidence branch are concrete engineering contributions with clear ablations. Strengths include closed-loop evaluation (not open-loop only), multi-ability breakdown, latency comparison against full-trajectory AR, and an honest Appendix A.2 failure analysis. The work is incremental relative to SimLingo/LinkVLA/diffusion planners but addresses a real abstraction gap in VLA driving; significance is primarily empirical and architectural rather than theoretical.

major comments (3)
  1. [§4.1; Eqs. 1–2, 9–13; Table 3; Appendix A.2] Section 4.1 and Eqs. 1–2, 9–13: the central reliability claim depends on a fixed training-only k-means codebook (K=100) defining a complete enough discrete decision space, with DARF refining only inside the chosen residual subspace. Appendix A.2 and Fig. 5 explicitly show that incorrect anchors (off-route or instruction-inconsistent) propagate because residual flow does not override the high-level pattern; the no-decision ablation (Table 3) drops SR from 77.28 to 70.91, confirming dependence on correct selection. There is no sensitivity study on K, no codebook coverage/out-of-cluster analysis, and no closed-loop evaluation of regimes whose needed motion patterns are poorly represented. Without such analysis (or a mitigation such as the reflection mechanism only suggested in A.2), the SOTA SR gain cannot be assessed as robust beyond the training-distribution clusters.
  2. [Table 1; §4.2] Table 1 and §4.2: Success Rate 77.28 and DS 89.92 are reported as single point estimates with no multi-seed runs, error bars, or statistical tests against LinkVLA (SR 74.55) and BridgeDrive (SR 74.99). Given the small absolute margins and known CARLA/Bench2Drive variance, the state-of-the-art SR claim is not yet statistically supported. Please report mean±std over multiple seeds (or at least repeated evaluation) for the main method and key ablations.
  3. [§3.4; Eq. 11–12; Table 4] §3.3–3.4 and Table 4: DARF is motivated as modeling multi-modal residuals under a fixed high-level decision, yet training uses matched-anchor L1 flow loss on only the nearest of the top-M candidates (Eq. 11) plus a softmax confidence loss (Eq. 12). It is unclear whether multi-modality is actually realized at inference (diversity of residuals under one anchor) versus merely multi-hypothesis selection across anchors. Please quantify residual diversity under a fixed anchor and show that flow matching improves over deterministic residual regression for reasons other than the matched-anchor selection path already present in the deterministic baseline.
minor comments (6)
  1. [Table 1; §4.2] Comfort is 28.94, below several strong baselines (Table 1); the brief trade-off remark in §4.2 should be expanded with a smoothness metric or ablation if comfort is a reported metric.
  2. [Table 3] Table 3 lists LinkVLA-AR extra latency 361 ms and autoregressive AnchorVLA 64 ms, but absolute end-to-end latency and hardware conditions are not fully specified; clarify measurement protocol.
  3. [Eq. 4; §4.1] Eq. 4: temperature γ and top-N for soft targets are free parameters; report chosen values and any sensitivity.
  4. [Figure 2; §3.2] Figure 2 caption says “pre-trained Qwen2-0.5B backbone” while §3.2 describes InternVL2-1B with InternViT-300M and Qwen2-0.5B-Instruct; align naming.
  5. [Table 1; References] Related work cites UniAD-Base as [16] in Table 1 while [16] is LinkVLA in the bibliography; fix citation numbering consistency.
  6. [Abstract; Fig. 4] Abstract and introduction claim improved “semantic-action alignment” and “explainability”; these are not measured beyond qualitative CoT visualizations in Fig. 4. Soften or add a simple alignment/explanation metric.

Circularity Check

0 steps flagged

No significant circularity: empirical hierarchical VLA planner with standard supervised targets and external closed-loop evaluation.

full rationale

AnchorVLA is an engineering paper proposing a hierarchical interface (trajectory-pattern anchors via k-means codebook + soft nearest-anchor targets from GT L2 distances in Eqs. 3-4, then residual flow matching in the selected subspace via Eqs. 1-2 and 9-13). Soft targets q_k and residual targets r_k = au_gt - a_k are ordinary supervised constructions from training trajectories; the model is trained to predict them and is then evaluated on the independent Bench2Drive closed-loop CARLA benchmark against external baselines (Table 1). No equation equates the reported SR 77.28 or DS 89.92 to a fitted constant by construction. There is no self-definitional loop, no fitted parameter re-labeled as a prediction of a related quantity, no load-bearing uniqueness theorem or ansatz imported via overlapping-author citation, and no renaming of a known result. Ablations (Tables 3-4) and failure analysis (Appendix A.2) further treat anchor selection as an empirical, fallible component rather than a definitional identity. The derivation chain is therefore self-contained and non-circular.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 2 invented entities

The central performance claim rests on a small set of design choices (codebook size, top-M candidates, residual flow steps, soft-target temperature) and on the modeling assumption that high-level driving decisions are well captured by k-means trajectory patterns. No new physical entities are postulated; the invented constructs are architectural interfaces whose value is measured by closed-loop metrics.

free parameters (4)
  • K (trajectory-pattern codebook size) = 100
    Number of k-means cluster centers used as anchors; fixed at 100 without sensitivity study beyond the reported setting.
  • M (top candidate anchors) = 6
    Number of anchors passed to DARF; fixed at 6.
  • soft-target temperature γ
    Controls the soft nearest-anchor distribution used for DAAR supervision (Eq. 4); value not numerically reported.
  • flow integration steps = 2
    Euler steps used at inference for residual flow; ablations use 2 steps.
axioms (3)
  • domain assumption k-means cluster centers of training trajectories form a discrete space of behavior-level decisions that generalizes to closed-loop test scenarios.
    Invoked in Section 3.3 and 4.1 when constructing the trajectory-pattern codebook A.
  • domain assumption After a high-level anchor is selected, multi-modal continuous refinements can be adequately modeled by flow matching in the residual coordinate system centered at that anchor.
    Core modeling choice of DARF (Section 3.4, Eq. 9–13).
  • ad hoc to paper Soft nearest-anchor targets (Eq. 4) provide better supervision than hard one-hot labels for both query-based and autoregressive DAAR heads.
    Design choice stated in Section 3.3 without external theoretical justification.
invented entities (2)
  • trajectory-pattern anchor (Decision-as-Anchor Representation) no independent evidence
    purpose: Compact discrete token that encodes an entire local motion pattern and serves as the explicit interface between VLA reasoning and continuous generation.
    Defined in Section 3.1–3.3; independent evidence is only the closed-loop gains reported in this paper.
  • Decision-Anchored Residual Flow (DARF) no independent evidence
    purpose: Flow-matching decoder that generates continuous residuals inside the selected anchor subspace and ranks candidates by confidence.
    Introduced in Section 3.4 and Appendix B; value measured solely by the paper’s own ablations and Bench2Drive scores.

pith-pipeline@v1.1.0-grok45 · 18927 in / 2938 out tokens · 28344 ms · 2026-07-12T04:17:09.316124+00:00 · methodology

0 comments
read the original abstract

Autonomous driving planning requires translating navigation intent, traffic rules, dynamic interactions, and language instructions into executable continuous trajectories. Vision-Language-Action models have been introduced into driving planning to improve long-tail generalization, commonsense reasoning, high-level semantic understanding, and explainability. However, existing VLA planners mainly follow planning-head-based trajectory prediction or full-trajectory autoregressive generation. The former only weakly constrains continuous trajectory generation with VLA reasoning, while the latter relies on long sequences of low-information-density coordinate tokens, making semantic-action alignment difficult and leading to discretization errors and inefficient inference. To address these limitations, we propose AnchorVLA, a hierarchical decision-anchored VLA planning framework that uses trajectory-pattern anchors as an explicit interface between high-level VLA reasoning and continuous trajectory execution. Specifically, Decision-as-Anchor Representation represents behavior-level driving decisions with anchor tokens, each encoding an entire local motion pattern rather than a single coordinate point. Decision-Anchored Residual Flow then generates fine-grained continuous trajectories in the selected anchor-defined residual space, capturing multi-modal execution refinements after high-level decision making. By reasoning over compact and semantically meaningful anchors instead of autoregressively generating waypoint sequences, AnchorVLA preserves LLM-based decision making while improving inference efficiency, semantic-action alignment, and continuous generation flexibility. Experiments on the Bench2Drive closed-loop benchmark show that AnchorVLA achieves a state-of-the-art Success Rate of 77.28 and a competitive Driving Score of 89.92.

Figures

Figures reproduced from arXiv: 2607.03182 by Heng Zhang, Hongsong Wang, Lei He, Qi Liu, Yabei Li.

Figure 1
Figure 1. Figure 1: Comparison of autonomous driving planning paradigms. (a) End-to-end planners unify [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed AnchorVLA framework. The model encodes language, vision, [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of Decision-Anchored Residual Flow (DARF). The global trajectory space is decomposed into multiple local trajectory-pattern subspaces, each represented by a trajectory￾pattern anchor ak. Given a selected anchor, the target trajectory is represented as τgt = ak + r, where r is the continuous residual defined by the anchor. DARF performs flow matching in the corresponding anchor-defined residual… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative visualization of AnchorVLA. Each case contains three rows: the CoT-based [PITH_FULL_IMAGE:figures/full_fig_p015_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Failure case analysis of AnchorVLA. We show three representative failure modes: off [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 11 linked inside Pith

  1. [1]

    Vadv2: End-to-end vectorized autonomous driving via probabilistic planning.arXiv preprint arXiv:2402.13243, 2024

    Shaoyu Chen, Bo Jiang, Hao Gao, Bencheng Liao, Qing Xu, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Vadv2: End-to-end vectorized autonomous driving via probabilistic planning.arXiv preprint arXiv:2402.13243, 2024

    Pith/arXiv arXiv 2024

  2. [2]

    Transfuser: Imitation with transformer-based sensor fusion for autonomous driving.IEEE transactions on pattern analysis and machine intelligence, 45(11):12878–12895, 2022

    Kashyap Chitta, Aditya Prakash, Bernhard Jaeger, Zehao Yu, Katrin Renz, and Andreas Geiger. Transfuser: Imitation with transformer-based sensor fusion for autonomous driving.IEEE transactions on pattern analysis and machine intelligence, 45(11):12878–12895, 2022

    2022

  3. [3]

    St-p3: End-to- end vision-based autonomous driving via spatial-temporal feature learning

    Shengchao Hu, Li Chen, Penghao Wu, Hongyang Li, Junchi Yan, and Dacheng Tao. St-p3: End-to- end vision-based autonomous driving via spatial-temporal feature learning. InEuropean Conference on Computer Vision, pages 533–549. Springer, 2022

    2022

  4. [4]

    Planning-oriented autonomous driving

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17853–17862, 2023

    2023

  5. [5]

    Vad: Vectorized scene representation for efficient autonomous driving

    Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene representation for efficient autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 8340–8350, 2023

    2023

  6. [6]

    Hydra-mdp: End-to-end multimodal planning with multi-target hydra-distillation.arXiv preprint arXiv:2406.06978, 2024

    Zhenxin Li, Kailin Li, Shihao Wang, Shiyi Lan, Zhiding Yu, Yishen Ji, Zhiqi Li, Ziyue Zhu, Jan Kautz, Zuxuan Wu, et al. Hydra-mdp: End-to-end multimodal planning with multi-target hydra-distillation.arXiv preprint arXiv:2406.06978, 2024

    Pith/arXiv arXiv 2024

  7. [7]

    Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving

    Bencheng Liao, Shaoyu Chen, Haoran Yin, Bo Jiang, Cheng Wang, Sixu Yan, Xinbang Zhang, Xiangyu Li, Ying Zhang, Qian Zhang, et al. Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 12037–12047, 2025

    2025

  8. [8]

    Sparsedrive: End- to-end autonomous driving via sparse scene representation

    Wenchao Sun, Xuewu Lin, Yining Shi, Chuang Zhang, Haoran Wu, and Sifa Zheng. Sparsedrive: End- to-end autonomous driving via sparse scene representation. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 8795–8801. IEEE, 2025

    2025

  9. [9]

    Para-drive: Parallelized architecture for real-time autonomous driving

    Xinshuo Weng, Boris Ivanovic, Yan Wang, Yue Wang, and Marco Pavone. Para-drive: Parallelized architecture for real-time autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15449–15458, 2024

    2024

  10. [10]

    Goalflow: Goal-driven flow matching for multimodal trajectories generation in end-to-end autonomous driving

    Zebin Xing, Xingyu Zhang, Yang Hu, Bo Jiang, Tong He, Qian Zhang, Xiaoxiao Long, and Wei Yin. Goalflow: Goal-driven flow matching for multimodal trajectories generation in end-to-end autonomous driving. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 1602–1611, 2025

    2025

  11. [11]

    Genad: Generative end-to-end autonomous driving

    Wenzhao Zheng, Ruiqi Song, Xianda Guo, Chenming Zhang, and Long Chen. Genad: Generative end-to-end autonomous driving. InEuropean Conference on Computer Vision, pages 87–104. Springer, 2024

    2024

  12. [12]

    Orion: A holistic end-to-end autonomous driving framework by vision-language instructed action generation

    Haoyu Fu, Diankun Zhang, Zongchuang Zhao, Jianfeng Cui, Dingkang Liang, Chong Zhang, Dingyuan Zhang, Hongwei Xie, Bing Wang, and Xiang Bai. Orion: A holistic end-to-end autonomous driving framework by vision-language instructed action generation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 24823–24834, 2025

    2025

  13. [13]

    Emma: End-to-end multimodal model for autonomous driving

    Jyh-Jing Hwang, Runsheng Xu, Hubert Lin, Wei-Chih Hung, Jingwei Ji, Kristy Choi, Di Huang, Tong He, Paul Covington, Benjamin Sapp, et al. Emma: End-to-end multimodal model for autonomous driving. arXiv preprint arXiv:2410.23262, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Simlingo: Vision-only closed-loop autonomous driving with language-action alignment

    Katrin Renz, Long Chen, Elahe Arani, and Oleg Sinavski. Simlingo: Vision-only closed-loop autonomous driving with language-action alignment. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 11993–12003, 2025

    2025

  15. [15]

    Au- tovla: A vision-language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning.arXiv preprint arXiv:2506.13757, 2025

    Zewei Zhou, Tianhui Cai, Seth Z Zhao, Yun Zhang, Zhiyu Huang, Bolei Zhou, and Jiaqi Ma. Au- tovla: A vision-language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning.arXiv preprint arXiv:2506.13757, 2025

    Pith/arXiv arXiv 2025

  16. [16]

    Unifying language-action understanding and generation for autonomous driving

    Xinyang Wang, Qian Liu, Wenjie Ding, Zhao Yang, Wei Li, Chang Liu, Bailin Li, Kun Zhan, Xianpeng Lang, and Wei Chen. Unifying language-action understanding and generation for autonomous driving. arXiv preprint arXiv:2603.01441, 2026. 11

    arXiv 2026

  17. [17]

    Trajectory-guided con- trol prediction for end-to-end autonomous driving: A simple yet strong baseline.Advances in Neural Information Processing Systems, 35:6119–6132, 2022

    Penghao Wu, Xiaosong Jia, Li Chen, Junchi Yan, Hongyang Li, and Yu Qiao. Trajectory-guided con- trol prediction for end-to-end autonomous driving: A simple yet strong baseline.Advances in Neural Information Processing Systems, 35:6119–6132, 2022

    2022

  18. [18]

    Drivetransformer: Unified transformer for scalable end-to-end autonomous driving.arXiv preprint arXiv:2503.07656, 2025

    Xiaosong Jia, Junqi You, Zhiyuan Zhang, and Junchi Yan. Drivetransformer: Unified transformer for scalable end-to-end autonomous driving.arXiv preprint arXiv:2503.07656, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    Bridgedrive: Diffusion bridge policy for closed-loop trajectory planning in autonomous driving.arXiv preprint arXiv:2509.23589, 2025

    Shu Liu, Wenlin Chen, Weihao Li, Zheng Wang, Lijin Yang, Jianing Huang, Yipin Zhang, Zhongzhan Huang, Ze Cheng, and Hao Yang. Bridgedrive: Diffusion bridge policy for closed-loop trajectory planning in autonomous driving.arXiv preprint arXiv:2509.23589, 2025

    arXiv 2025

  20. [20]

    Drivegpt4: Interpretable end-to-end autonomous driving via large language model

    Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee K Wong, Zhenguo Li, and Hengshuang Zhao. Drivegpt4: Interpretable end-to-end autonomous driving via large language model. IEEE Robotics and Automation Letters, 9(10):8186–8193, 2024

    2024

  21. [21]

    Drivelm: Driving with graph visual question answering

    Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Jens Beißwenger, Ping Luo, Andreas Geiger, and Hongyang Li. Drivelm: Driving with graph visual question answering. In European conference on computer vision, pages 256–274. Springer, 2024

    2024

  22. [22]

    Drivemlm: Aligning multi-modal large language models with behavioral planning states for autonomous driving.arXiv preprint arXiv:2312.09245, 2023

    Wenhai Wang, Jiangwei Xie, ChuanYang Hu, Haoming Zou, Jianan Fan, Wenwen Tong, Yang Wen, Silei Wu, Hanming Deng, Zhiqi Li, et al. Drivemlm: Aligning multi-modal large language models with behavioral planning states for autonomous driving.arXiv preprint arXiv:2312.09245, 2023

    arXiv 2023

  23. [23]

    Opendrivevla: Towards end-to-end autonomous driving with large vision language action model

    Xingcheng Zhou, Xuyuan Han, Feng Yang, Yunpu Ma, V olker Tresp, and Alois Knoll. Opendrivevla: Towards end-to-end autonomous driving with large vision language action model. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 13782–13790, 2026

    2026

  24. [24]

    Drivemoe: Mixture-of-experts for vision-language-action model in end-to-end autonomous driving

    Zhenjie Yang, Yilin Chai, Xiaosong Jia, Qifeng Li, Yuqian Shao, Xuekai Zhu, Haisheng Su, and Junchi Yan. Drivemoe: Mixture-of-experts for vision-language-action model in end-to-end autonomous driving. arXiv preprint arXiv:2505.16278, 2025

    Pith/arXiv arXiv 2025

  25. [25]

    Recogdrive: A reinforced cognitive framework for end-to-end autonomous driving.arXiv preprint arXiv:2506.08052, 2025

    Yongkang Li, Kaixin Xiong, Xiangyu Guo, Fang Li, Sixu Yan, Gangwei Xu, Lijun Zhou, Long Chen, Haiyang Sun, Bing Wang, et al. Recogdrive: A reinforced cognitive framework for end-to-end autonomous driving.arXiv preprint arXiv:2506.08052, 2025

    Pith/arXiv arXiv 2025

  26. [26]

    Drivevla-w0: World models amplify data scaling law in autonomous driving.arXiv preprint arXiv:2510.12796, 2025

    Yingyan Li, Shuyao Shang, Weisong Liu, Bing Zhan, Haochen Wang, Yuqi Wang, Yuntao Chen, Xiaoman Wang, Yasong An, Chufeng Tang, et al. Drivevla-w0: World models amplify data scaling law in autonomous driving.arXiv preprint arXiv:2510.12796, 2025

    Pith/arXiv arXiv 2025

  27. [27]

    Diffvla: Vision-language guided diffusion planning for autonomous driving

    Anqing Jiang, Yu Gao, Zhigang Sun, Yiru Wang, Jijun Wang, Jinghao Chai, Qian Cao, Yuweng Heng, Hao Jiang, Yunda Dong, et al. Diffvla: Vision-language guided diffusion planning for autonomous driving. arXiv preprint arXiv:2505.19381, 2025

    Pith/arXiv arXiv 2025

  28. [28]

    Mini-internvl: a flexible-transfer pocket multi-modal model with 5% parameters and 90% performance.Visual Intelligence, 2(1):32, 2024

    Zhangwei Gao, Zhe Chen, Erfei Cui, Yiming Ren, Weiyun Wang, Jinguo Zhu, Hao Tian, Shenglong Ye, Junjun He, Xizhou Zhu, et al. Mini-internvl: a flexible-transfer pocket multi-modal model with 5% parameters and 90% performance.Visual Intelligence, 2(1):32, 2024

    2024

  29. [29]

    Qwen2 technical report, 2024

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei...

    Pith/arXiv arXiv 2024

  30. [30]

    Think twice before driving: Towards scalable decoders for end-to-end autonomous driving

    Xiaosong Jia, Penghao Wu, Li Chen, Jiangwei Xie, Conghui He, Junchi Yan, and Hongyang Li. Think twice before driving: Towards scalable decoders for end-to-end autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21983–21994, 2023

    2023

  31. [31]

    Driveadapter: Breaking the coupling barrier of perception and planning in end-to-end autonomous driving

    Xiaosong Jia, Yulu Gao, Li Chen, Junchi Yan, Patrick Langechuan Liu, and Hongyang Li. Driveadapter: Breaking the coupling barrier of perception and planning in end-to-end autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 7953–7963, 2023

    2023

  32. [32]

    Rethinking the open-loop evaluation of end-to-end autonomous driving in nuscenes

    Jiang-Tian Zhai, Ze Feng, Jinhao Du, Yongqiang Mao, Jiang-Jiang Liu, Zichang Tan, Yifu Zhang, Xiaoqing Ye, and Jingdong Wang. Rethinking the open-loop evaluation of end-to-end autonomous driving in nuscenes. arXiv preprint arXiv:2305.10430, 2023. 12

    Pith/arXiv arXiv 2023

  33. [33]

    Bench2drive: Towards multi- ability benchmarking of closed-loop end-to-end autonomous driving.Advances in Neural Information Processing Systems, 37:819–844, 2024

    Xiaosong Jia, Zhenjie Yang, Qifeng Li, Zhiyuan Zhang, and Junchi Yan. Bench2drive: Towards multi- ability benchmarking of closed-loop end-to-end autonomous driving.Advances in Neural Information Processing Systems, 37:819–844, 2024. 13 A Visualization Analysis In this section, we provide qualitative visualization and failure case analysis to better unders...

    2024