Pith. sign in

REVIEW 3 major objections 5 minor 33 references

Box-QAymo: Box-Referring VQA Dataset for Autonomous Driving

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Box-QAymo grounds driving questions in user-drawn boxes and shows current VLMs score 66.1% F1 on binary checks, 18.3% on attributes, and 37.6% on motion.

desk verdict Useful box-referring driving VQA benchmark, but motion labels partly rely on 3D-track interpolation and need visual-grounding validation before the motion scores are taken at face value. read the letter →

arxiv 2507.00525 v1 pith:A5BFKJ2D submitted 2025-07-01 cs.CV cs.AI

Djamahl Etchegaray , Yuxia Fu , Zi Huang , Yadan Luo This is my paper
classification cs.CVcs.AI
keywords visualquestionansweringautonomousdrivingbox-referringspatiotemporalreasoningbenchmarkvision-languagemodelsobjectgroundingdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Box-QAymo is a box-referring visual question answering dataset for autonomous driving, built on Waymo scenes. Its central claim is that grounding a question in a user-drawn bounding box, the way a passenger points at something, is a natural and necessary format for testing whether vision-language models actually perceive what a specific person is asking about. The paper argues that existing driving VQA benchmarks only test full-scene descriptions or planning, and therefore miss failures in localized, intent-driven perception. On this benchmark, three VLMs score 66.1% F1 on binary sanity checks, 18.3% on attributes, and 37.6% on motion questions, and even a driving-specific model often fails to produce usable answers. If the benchmark is right, progress on human-vehicle communication is being overestimated by scene-level evaluations.

What carries the argument

The central mechanism is the box-referring question format: a red bounding box overlaid on a camera frame, paired with a question about the object inside that box (e.g., 'What color is the object highlighted in red?'). This turns a user's pointing gesture into a visual prompt that the VLM must attend to, letting the benchmark test whether a model can separate a referred instance from scene-level statistics. Around this, the dataset builds a three-level protocol, binary sanity checks, attribute prediction, and motion reasoning, with motion answers derived from Waymo 3D tracks, negative sampling for balance, and temporal consistency checks.

What would settle it

Give a held-out set of motion questions to a VLM while removing or blurring the target object's trajectory cues in the frames; if accuracy barely drops, the questions are answerable without temporal visual information and the benchmark is not measuring what it claims.

Watch

Extended reading notes

Core claim

The paper's central claim is that Box-QAymo is the first driving VQA benchmark that answers questions about a specific object the user has highlighted with a bounding box, and that this format exposes a hierarchy of capabilities in current VLMs. The benchmark moves from binary sanity questions (movement status, orientation) to attribute prediction (fine-grained class, color, facing direction) to motion reasoning (speed, movement direction, trajectory, path conflict, relative motion). Across LLaVA-1.5, Qwen-VL, and the driving-specific Senna, mean F1 falls from 66.1% on binary, to 18.3% on attributes, to 37.6% on motion; the driving-specialized model outputs parseable answers only 34% of the time and scores zero on all attribute tasks. Finetuning with LoRA lifts performance, but providing two consecutive frames instead of one consistently hurts motion reasoning, suggesting current VLMs do not truly integrate short-term temporal information.

Load-bearing premise

Motion answers are derived from 3D tracks, so the benchmark's validity rests on the assumption that the speed and trajectory questions can be answered from the supplied images alone, rather than requiring hidden track data.

Editorial extensions

If this is right

  • Current general-purpose VLMs are unreliable for localized, user-driven perception in driving, and finetuning on the dataset partially closes the gap.
  • Providing two frames instead of one does not help current VLMs understand motion, so the bottleneck appears to be temporal integration rather than input length.
  • A driving-specialized model trained mainly for planning can score zero on structured attribute questions, meaning narrow domain training does not transfer to perception communication.
  • Box-referring questions are a workable interface: Qwen-VL consistently improves when the red box is present, though the gain is architecture-dependent.
  • Hierarchical evaluation separates basic scene understanding from fine-grained and temporal reasoning, allowing future benchmarks to measure progress at each level.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the motion answers are not visually inferable from the images alone, part of the reported motion gap may measure access to privileged 3D-track information rather than visual reasoning; a human-perception study on the same frames would settle this.
  • The red-box grounding result hints that overlaying a user-specified region could improve deployed systems at little cost, but the effect may not generalize to models with weaker visual encoders.
  • The two-frame degradation points to a training-data mismatch, suggesting that pretraining VLMs on short frame pairs rather than single images or long videos could directly address the temporal reasoning gap.
  • A natural extension is to ask the same box-referring questions with the target object occluded or removed, turning the benchmark into a test of whether models can flag uncertainty when they cannot see the referred object.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. Box-QAymo is a box-referring VQA dataset and benchmark built from Waymo Open Dataset scenes. The authors crowd-source fine-grained object classes and visual attributes, generate a hierarchy of QA pairs (binary sanity checks, attribute prediction, and motion/spatio-temporal reasoning), and evaluate LLaVA-1.5, Qwen-VL, and Senna in zero-shot and LoRA-finetuned settings. The main reported results are average F1 of 66.1% on binary, 18.3% on attribute, and 37.6% on motion questions, with finetuning improving most tasks and two-frame inputs degrading performance.

Significance. The contribution is timely and potentially useful: a public dataset that grounds driving VQA in user-drawn boxes would fill a real gap, and the hierarchical protocol plus difficulty balancing are sensible design choices. The paper's strengths include the crowdsourced semantic expansion beyond Waymo's coarse labels, explicit negative sampling, temporal-consistency checks, and the observation that current VLMs often fail on localized, intent-driven queries. However, the benchmark's central interpretation—that motion scores measure visual spatio-temporal reasoning—depends on the assumption that trajectory-derived answers are inferable from the RGB frames shown to the model; this assumption is not currently verified.

major comments (3)
  1. [3.2.3 and 3.3] The motion question answers are derived from Waymo 3D tracks, including linear interpolation of object positions and smooth velocity calculations using temporal derivatives. The paper interprets low motion F1 (e.g., 37.6% average, Table 4) as a deficit in visual spatio-temporal reasoning, but this interpretation requires that the ground-truth answers be recoverable from the supplied camera frames. For single-frame implicit questions about speed or relative motion, distant objects with sub-pixel displacement over 100 ms may not provide any visual evidence; for interpolated trajectories, objects may be occluded or outside the camera frustum at the interpolated position. The authors should provide a human-visual oracle or a per-question answerability audit demonstrating that the labels can be inferred from the exact inputs given to the models, and should either remove or re-flag questions that fail this audit. Without this, the reported motion deficits could partly measure privileged-metadata access rather than visual reasoning.
  2. [3.1] The crowdsourced semantic labeling procedure is described at a high level, but no inter-annotator agreement statistics are reported. Since the attribute questions (Table 3) are evaluated against these crowd labels, annotation noise is conflated with model error. The authors should report per-task agreement metrics such as Cohen's kappa or Fleiss' kappa, the number of annotators per item, and how disagreements were adjudicated.
  3. [4.2, Tables 2-4] All F1, precision, and recall numbers are reported as point estimates with no confidence intervals or significance tests. Given the relatively small validation split (101 scenes) and uneven question counts across subcategories, the performance gaps that drive the paper's conclusions—e.g., binary vs. attribute vs. motion, and finetuning gains such as LLaVA† Trajectory Analysis 98.90 vs. Qwen-VL 17.03—should be accompanied by bootstrap confidence intervals or a significance test. The authors should also report the exact number of questions per subcategory in Tables 2-4.
minor comments (5)
  1. [4.1] The evaluated model is listed as 'Qwen-7B' but the citation [4] is the Qwen2.5-VL technical report; please clarify the exact model name and version used in the experiments.
  2. [Table 3] Senna is reported with 0.00 F1 on all attribute tasks, but Section 4.3 notes that Senna produces only 34% valid responses overall. Please clarify whether these zeros are entirely due to invalid response parsing or also reflect incorrect answers, since this affects how the attribute failures should be interpreted.
  3. [3.3] The negative sampling description is difficult to follow: 'We find samples that do not fit the criteria for the ground truth answer and create additional samples from these criteria.' Please rephrase to specify how negative examples are constructed and how their labels are verified.
  4. [3.1] The paper states that approximately 50% of objects are labeled and that objects with poor visibility are skipped, but it does not explain how this 50% was selected or how 'poor visibility' was operationalized. This matters for potential selection bias in the attribute and motion splits.
  5. [Table 5] The 'Improvement (%)' column reports negative values for decreases (e.g., -5.42 for LLaVA two-frame), which is clearer if labeled 'Change (%)' or 'Difference (%)'. Also, the 100-ms interval between frames may be too short to produce visible motion in many driving scenes; a brief discussion of this limitation would help interpret the two-frame results.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Box-QAymo is an external benchmark, and the reported F1 scores are empirical measurements rather than quantities forced by the dataset construction.

full rationale

The paper makes no fitted-parameter derivation and no claim that reduces to its own inputs. The central contribution is a dataset and a benchmark protocol: bounding-box referring VQA pairs are constructed from Waymo annotations, crowd-sourced fine-grained labels, and trajectory rules, and model F1 scores are then measured on held-out validation splits. No quantity is fit to the benchmark and then renamed as a prediction, and no load-bearing argument is justified by a self-citation chain. The reader's skeptical concern about motion questions is a benchmark-validity issue, not circularity: generating ground-truth answers from 3D tracks with linear interpolation, as described in Section 3.3, may make some labels unrecoverable from the supplied camera images, which would mean low F1 partially reflects privileged-metadata access rather than visual reasoning. That is a substantive construct-validity threat, but it is not a circular reduction, because the paper does not use the model scores to define the labels or derive the ground truth from the models. There is also no evidence of the authors' own prior work being invoked as the sole support for the central premise. The derivation chain is therefore self-contained with respect to circularity, so the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. Its load-bearing assumptions concern the accuracy and visual answerability of automatically generated ground truth, the reliability of crowdsourced labels, and the effectiveness of red-box visual grounding.

assumptions (4)
  • domain assumption Waymo Open Dataset 3D annotations and interpolated object trajectories provide accurate ground truth for object position, speed, and relative motion used to generate QA answers.
    Section 3.2.3 uses these trajectories to construct motion questions. If the interpolated tracks are noisy or not visually verifiable, the benchmark's motion ground truth is unreliable.
  • domain assumption Crowd-sourced annotators following the Argoverse 2.0 taxonomy produce reliable fine-grained class and color labels for about 50% of objects.
    Section 3.1 uses these labels for attribute questions. No inter-annotator agreement is reported, so label consistency is assumed rather than demonstrated.
  • domain assumption A red bounding box drawn around an object is an effective visual reference that VLMs can exploit, per prior work with red circles.
    Section 3.2.2 motivates box grounding with reference [23]. Table 6 shows gains are small and inconsistent, so the premise is only weakly supported by the paper's own results.
  • domain assumption Splitting the Waymo validation set into 101-scene finetuning and validation splits provides independent evaluation without leakage.
    Section 3.1 states this split. The paper does not report scene-level cross-checking, so temporal or scene overlap between splits is not verified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Box-QAymo: Box-Referring VQA Dataset for Autonomous Driving." pith.science (2026). https://pith.science/paper/A5BFKJ2D

@misc{pith2026250700525,
  author       = {Pith},
  title        = {Pith review of: Box-QAymo: Box-Referring VQA Dataset for Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A5BFKJ2D}},
  note         = {Machine review of arXiv:2507.00525}
}
read the original abstract

Interpretable communication is essential for safe and trustworthy autonomous driving, yet current vision-language models (VLMs) often operate under idealized assumptions and struggle to capture user intent in real-world scenarios. Existing driving-oriented VQA datasets are limited to full-scene descriptions or waypoint prediction, preventing the assessment of whether VLMs can respond to localized user-driven queries. We introduce Box-QAymo, a box-referring dataset and benchmark designed to both evaluate and finetune VLMs on spatial and temporal reasoning over user-specified objects. Users express intent by drawing bounding boxes, offering a fast and intuitive interface for focused queries in complex scenes. Specifically, we propose a hierarchical evaluation protocol that begins with binary sanity-check questions to assess basic model capacities, and progresses to (1) attribute prediction for box-referred objects, (2) motion understanding of target instances, and (3) spatiotemporal motion reasoning over inter-object dynamics across frames. To support this, we crowd-sourced fine-grained object classes and visual attributes that reflect the complexity drivers encounter, and extract object trajectories to construct temporally grounded QA pairs. Rigorous quality control through negative sampling, temporal consistency checks, and difficulty-aware balancing guarantee dataset robustness and diversity. Our comprehensive evaluation reveals significant limitations in current VLMs when queried about perception questions, highlighting the gap in achieving real-world performance. This work provides a foundation for developing more robust and interpretable autonomous driving systems that can communicate effectively with users under real-world conditions. Project page and dataset are available at https://djamahl99.github.io/qaymo-pages/.

Figures

Figures reproduced from arXiv: 2507.00525 by the authors.

Figure 1
Figure 1. Overview of the Box-QAymo dataset pipeline for evaluating vision-language models (VLMs). Step 1 [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Question distribution across semantic cate [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Overall F1 performance across question types [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Example of a path conflict detection question [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 24 canonical work pages

  1. [1]

    Fear of Self-Driving Cars on the Rise

    AAA. Fear of Self-Driving Cars on the Rise. https://newsroom.aaa.com/2023/03/ aaa-fear-of-self-driving-cars-on-the-rise/ ,

    work page 2023

  2. [2]

    Drivers Still Wary of Self-Driving Cars

    AAA. Drivers Still Wary of Self-Driving Cars. https://info.oregon.aaa.com/ aaa-drivers-still-wary-of-self-driving-cars/ ,

    work page

  3. [3]

    Summary report: standing general order on crash reporting for level 2 advanced driver assistance systems

    National Highway Traffic Safety Administration et al. Summary report: standing general order on crash reporting for level 2 advanced driver assistance systems. US Department of Trans- port: Washington, DC, USA , 2022

    work page 2022

  4. [4]

    Qwen2.5- vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Ming-Hsuan Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5- vl technical repor...

    arXiv 2025

  5. [5]

    Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom

    Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seat- tle, WA, USA, June 13-19, 2020 , pages 11618– 11628. Computer V...

    work page 2020

  6. [6]

    Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom

    Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceed- ings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR) , June 2020

    work page 2020

  7. [7]

    Driv- ing with llms: Fusing object-level vector modal- ity for explainable autonomous driving

    Long Chen, Oleg Sinavski, Jan H¨ unermann, Al- ice Karnsund, Andrew James Willmott, Danny Birch, Daniel Maund, and Jamie Shotton. Driv- ing with llms: Fusing object-level vector modal- ity for explainable autonomous driving. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 14093–14100. IEEE, 2024

    work page 2024

  8. [8]

    Panchal, and Ziran Wang

    Can Cui, Zichong Yang, Yupeng Zhou, Yun- sheng Ma, Juanwu Lu, Lingxi Li, Yaobin Chen, Jitesh H. Panchal, and Ziran Wang. Person- alized autonomous driving with large language models: Field experiments. In 27th IEEE International Conference on Intelligent Trans- portation Systems, ITSC 2024, Edmonton, AB, Canada, September 24-27, 2024 , pages 20–27. IEEE, 2024

    work page 2024

Show all 33 references
  1. [9]

    Talk2car: Taking control of your self- driving car

    Thierry Deruyttere, Simon Vandenhende, Du- san Grujicic, Luc Van Gool, and Marie-Francine Moens. Talk2car: Taking control of your self- driving car. arXiv preprint arXiv:1909.10838 , 2019. 8

    1909 arXiv

  2. [10]

    Are we ready for autonomous driving? the KITTI vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urta- sun. Are we ready for autonomous driving? the KITTI vision benchmark suite. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, June 16-21, 2012, pages 3354–3361. IEEE Computer Soci- ety, 2012

    2012

  3. [11]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. Lora: Low-rank adap- tation of large language models. CoRR, abs/2106.09685, 2021

    2021 arXiv

  4. [12]

    Senna: Bridg- ing large vision-language models and end-to-end autonomous driving

    Bo Jiang, Shaoyu Chen, Bencheng Liao, Xingyu Zhang, Wei Yin, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Senna: Bridg- ing large vision-language models and end-to-end autonomous driving. CoRR, abs/2410.22313, 2024

    2024 arXiv

  5. [13]

    Grounding human-to-vehicle advice for self-driving vehi- cles

    Jinkyu Kim, Teruhisa Misu, Yi-Ting Chen, Ashish Tawari, and John Canny. Grounding human-to-vehicle advice for self-driving vehi- cles. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recogni- tion, pages 10591–10599, 2019

    2019

  6. [14]

    Textual expla- nations for self-driving vehicles

    Jinkyu Kim, Anna Rohrbach, Trevor Darrell, John Canny, and Zeynep Akata. Textual expla- nations for self-driving vehicles. In Proceedings of the European conference on computer vision (ECCV), pages 563–578, 2018

    2018

  7. [15]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Pro- cessing Systems 36: Annual Conference on Neural Informat...

    2023

  8. [16]

    Lampilot: An open benchmark dataset for au- tonomous driving with language model pro- grams

    Yunsheng Ma, Can Cui, Xu Cao, Wenqian Ye, Peiran Liu, Juanwu Lu, Amr Abdelraouf, Ro- hit Gupta, Kyungtae Han, Aniket Bera, et al. Lampilot: An open benchmark dataset for au- tonomous driving with language model pro- grams. In Proceedings of the IEEE/CVF Con- ference on Compute...

    2024

  9. [17]

    Drama: Joint risk localization and captioning in driving

    Srikanth Malla, Chiho Choi, Isht Dwivedi, Joon Hee Choi, and Jiachen Li. Drama: Joint risk localization and captioning in driving. In Proceedings of the IEEE/CVF winter confer- ence on applications of computer vision , pages 1043–1052, 2023

    2023

  10. [18]

    Lingoqa: Visual question answering for autonomous driving

    Ana-Maria Marcu, Long Chen, Jan H¨ unermann, Alice Karnsund, Beno ˆ ıt Han- otte, Prajwal Chidananda, Saurabh Nair, Vijay Badrinarayanan, Alex Kendall, Jamie Shotton, Elahe Arani, and Oleg Sinavski. Lingoqa: Visual question answering for autonomous driving. In Ales Leonardis, ...

    2024

  11. [19]

    Lingoqa: Video question answering for autonomous driving

    Ana-Maria Marcu, Long Chen, Jan H¨ unermann, Alice Karnsund, Beno ˆ ıt Han- otte, Prajwal Chidananda, Saurabh Nair, Vijay Badrinarayanan, Alex Kendall, Jamie Shotton, and Oleg Sinavski. Lingoqa: Video question answering for autonomous driving. CoRR, abs/2312.14115, 2023

    2023 arXiv

  12. [20]

    Reason2drive: Towards interpretable and chain-based reasoning for autonomous driv- ing

    Ming Nie, Renyuan Peng, Chunwei Wang, Xinyue Cai, Jianhua Han, Hang Xu, and Li Zhang. Reason2drive: Towards interpretable and chain-based reasoning for autonomous driv- ing. In European Conference on Computer Vi- sion, pages 292–308. Springer, 2024

    2024

  13. [21]

    Nuscenes-qa: A multi-modal visual question answering bench- mark for autonomous driving scenario

    Tianwen Qian, Jingjing Chen, Linhai Zhuo, Yang Jiao, and Yu-Gang Jiang. Nuscenes-qa: A multi-modal visual question answering bench- mark for autonomous driving scenario. In Pro- ceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 4542–4550, 2024

    2024

  14. [22]

    Rank2tell: A multimodal driving dataset for joint importance ranking and reasoning

    Enna Sachdeva, Nakul Agarwal, Suhas Chundi, Sean Roelofs, Jiachen Li, Mykel Kochenderfer, Chiho Choi, and Behzad Dariush. Rank2tell: A multimodal driving dataset for joint importance ranking and reasoning. In Proceedings of the IEEE/CVF winter conference on applications of com...

    2024

  15. [23]

    What does clip know about a red circle? visual prompt engineer- ing for vlms

    Aleksandar Shtedritski, Christian Rupprecht, and Andrea Vedaldi. What does clip know about a red circle? visual prompt engineer- ing for vlms. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 11987–11997, October 2023

    2023

  16. [24]

    Driv- elm: Driving with graph visual question answer- ing

    Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Ping Luo, Andreas Geiger, and Hongyang Li. Driv- elm: Driving with graph visual question answer- ing. CoRR, abs/2312.14150, 2023

    2023 arXiv

  17. [25]

    Scal- ability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Doti- walla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Tim- ofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zh...

    2020

  18. [26]

    Drivevlm: The convergence of autonomous driving and large vision-language models

    Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, and Hang Zhao. Drivevlm: The convergence of autonomous driving and large vision-language models. In Pulkit Agrawal, Oliver Kroemer, and Wolfram Burgard, editors, Conferen...

    2024

  19. [27]

    ´Alvarez

    Shihao Wang, Zhiding Yu, Xiaohui Jiang, Shiyi Lan, Min Shi, Nadine Chang, Jan Kautz, Ying Li, and Jos´ e M. ´Alvarez. Omnidrive: A holistic vision-language dataset for autonomous driving with counterfactual reasoning. CoRR, abs/2504.04348, 2025

    2025 arXiv

  20. [28]

    Drivemlm: Aligning multi-modal large language models with behavioral planning states for autonomous driving

    Wenhai Wang, Jiangwei Xie, ChuanYang Hu, Haoming Zou, Jianan Fan, Wenwen Tong, Yang Wen, Silei Wu, Hanming Deng, Zhiqi Li, et al. Drivemlm: Aligning multi-modal large language models with behavioral planning states for autonomous driving. arXiv preprint arXiv:2312.09245, 2023

    2023

  21. [29]

    Ar- goverse 2: Next generation datasets for self- driving perception and forecasting

    Benjamin Wilson, William Qi, Tanmay Agar- wal, John Lambert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, An- drew Hartnett, Jhony Kaesemodel Pontes, Deva Ramanan, Peter Carr, and James Hays. Ar- goverse 2: Next generation datasets for self- driving perception...

    2023 arXiv

  22. [30]

    Language prompt for autonomous driving

    Dongming Wu, Wencheng Han, Yingfei Liu, Tiancai Wang, Cheng-zhong Xu, Xiangyu Zhang, and Jianbing Shen. Language prompt for autonomous driving. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 39, pages 8359–8367, 2025

    2025

  23. [31]

    Explainable object-induced action decision for autonomous vehicles

    Yiran Xu, Xiaoyin Yang, Lihang Gong, Hsuan- Chu Lin, Tz-Ying Wu, Yunsheng Li, and Nuno Vasconcelos. Explainable object-induced action decision for autonomous vehicles. In Proceed- ings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 9523–9532, 2020

    2020

  24. [32]

    Wong, Zhen- guo Li, and Hengshuang Zhao

    Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee K. Wong, Zhen- guo Li, and Hengshuang Zhao. Drivegpt4: In- terpretable end-to-end autonomous driving via large language model. IEEE Robotics Autom. Lett., 9(10):8186–8193, 2024. 10

    2024

  25. [2025]

    Accessed: 2025-05-26

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.