Pith. sign in

REVIEW 4 major objections 8 minor 34 references

Under real fog, rain, snow, and night, driving VLMs fail first at knowing what they can see, then at space and safe paths.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 19:45 UTC pith:NKCNXQX6

load-bearing objection Useful real multi-modal adverse-weather VLM benchmark with a clear observability→spatial→decision frame; ranking strength is tempered by teacher-model labels, but the core degradation story still holds. the 4 major comments →

arxiv 2607.23537 v1 pith:NKCNXQX6 submitted 2026-07-26 cs.AI

ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness

Qiao Yan , Yihan Wang , Zhenghao Xing , Jiaqi Xu , Pheng-Ann Heng This is my paper
classification cs.AI
keywords adverse weatherobservability awarenessmultimodal VLMsautonomous driving benchmarkspatial reliabilityrisk-aware decision-makingcamera-LiDAR-radarreinforcement learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that the hard problem for vision-language models in bad weather is not just noisy sensors, but degraded environmental observability: camera, LiDAR, and radar become unreliable and disagree with one another. Existing driving benchmarks mostly stay in clear weather, use synthetic corruptions, or skip the intermediate steps between perception and action, so they cannot show where understanding breaks. ObsDriveBench is a real multi-modal benchmark built on synchronized camera, LiDAR, and radar with object-level observability labels and more than 14k training and 13k test multiple-choice questions along three linked capabilities: knowing what is observable, keeping spatial understanding reliable, and making risk-aware decisions. Across many models the same pattern appears—observability is the weakest link, spatial grounding stays limited, and models often pick cautious fallbacks while still misjudging where risk is and which trajectories are safe. A two-stage model trained with normal-weather supervised fine-tuning then adverse-weather reinforcement learning raises scores on all three dimensions, supporting the claim that explicit observability-aware training helps.

Core claim

Under real adverse-weather multi-modal inputs, current vision-language models show a systematic gap along an observability → spatial reliability → risk-aware decision chain: they struggle most to estimate what each sensor can reliably see and when sensors agree, maintain only limited spatial consistency under partial views, and often choose conservative actions while still failing risk localization and trajectory safety checks. ObsDriveBench makes that chain measurable, and normal-weather SFT plus adverse-weather RL improves all three capabilities.

What carries the argument

ObsDriveBench’s three-dimension task chain—observability awareness, spatial reliability, and risk-aware decision-making—built from object-level observability meta-annotations (fully / partially / unobservable per camera, LiDAR, radar), scene descriptions, and capability-oriented multiple-choice questions over synchronized multi-modal inputs.

Load-bearing premise

The object-level observability labels and multiple-choice answers, first drafted by large models then human-checked, are accurate and independent enough to fairly rank the same class of models and to stand in for how real failures would propagate into driving decisions.

What would settle it

If strong models scored high and evenly on modality-specific observability, cross-modal comparison, 2D–3D grounding, risk identification, and trajectory safety on the held-out adverse-weather split—or if normal-weather SFT plus adverse-weather RL did not improve those scores relative to the untuned baseline—the claimed bottleneck chain and training fix would not hold.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Benchmarking only final trajectories will miss whether failure started in observability, spatial grounding, or risk assessment.
  • Training that never forces models to state what is unobservable will leave the primary bottleneck intact.
  • Conservative fallback behavior can look safe while risk localization and trajectory evaluation remain weak.
  • Multi-modal BEV inputs help some spatial and decision scores but do not by themselves fix observability estimation.
  • Released training annotations enable future driving VLMs to be supervised explicitly for observability under real weather.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Regulators and fleet tests that only score end trajectories may certify systems that still cannot say which sensor to trust in fog or snow.
  • The same observability-first chain could transfer to other multi-sensor robots (drones, marine, warehouse) where weather or dust breaks cross-modal agreement.
  • Closed-loop simulators that inject the benchmark’s observability labels into planning rewards would test whether MCQ gains become safer control.
  • If label generation still leans on the same VLM family being ranked, a fully human-only observability re-annotation subset would be the cleanest stress test of ranking validity.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper introduces ObsDriveBench, a multiple-choice benchmark for vision-language models in autonomous driving under real adverse weather, built on K-Radar sequences with synchronized camera, LiDAR, and radar inputs (LiDAR/radar rendered as BEV images). Tasks are organized along three capability dimensions — observability awareness, spatial reliability, and risk-aware decision-making — with ground truth produced by a pipeline in which InternVL3-38B and Qwen3VL-32B draft labels/questions, Gemini-3.1-Pro refines them, and PhD-level annotators verify them (reported pass rates 99.96%/97.20%/99.96%). Evaluation of 14 off-the-shelf models plus two trained variants (ObsDrive-SFT, ObsDrive-RL) shows: (i) observability awareness is the weakest dimension, (ii) spatial reliability is limited, (iii) models select conservative fallback actions (>90%) while remaining near-chance on risk identification and trajectory evaluation, and (iv) the authors' normal-weather SFT + adverse-weather GRPO training improves all three dimensions, with limited cross-dataset transfer checks on DriveBench and DriveLMM-o1.

Significance. If the ground truth holds up, this is a useful and well-scoped contribution: it is the first driving VLM benchmark built on real adverse-weather camera+LiDAR+radar data (K-Radar), it formalizes an object-level, per-modality observability annotation (O_m(o_i) in {0,1,2}) that other benchmarks lack, it evaluates 14 off-the-shelf systems with modality- and weather-level breakdowns (Tables 1–3), and it ships a train/test split that respects the official K-Radar sequence split rather than random frame splitting. The cross-dataset checks on DriveBench and DriveLMM-o1 (Tables 4–5) and the input-modality ablation (Table 3) are welcome, if modest. The observability-aware framing — diagnosing where along the perception→spatial→decision chain failures originate — is genuinely more informative than trajectory-only metrics, and the authors appropriately disclaim closed-loop validity (§2.2). The main risk to significance is not the design but the provenance of the answer key, which is addressable.

major comments (4)
  1. [§3.2 (Data Preparation); Table 1; §5.2.1] The headline claims (observability awareness is the primary bottleneck; the overall model ranking in Table 1) rest on an answer key drafted by InternVL3-38B and Qwen3VL-32B and refined by Gemini-3.1-Pro (§3.2, Stage 1/Stage 3) — the same model family being ranked. Gemini-3.1-Pro is the top off-the-shelf model in Table 1 (88.04 spatial AVG, 80.22 risk AVG) and Qwen3VL-32B is the top open model (79.71 risk AVG). Human verification mitigates this but does not eliminate it, because annotators were shown the generated MCQs and asked to correct errors: the reported pass rates (99.96%/97.20%/99.96%) measure agreement with VLM-drafted content, not independent derivation, and correction-on-review systematically under-detects plausible-but-wrong keys. A concrete, bounded fix: (i) have annotators independently answer a random subset (e.g., 300–500 test MCQs, stratified across the nine sub-tasks) wi
  2. [§5.2.1, finding (3); §3.3 task family (3b)] The 'apparent paradox' (models adopt conservative fallback actions yet fail risk identification and trajectory evaluation) is partly an artifact of task design. The fallback-action template (Fig. 2) offers options of the form 'slow down and proceed with caution' vs. 'maintain speed' vs. 'speed up to pass quickly' — the safe option is identifiable from the question text alone, without any scene understanding. Scores >90% for nearly all models (Table 1) are therefore a template prior, not measured capability, and the contrast with risk identification (~50–68%) conflates task difficulty asymmetry with a capability dissociation. The claim 'models know they should be cautious, but do not fully understand what exactly is risky' would be much better supported if the fallback task included plausible conservative distractors (e.g., 'change lanes to overtake the slowing vehicle' vs. 'brake to a st
  3. [Table 1, Traj. Eval. column; §3.3 (3c)] Trajectory evaluation is a binary yes/no task (chance = 50%). Most off-the-shelf models score 49–66%, i.e., near or modestly above chance, while ObsDrive-SFT/RL reach 98.09/99.00. A near-perfect score on a task where all 14 evaluated models sit near chance suggests the feasible/counterfactual trajectory pairs may carry systematic surface cues (e.g., counterfactuals drawn from a different distribution of curvature, length, or endpoint offset) that RL on the training split can exploit without genuine scene reasoning. The authors should (i) report the base rate of safe vs. unsafe labels in train and test, (ii) run a text-only or geometry-only control on the trajectory strings to measure how much of the gap is visual, and (iii) describe the counterfactual generation procedure (currently deferred to the supplement). Without this, the largest single-cell improvement in Table 1 is also the leas
  4. [§5.4, Tables 4 and 5] The transfer claims are stated strongly ('improvements transfer across scenes and corruption patterns', §5.4) but the effect sizes are small: Table 4 shows gains of roughly 0.1–1.5 points across 20 settings, and Table 5 gains of 0.93–2.62 points, with no variance estimates, no significance testing, and no reported number of evaluation samples. Uniform-sign improvements of this magnitude on multiple-choice accuracy are consistent with noise unless per-condition sample sizes are large. Please report per-setting sample counts and either confidence intervals or a paired significance test, and temper the language if the margins are not significant. This matters because §5.4 is the only evidence that ObsDrive's gains are not K-Radar-specific.
minor comments (8)
  1. [Table 2] Table 2 omits several models present in Table 1 (InternVL3-8B, InternVL3-38B, Qwen3VL-8B, Qwen3VL-32B). Since Qwen3VL-32B is a label seed model, its per-weather and per-modality breakdown is of particular interest and should be included for completeness.
  2. [Table 2] Table 2 appears internally inconsistent for some rows: e.g., Perception-R1-7B scores 13.91/7.27/11.70/11.76/11.71 across weather conditions yet 47.43/46.07/45.75 across CAM/LIDAR/RADAR, and Alpamayo-1.5 shows the identical weather row (13.91/7.27/11.70/11.76/11.71) as Perception-R1-7B. Identical rows for two different models strongly suggest a copy error; please audit this table.
  3. [§3.2, Stage 1] The observability meta-annotation relies on VLM judgment of 3D boxes projected into BEV renderings, but the rendering details (color coding, point density thresholds, occlusion handling) are deferred to the supplement. Given that observability labels anchor the entire benchmark, a brief summary of the rendering convention and its sensitivity belongs in the main text of §3.2.
  4. [§5.1] Evaluation uses LLM-as-Judge [15] to select the chosen option (§5.1). For multiple-choice answers, exact-match parsing would be more reproducible; if the judge is needed for formatting failures, report the judge's agreement with exact-match on a sample and the judge model used.
  5. [§3.4, Fig. 3] Fig. 3 shows the weather distribution of train vs. test frames/samples, but the proportions visibly differ between splits (e.g., normal-weather share). Please discuss whether per-weather results in Table 2 are confounded by differing condition mixtures, or confirm conditioning is per-weather as the column structure implies.
  6. [§3.3] Notation: the object token format <CAM, x, y, BEV, X, Y> is used throughout (Fig. 1–2) but never formally defined; state the coordinate frames and units once in §3.3.
  7. [Various] Typos: 'muti-modal' (Fig. 2 caption), 'inherit split of source sequence' (§3.2, grammar), 'Will it possible intersect' (Fig. 2), spacing around bolded terms in the abstract and contribution list.
  8. [§5.1] The claim in §5.1 that 'all 14 off-the-shelf models in Table 1 are evaluated without access to the training split' is good practice, but Table 1 lists 15 rows including ObsDrive variants; please count and phrase precisely.

Circularity Check

1 steps flagged

No derivation-by-construction circularity; only a mild annotation-pipeline overlap that does not force the reported rankings or capability-gap claims.

specific steps
  1. other [§3.2 Stages 1–3; cf. Table 1]
    "To obtain these annotations, we leverage strong vision-language models (InternVL3-38B and Qwen3VL-32B-Instruct) to analyze multimodal inputs with 3D bounding boxes and generate initial observability labels. These annotations are further refined using Gemini-3.1-Pro... All questions are initially generated using InternVL3-38B and Qwen3VL-32B-Instruct, followed by refinement with Gemini-3.1-Pro... Finally, we perform human verification to correct erroneous samples."

    Not circular by construction: answers are checked against K-Radar boxes/trajectories and human review, and seed models do not top observability scores. Residual concern only—MCQ keys begin in the same VLM family later ranked—so stylistic agreement with the annotator ensemble could slightly inflate relative standing of Gemini/Qwen-like models versus distant systems. This is annotation-bias risk, not a fitted parameter or definitional identity that forces Table 1.

full rationale

ObsDriveBench is an empirical benchmark paper, not a first-principles derivation. Held-out evaluation uses the official K-Radar sequence split; the 14 off-the-shelf models in Table 1 never see the training MCQs; and ObsDrive’s gains come from ordinary train-on-train / test-on-test SFT+GRPO, with additional transfer checks on DriveBench and DriveLMM-o1. Ground truth is anchored in K-Radar 3D boxes, ego trajectories, and human verification against sensor views, not in a self-referential equation. Label seeding by InternVL3/Qwen3VL and refinement by Gemini is a real methodology risk (possible prior alignment), but it is not equivalence-by-construction: those same seed/refine models do not dominate the leaderboard on observability (InternVL3-38B 40.82, Qwen3VL-32B 41.44 vs ObsDrive-RL 65.31), so the central “observability bottleneck” ordering is not forced by the annotator ensemble. No self-citation uniqueness theorem, fitted constant renamed as prediction, or definitional X↔Y loop appears in the claimed chain. Score 1 only flags residual annotation-overlap concern, not load-bearing circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 3 invented entities

This is an empirical systems/benchmark paper. Load-bearing premises are domain modeling choices (discrete observability, BEV projections as VLM inputs, MCQ proxies for intermediate driving competence) and annotation/training procedures, not fitted physical constants. Free parameters are mostly training/reward and teacher-model choices; invented entities are benchmark constructs rather than new physics.

free parameters (4)
  • Observability scale O_m(o_i) ∈ {0,1,2} = ternary labels via VLM+human pipeline
    Discrete fully/partially/unobservable levels are a design choice that defines labels and many MCQ answers; thresholds for ‘partial’ are not uniquely determined by sensors.
  • GRPO reward R = R_acc + R_format = unspecified numeric weights in main text
    Relative weighting and format reward shape ObsDrive-RL; exact coefficients/hyperparameters are not fully specified in the main text.
  • Temporal window of 21 frames (±1 s) = 21 frames
    Sample construction horizon is chosen by authors and affects interaction/trajectory questions.
  • Teacher/refiner model stack for labels and MCQs = InternVL3-38B + Qwen3VL-32B → Gemini-3.1-Pro
    InternVL3-38B, Qwen3VL-32B, Gemini-3.1-Pro choices materially define the benchmark content before human edit.
axioms (6)
  • domain assumption Degraded environmental observability (unreliable, cross-modally inconsistent multi-sensor observations) is the key difficulty for driving VLMs under adverse weather.
    Stated in abstract and §1 as the organizing premise for benchmark design; not independently proven, but used to structure all tasks.
  • domain assumption Rendering LiDAR and radar as BEV images into an unmodified vision encoder is a valid multimodal interface for fair VLM evaluation.
    §3.2 and §5.1 multimodal adaptation; alternative point-cloud/radar tokenizers could change results.
  • domain assumption Capability-oriented MCQ accuracy isolates intermediate failure sources along observability–spatial–decision without claiming closed-loop planning performance.
    Explicitly argued in §2.2; underpins interpretation of Table 1 decision scores.
  • domain assumption Official K-Radar detection split yields a valid train/test separation for generated questions.
    §3.2 Train-test split; questions inherit sequence split rather than being adversarially decontaminated for language leakage.
  • standard math Standard supervised fine-tuning and GRPO-style RL are appropriate optimization methods for improving the measured capabilities.
    §4 cites GRPO framework; used as engineering baseline, not a new theorem.
  • ad hoc to paper Human verification by PhD annotators with sensor views and 3D boxes yields near-correct unique answers (reported pass rates 99.96%/97.20%/99.96%).
    §3.2 Human verification; quality of the entire benchmark rests on this process and unreported inter-annotator agreement details in the main text.
invented entities (3)
  • ObsDriveBench (three-dimension observability-aware MCQ benchmark) no independent evidence
    purpose: Provide diagnostic evaluation of driving VLMs under real adverse weather multi-modal inputs.
    New dataset/task suite constructed on K-Radar; existence is by author construction, validated only via internal stats and model leaderboards.
  • Object-level cross-modal observability meta-annotation no independent evidence
    purpose: Ground modality-specific and cross-modal observability questions.
    Labels are newly defined and produced for this paper; external sensors do not emit these ternary labels natively.
  • ObsDrive model (normal-weather SFT + adverse-weather RL) no independent evidence
    purpose: Show that diagnosed gaps are partly reducible with a two-stage training recipe.
    Reference baseline on Qwen2.5VL-7B; not required for benchmark findings per authors, but central to method claims.

pith-pipeline@v1.2.0-grok45-kimik3 · 21264 in / 4166 out tokens · 87410 ms · 2026-07-30T19:45:54.783908+00:00 · methodology

0 comments
read the original abstract

Autonomous driving under adverse weather remains a critical challenge, yet existing vision-language benchmarks mainly evaluate under standard conditions, synthetic corruptions, or single modality. As a result, it remains unclear how vision-language models behave under real-world adverse weather with multi-modal inputs. We argue that a key difficulty lies in degraded environmental observability: under fog, rain, snow, and low illumination, multi-modal observations become unreliable and cross-modally inconsistent, posing challenges to scene understanding, and subsequent decision-making. To study this, we introduce \textbf{ObsDriveBench}, a real-world multi-modal benchmark for adverse-weather autonomous driving. Our benchmark is designed with three capability dimensions: \textbf{observability awareness}, \textbf{spatial reliability}, and \textbf{risk-aware decision-making}, enabling fine-grained diagnosis of model behavior under degraded observations. We construct the benchmark through observability meta-annotation, scene description, and capability oriented multiple-choice tasks over synchronized camera, LiDAR, and radar inputs, forming a benchmark with over 14k training and 13k test questions. Experiments reveal consistent performance degradation of existing vision-language models. We further introduce \textbf{ObsDrive} model with normal-weather supervised fine-tuning and adverse-weather reinforcement learning, improving robustness across all three capabilities. The dataset and evaluation code will be released at \href{https://github.com/russellyq/ObsDriveBench}{\texttt{ObsDriveBench}}.

Figures

Figures reproduced from arXiv: 2607.23537 by Jiaqi Xu, Pheng-Ann Heng, Qiao Yan, Yihan Wang, Zhenghao Xing.

Figure 1
Figure 1. Figure 1: Examples from ObsDriveBench. Compared with exist [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Pipeline of benchmark construction. Sampled from the real-world adverse-weather driving dataset, we first perform [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Data distribution of ObsDriveBench. Frame-level [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 12 linked inside Pith

  1. [1]

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhao- hai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Jun- yang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shix...

    Pith/arXiv arXiv 2025

  2. [2]

    Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhao- hai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Jun- yang Lin. Qwen2.5-vl technical repor...

    Pith/arXiv arXiv 2025

  3. [3]

    Reasondrive: Ef- ficient visual question answering for autonomous vehicles with reasoning-enhanced small vision-language models

    Amirhosein Chahe and Lifeng Zhou. Reasondrive: Ef- ficient visual question answering for autonomous vehicles with reasoning-enhanced small vision-language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 3870–3879, 2025. 6

    2025

  4. [4]

    Probing the reliability of driving vlms: From inconsistent responses to grounded temporal reason- ing.arXiv preprint arXiv:2603.09512, 2026

    Chun-Peng Chang, Chen-Yu Wang, Holger Caesar, and Alain Pagani. Probing the reliability of driving vlms: From inconsistent responses to grounded temporal reason- ing.arXiv preprint arXiv:2603.09512, 2026. 1, 3

    arXiv 2026

  5. [5]

    Expanding performance boundaries of open-source multimodal models with model, data, and test- time scaling, 2025.URL https://arxiv

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhang- wei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test- time scaling, 2025.URL https://arxiv. org/abs/2412.05271, 6, 2025. 6

    Pith/arXiv arXiv 2025

  6. [6]

    Survey on lidar perception in adverse weather conditions

    Mariella Dreissig, Dominik Scheuble, Florian Piewak, and Joschka Boedecker. Survey on lidar perception in adverse weather conditions. In2023 IEEE Intelligent Vehicles Sym- posium (IV), pages 1–8. IEEE, 2023. 2

    2023

  7. [7]

    Stsbench: A spatio-temporal scenario benchmark for multi-modal large language models in autonomous driving.arXiv preprint arXiv:2506.06218, 2025

    Christian Fruhwirth-Reisinger, Du ˇsan Mali´c, Wei Lin, David Schinagl, Samuel Schulter, and Horst Possegger. Stsbench: A spatio-temporal scenario benchmark for multi-modal large language models in autonomous driving.arXiv preprint arXiv:2506.06218, 2025. 1, 3

    Pith/arXiv arXiv 2025

  8. [8]

    Gemini 3.1 pro, 2026

    Google DeepMind. Gemini 3.1 pro, 2026. Multimodal large language model. 4, 6

    2026

  9. [9]

    4d millimeter-wave radar in autonomous driving: A survey

    Zeyu Han, Jiahao Wang, Zikun Xu, Shuocheng Yang, Lei He, Shaobing Xu, Jianqiang Wang, and Keqiang Li. 4d millimeter-wave radar in autonomous driving: A survey. arXiv preprint arXiv:2306.04242, 2023. 2

    Pith/arXiv arXiv 2023

  10. [10]

    Drivelmm-o1: A step-by-step reasoning dataset and large multimodal model for driving scenario understanding

    Ayesha Ishaq, Jean Lahoud, Ketan More, Omkar Thawakar, Ritesh Thawkar, Dinura Dissanayake, Noor Ahsan, Yuhao Li, Fahad Shahbaz Khan, Hisham Cholakkal, et al. Drivelmm-o1: A step-by-step reasoning dataset and large multimodal model for driving scenario understanding. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages ...

    2025

  11. [11]

    The robodrive chal- lenge: Drive anytime anywhere in any condition.arXiv preprint arXiv:2405.08816, 2024

    Lingdong Kong, Shaoyuan Xie, Hanjiang Hu, Yaru Niu, Wei Tsang Ooi, Benoit R Cottereau, Lai Xing Ng, Yuexin Ma, Wenwei Zhang, Liang Pan, et al. The robodrive chal- lenge: Drive anytime anywhere in any condition.arXiv preprint arXiv:2405.08816, 2024. 3

    Pith/arXiv arXiv 2024

  12. [12]

    Spacedrive: Infusing spatial aware- ness into vlm-based autonomous driving.arXiv preprint arXiv:2512.10719, 2, 2025

    Peizheng Li, Zhenghao Zhang, David Holtz, Hang Yu, Yutong Yang, Yuzhi Lai, Rui Song, Andreas Geiger, and Andreas Zell. Spacedrive: Infusing spatial aware- ness into vlm-based autonomous driving.arXiv preprint arXiv:2512.10719, 2, 2025. 3

    Pith/arXiv arXiv 2025

  13. [13]

    Recogdrive: A reinforced cogni- tive framework for end-to-end autonomous driving

    Yongkang Li, Kaixin Xiong, Xiangyu Guo, Fang Li, Sixu Yan, Gangwei Xu, Lijun Zhou, Long Chen, Haiyang Sun, BING W ANG, Kun Ma, Guang Chen, Hangjun Ye, Wenyu Liu, and Xinggang Wang. Recogdrive: A reinforced cogni- tive framework for end-to-end autonomous driving. InThe Fourteenth International Conference on Learning Represen- tations, 2026. 3, 6

    2026

  14. [14]

    Robodrivevlm: A novel benchmark and baseline towards robust vision- language models for autonomous driving.arXiv preprint arXiv:2512.01300, 2025

    Dacheng Liao, Mengshi Qi, Peng Shu, Zhining Zhang, Yuxin Lin, Liang Liu, and Huadong Ma. Robodrivevlm: A novel benchmark and baseline towards robust vision- language models for autonomous driving.arXiv preprint arXiv:2512.01300, 2025. 1, 3

    arXiv 2025

  15. [15]

    Calibrating llm-based evaluator

    Yuxuan Liu, Tianchi Yang, Shaohan Huang, Zihan Zhang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, and Qi Zhang. Calibrating llm-based evaluator. InProceedings of the 2024 joint international conference on computational linguistics, language resources and evaluation (lrec-coling 2024), pages 2638–2656, 2024. 6

    2024

  16. [16]

    Robust object detection in adverse weather conditions: Ecl-yolov11 for automotive vision systems.Sensors, 26(1): 304, 2026

    Zhaohui Liu, Jiaxu Zhang, Xiaojun Zhang, and Hongle Song. Robust object detection in adverse weather conditions: Ecl-yolov11 for automotive vision systems.Sensors, 26(1): 304, 2026. 2

    2026

  17. [17]

    Dolphins: Multimodal language model for driving

    Yingzi Ma, Yulong Cao, Jiachen Sun, Marco Pavone, and Chaowei Xiao. Dolphins: Multimodal language model for driving. InEuropean Conference on Computer Vision, pages 403–420. Springer, 2024. 6

    2024

  18. [19]

    K-radar: 4d radar object detection for autonomous driving in various weather conditions.Advances in Neural Information Processing Systems, 35:3819–3829, 2022

    Dong-Hee Paek, Seung-Hyun Kong, and Kevin Tirta Wijaya. K-radar: 4d radar object detection for autonomous driving in various weather conditions.Advances in Neural Information Processing Systems, 35:3819–3829, 2022. 2, 4, 5

    2022

  19. [20]

    Nuscenes-qa: A multi-modal visual ques- tion answering benchmark for autonomous driving scenario

    Tianwen Qian, Jingjing Chen, Linhai Zhuo, Yang Jiao, and Yu-Gang Jiang. Nuscenes-qa: A multi-modal visual ques- tion answering benchmark for autonomous driving scenario. InProceedings of the AAAI Conference on Artificial Intelli- gence, pages 4542–4550, 2024. 3

    2024

  20. [21]

    Deepseekmath: Pushing the limits of math- ematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of math- ematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024. 5

    Pith/arXiv arXiv 2024

  21. [22]

    Radiate: A radar dataset for automotive perception in bad weather

    Marcel Sheeny, Emanuele De Pellegrin, Saptarshi Mukher- jee, Alireza Ahrabian, Sen Wang, and Andrew Wallace. Radiate: A radar dataset for automotive perception in bad weather. In2021 IEEE International Conference on Robotics and Automation (ICRA), pages 1–7. IEEE, 2021. 2

    2021

  22. [23]

    Drivelm: Driving with graph visual question answering

    Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Jens Beißwenger, Ping Luo, Andreas Geiger, and Hongyang Li. Drivelm: Driving with graph visual question answering. InEuropean conference on computer vision, pages 256–274. Springer, 2024. 1, 3

    2024

  23. [24]

    Openai gpt-5 system card.arXiv preprint arXiv:2601.03267, 2025

    Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. Openai gpt-5 system card.arXiv preprint arXiv:2601.03267, 2025. 6

    Pith/arXiv arXiv 2025

  24. [25]

    Minddrive: An all-in-one framework bridging world models and vision-language model for end-to-end au- tonomous driving.arXiv preprint arXiv:2512.04441, 2025

    Bin Sun, Yaoguang Cao, Yan Wang, Rui Wang, Jiachen Shang, Xiejie Feng, Jiayi Lu, Jia Shi, Shichun Yang, Xiaoyu Yan, et al. Minddrive: An all-in-one framework bridging world models and vision-language model for end-to-end au- tonomous driving.arXiv preprint arXiv:2512.04441, 2025. 3

    arXiv 2025

  25. [26]

    Nuscenes-spatialqa: A spatial understanding and reasoning benchmark for vision- language models in autonomous driving

    Kexin Tian, Jingrui Mao, Yunlong Zhang, Jiwan Jiang, Yang Zhou, and Zhengzhong Tu. Nuscenes-spatialqa: A spatial understanding and reasoning benchmark for vision- language models in autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 4567–4576, 2025. 1, 3

    2025

  26. [27]

    Omnidrive: A holistic vision-language dataset for autonomous driving with counterfactual reasoning

    Shihao Wang, Zhiding Yu, Xiaohui Jiang, Shiyi Lan, Min Shi, Nadine Chang, Jan Kautz, Ying Li, and Jose M Al- varez. Omnidrive: A holistic vision-language dataset for autonomous driving with counterfactual reasoning. InPro- ceedings of the computer vision and pattern recognition con- ference, pages 22442–22452, 2025. 1, 3

    2025

  27. [28]

    Alpamayo-r1: Bridging reasoning and action prediction for generalizable autonomous driving in the long tail.arXiv preprint arXiv:2511.00088, 2025

    Yan Wang, Wenjie Luo, Junjie Bai, Yulong Cao, Tong Che, Ke Chen, Yuxiao Chen, Jenna Diamond, Yifan Ding, Wen- hao Ding, et al. Alpamayo-r1: Bridging reasoning and action prediction for generalizable autonomous driving in the long tail.arXiv preprint arXiv:2511.00088, 2025. 3

    Pith/arXiv arXiv 2025

  28. [29]

    Perception-r1: Advancing mul- timodal reasoning capabilities of MLLMs via visual percep- tion reward

    Tong Xiao, Xin Xu, Zhenya Huang, Hongyu Gao, Quan Liu, Qi Liu, and Enhong Chen. Perception-r1: Advancing mul- timodal reasoning capabilities of MLLMs via visual percep- tion reward. InThe Fourteenth International Conference on Learning Representations, 2026. 6

    2026

  29. [30]

    Are vlms ready for autonomous driving? an empirical study from the reliability, data and metric perspectives

    Shaoyuan Xie, Lingdong Kong, Yuhao Dong, Chonghao Sima, Wenwei Zhang, Qi Alfred Chen, Ziwei Liu, and Liang Pan. Are vlms ready for autonomous driving? an empirical study from the reliability, data and metric perspectives. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6585–6597, 2025. 1, 3, 8

    2025

  30. [31]

    R1-vl: Learning to reason with multimodal large language models via step- wise group relative policy optimization, 2025

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step- wise group relative policy optimization, 2025. 6

    2025

  31. [32]

    Perception and sensing for autonomous ve- hicles under adverse weather conditions: A survey.ISPRS Journal of Photogrammetry and Remote Sensing, 196:146– 177, 2023

    Yuxiao Zhang, Alexander Carballo, Hanting Yang, and Kazuya Takeda. Perception and sensing for autonomous ve- hicles under adverse weather conditions: A survey.ISPRS Journal of Photogrammetry and Remote Sensing, 196:146– 177, 2023. 2

    2023

  32. [33]

    Omnidrive-r1: Reinforcement- driven interleaved multi-modal chain-of-thought for trust- worthy vision-language autonomous driving.arXiv preprint arXiv:2512.14044, 2025

    Zhenguo Zhang, Haohan Zheng, Yishen Wang, Le Xu, Tianchen Deng, Xuefeng Chen, Qu Chen, Bo Zhang, and Wuxiong Huang. Omnidrive-r1: Reinforcement- driven interleaved multi-modal chain-of-thought for trust- worthy vision-language autonomous driving.arXiv preprint arXiv:2512.14044, 2025. 3

    Pith/arXiv arXiv 2025

  33. [34]

    Opendrivevla: Towards end- to-end autonomous driving with large vision language action model.arXiv preprint arXiv:2503.23463, 2025

    Xingcheng Zhou, Xuyuan Han, Feng Yang, Yunpu Ma, V olker Tresp, and Alois Knoll. Opendrivevla: Towards end- to-end autonomous driving with large vision language action model.arXiv preprint arXiv:2503.23463, 2025. 3

    arXiv 2025

  34. [35]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shen- glong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025. 4, 6

    Pith/arXiv arXiv 2025