{"id":"fcab4735-d3a6-4bed-95af-cea4cf097f4b","arxiv_id":"2501.04003","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A new benchmark with clean, corrupted, and text-only driving inputs shows that vision-language models can answer many driving questions without visual information, so standard accuracy metrics overestimate visual grounding.","lead":"DriveBench tests 12 vision-language models on driving questions using clean, corrupted, and blank images, and finds the models often keep producing plausible answers even when the image is removed. The results suggest that current benchmarks can overstate how much these models actually see, which is important for using them in self-driving systems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DriveBench's GPT-4o curation filter may select for textually guessable cases, so the text-only control could be an artifact of the benchmark rather than a general VLM property.","rationale":"The reader's weakest_assumption correctly identifies the GPT-4o-based filtering in Section 3.1 as the load-bearing step. The paper needs the filtered set to contain instances where a single frame carries the visual information needed to answer; otherwise, the clean-vs-text-only comparison does not measure visual grounding. The filter explicitly removes 'instances highly dependent on temporal context or present significant interpretive challenges for current VLMs,' which may be exactly the cases where text priors are insufficient. Since the prompts contain camera and coordinate cues, GPT-4o's success does not guarantee that the image is necessary. This concern is concrete and testable: re-running on the filtered-out instances would show whether the headline text-only robustness is a property of DriveBench's curation or a general VLM trait. It does not invalidate the paper's critique of existing benchmarks—the DriveLM distribution bias and the prior DriveLM-Agent results stand independently—but it limits the generality of the DriveBench-based evidence. The reader's CONDITIONAL verdict already accommodates this uncertainty, so no change is needed. The proposed test should be added as a condition for acceptance along with releasing the exact filtering pipeline and reporting variance in the GPT scores.","tokens_in":45919,"tokens_out":5121,"duration_ms":53006,"concrete_test":"Re-run the full clean/corrupted/text-only evaluation on the instances removed by the GPT-4o filter (or a random sample of the unfiltered DriveLM-nuScenes set). If text-only accuracy/GPT scores are substantially lower on those removed instances than on DriveBench, the headline result is driven by the curation filter and the paper should scope its claim accordingly; if text-only performance remains comparably high, the concern does not land.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that VLMs can pass driving benchmarks without visual grounding—is important and partly supported by the DriveLM distribution imbalance (78.6% 'Going Straight'). However, DriveBench's construction in Section 3.1 filters out any instance GPT-4o cannot correctly interpret, on the assumption that GPT-4o's success indicates sufficient single-frame visual cues. Because prompts include camera names and normalized coordinates, GPT-4o can answer correctly using spatial/textual priors (the paper's own Fig. 10 shows this behavior). The filter may therefore preferentially keep cases where text-only guessing succeeds and remove cases where genuine visual grounding is required. If so, the comparable clean vs. text-only results in Tab. 2 and Tab. 3 partly measure the guessability of the filtered subset, not a general inability of VLMs to use visual input. The broad conclusion that 'VLMs often generate plausible responses from general knowledge or textual cues' remains plausible, but the DriveBench-based quantification and the claim that current VLM driving assistants can pass benchmarks without seeing the road would be over-scoped if the filtered-out instances behave differently.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"DriveBench is an empirical benchmark study that evaluates 12 vision-language models (VLMs) on four driving-language tasks under clean, corrupted, and text-only inputs. The benchmark is built from DriveLM, rebalanced to counter the 78.6% 'Going Straight' bias, and supplemented with 15 corruption types. The paper reports three main findings: (1) VLMs often maintain comparable accuracy and GPT-scores under heavy corruption and even with no image at all, suggesting they rely on general knowledge and textual priors rather than visual grounding; (2) VLMs exhibit corruption awareness only when explicitly prompted; and (3) standard metrics (ROUGE/BLEU, and even naive GPT scores) are insensitive to fabrication, while rubric- and context-augmented GPT evaluation is more discriminative. The authors release the benchmark toolkit and dataset publicly.","tokens_in":46116,"tokens_out":4606,"duration_ms":45530,"significance":"If the findings hold, this is a valuable and timely empirical contribution: it provides a concrete, publicly available testbed with 17 input settings, introduces a text-only control that exposes a failure mode common to driving VLMs, and offers a transparent analysis of dataset imbalance (the 78.6% 'Going Straight' statistic) and metric pathologies. The paper's strengths include a large model sweep, a clear corruption taxonomy, a falsifiable text-only experimental design, and the public release of code and data. However, the central claim that 'VLMs can pass driving benchmarks without seeing the road' rests on the curated subset of DriveBench, and the curation procedure plus the evaluation methodology introduce load-bearing concerns that currently prevent the quantitative results from fully supporting the broad conclusions.","major_comments":[{"comment":"The curation filter keeps only instances that 'GPT-4o can correctly interpret,' on the assumption that GPT-4o's success indicates sufficient single-frame visual information. This assumption is load-bearing for the central claim, but the paper's own Fig. 10 shows that models can answer questions from camera names and coordinates without any visual input. If GPT-4o's successes are partly driven by the same textual priors, the filter may preferentially retain textually guessable cases and remove the cases that genuinely require visual grounding. The comparable clean vs. text-only results in Tabs. 2 and 3 could then partly measure the guessability of the filtered subset rather than a general VLM property. Please provide a concrete test: evaluate the models on the filtered-out instances (where GPT-4o fails) and show whether text-only performance drops, or quantify the distribution of answer priors in the kept versus removed sets.","section":"Sec. 3.1 (Dataset curation)"},{"comment":"The main quantitative evidence for the 'comparable performance' claim is presented as single point estimates without error bars, confidence intervals, or significance tests. Several of the decisive differences are small (e.g., GPT-4o perception GPT score 35.37 clean vs. 36.48 text-only; accuracy 59.0 vs. 59.5 in Tab. 3). With roughly 200 perception MCQs and 200 behavior MCQs, these differences are plausibly within sampling noise. Please report bootstrap confidence intervals or per-keyframe variance and state which clean/text-only/corrupted comparisons are statistically distinguishable.","section":"Tabs. 2 and 3"},{"comment":"The human evaluation is based on only 15 keyframes. The large human accuracy drops under corruption (e.g., perception 93.3 clean vs. 33.3 fog) are used to argue that the corruptions are genuinely impactful and that VLMs' lack of degradation is abnormal. With n=15, the standard error is large and the reported accuracy differences are fragile. Please either increase the human sample or report confidence intervals; without this, the human-vs-VLM contrast in Sec. 4.2.1 is not quantitatively supported.","section":"Appendix C.4 and Fig. 8 (Human evaluation)"}],"minor_comments":[{"comment":"The text in A.3 states that the curated dataset contains 1,461 questions, but Tab. 6 sums to 1,261 (400+61+600+200). The abstract reports 20,498 total QA pairs, which matches 19,237 robustness pairs plus 1,261, not 1,461. Please reconcile these numbers.","section":"A.3 and Tab. 6"},{"comment":"Tab. 3 reports only 8 of the 12 evaluated models; the selection criterion is stated only in the main text. Please add the criterion to the table caption for clarity.","section":"Tab. 3 caption"},{"comment":"The y-axis label 'Performance Change' is ambiguous; please specify whether it is absolute accuracy change, relative change, or another quantity.","section":"Fig. 8"},{"comment":"There is a typo in the table title ('Predicion'), and the Qwen2VL7B and Qwen2VL72B rows are duplicated verbatim for the 7B model entries.","section":"Tab. 14"},{"comment":"The abstract claims 'we propose refined evaluation metrics,' but the paper does not introduce a new metric; it advocates rubric- and context-augmented GPT scoring. Consider wording such as 'refined evaluation protocols' to avoid overclaiming.","section":"Abstract and Sec. 5"},{"comment":"The caption of Fig. 12 should state the task and condition (prediction task, clean inputs) so the reader can interpret the correlation plots without consulting the main text.","section":"Fig. 12"}],"recommendation":"major_revision","confidential_remarks":"The paper builds on DriveLM, which several of the authors co-authored, and uses GPT-4o for curation and GPT-3.5 for scoring. This is not disqualifying, but the circularity risk is real and should be addressed explicitly with a robustness check. The paper's scope is also narrower than the abstract suggests: all experiments use DriveLM, as the authors acknowledge in E.2. The manuscript would benefit from either softening the general claims or adding a cross-dataset sanity check. Overall, the direction is sound and the benchmark is potentially useful, but the curation-filter concern and the lack of statistical rigor in the headline comparisons should be resolved before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: worth reading and citing, but the central claim is strong (VLMs can fabricate plausible driving answers from text alone) while the quantification is weaker than the framing suggests. The text-only control across twelve models is the most convincing piece. DriveBench itself—15 corruption types, 19,200 frames, four tasks—is a clean artifact and likely to be reused. The paper also does a fair job showing that accuracy, ROUGE/BLEU, and naive GPT scores all fail to penalize ungrounded answers.\n\nThe soft spots are real but not fatal. Section 3.1 filters the dataset to cases GPT-4o can correctly interpret, justified as ensuring single-frame visual sufficiency. That filter can select for textually guessable examples—the paper's own Fig. 10 shows models using camera names and coordinates to guess. So the clean-vs-text-only gap on DriveBench may overstate how often VLMs can pass without visual input in general. I would not call this a refutation; even on the filtered set many models show no accuracy drop under text-only, and the DriveLM distribution imbalance (78.6% 'Going Straight') independently supports the concern. But the exact magnitudes should be treated as bounds, not point estimates.\n\nSecond soft spot: the main tables use GPT-3.5 rubric scores without error bars or significance tests. For a paper whose message is 'metrics mislead,' leaning on a noisy LLM judge is awkward. The accuracy tables help, but they are reported for a subset and without variance. The human evaluation uses 15 keyframes—small, and reported as point estimates.\n\nThe metric proposal is more of a rubric list than a concrete scoring function. That is fine as a direction, but it should be made operational or labeled as a call to action.\n\nOverall: the paper is honest, includes its own limitations (Section E.2), and the citation pattern is appropriate. The stress-test worry about curation is valid and should be addressed by releasing the exact filtering pipeline and checking whether text-only success survives on the excluded instances. I would send this to peer review rather than desk-reject: the benchmark is reusable, the claim is important, and the flaws are fixable. A good referee will ask for error bars, the filtering pipeline, and a clearer metric specification.","headline":"DriveBench is a useful, reusable warning about VLM overreliance on text and priors, but the curated subset and GPT-rubric scoring leave the magnitude of the problem less settled than the title implies.","tokens_in":46650,"tokens_out":2210,"would_cite":true,"duration_ms":22856,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DriveBench shows that vision-language models for driving can produce plausible answers with no image at all, so current benchmark scores overstate their visual grounding.","keywords":["vision-language models","autonomous driving","visual grounding","benchmark","out-of-distribution robustness","text-only evaluation","dataset bias","GPT-based evaluation"],"falsifier":"Take the DriveBench perception questions and replace the text-only condition with images whose pixels are randomly permuted or inverted, so no scene information remains but the question text is unchanged; if VLM accuracy stays at the clean-image level on these inputs, the claim that models lean on language priors is supported, whereas a drop to chance would falsify it. Alternatively, compute the always-'Going Ahead' accuracy on the balanced DriveBench behavior MCQs: if that baseline is near chance rather than above 70%, the dataset-imbalance explanation would need revision.","tokens_in":45719,"feed_emoji":"🚗","tokens_out":6024,"duration_ms":55686,"temperature":0.7,"pith_summary":"This paper asks whether vision-language models (VLMs) used for driving really ground their answers in what the camera sees. It builds DriveBench, a benchmark of 19,200 frames and 20,498 question-answer pairs evaluated under clean images, fifteen kinds of image corruption, and text-only inputs with no image at all. Across twelve VLMs, the paper finds that performance on text-only prompts stays close to performance on clean images, and that high accuracy can be achieved by repeating common answers such as 'Going Ahead'. The authors conclude that current VLMs often fabricate plausible driving responses from general knowledge and textual cues, and that existing accuracy, language, and GPT-based metrics systematically overstate their reliability. If true, this means benchmark results for VLM driving assistants cannot be trusted as evidence of visual grounding, and safety-critical deployment needs rethinking.","feed_headline":"Blindfolded, driving VLMs still answer as if they saw the road","feed_subtitle":"DriveBench's text-only tests show models lean on language priors, so current scores overstate driving reliability.","key_machinery":"The load-bearing object is DriveBench, a benchmark built by re-sampling DriveLM's nuScenes-based QA pairs to balance answer distributions and by filtering to instances GPT-4o can answer from a single frame, then testing every model under clean images, 15 synthesized corruptions (weather, sensor failures, blurs, transmission errors), and fully black text-only inputs. The critical mechanism is the text-only condition: comparing a model's answers with no visual input against its clean-image answers exposes how much of the apparent performance comes from language priors rather than vision. A second mechanism is the rubric-aware GPT evaluation, which the paper shows yields more discriminating scores when the prompt includes the question, task rubrics, and scene context rather than the answer and ground truth alone.","core_discovery":"DriveBench's central finding is that VLMs do not reliably ground driving answers in visual input: removing the image entirely leaves GPT scores nearly unchanged across tasks, with some models even scoring higher on text-only prompts, and accuracy on perception multiple-choice questions frequently stays at the same level with black images as with clean ones. The paper attributes this to learned priors, common-sense defaults such as 'Going Ahead', and textual cues embedded in the questions themselves, such as camera names and coordinates. Because existing datasets are imbalanced and current metrics reward fluent, template-like explanations over correct visual reasoning, these fabricated answers pass evaluation. The paper also finds that models are partly aware of corruptions when explicitly told about them, but do not volunteer that awareness or downgrade their confidence, and that driving-specific fine-tuning largely improves answer formatting rather than visual grounding.","pith_inferences":["My inference: the same blindfold test — comparing text-only against full-input performance — could be applied to any VLM claim in robotics, medical imaging, or other safety-critical domains, not just driving.","My inference: the paper's results imply a cheap diagnostic for future models: report the gap between clean, corrupted, and text-only accuracy, since a small gap means the model's answers are likely prior-driven.","My inference: because questions embed camera names and normalized coordinates, even 'text-only' inputs contain geometric cues; removing those cues by scrambling camera labels would isolate how much of the text-only performance comes from pure language priors versus coordinate heuristics.","My inference: a testable extension is to compute the majority-answer baseline ('always Going Ahead') on each balanced task; models near that baseline under corruption are best explained as prior-following rather than vision-based."],"forward_implications":["VLM driving assistants that score well on existing benchmarks may be answering from language priors, so a high benchmark score alone is not evidence of visual understanding.","Robustness to corruption cannot be read from stable scores under corrupted images, because the same stability appears under text-only inputs, indicating the scores may not involve the image at all.","Evaluation protocols for driving VLMs should include text-only and black-image controls, balanced answer distributions, and task-specific rubrics with scene context.","Models' awareness of corruptions, when explicitly prompted, could be used to make them abstain or hedge in degraded conditions, improving reliability.","Fine-tuned driving VLMs can show inflated gains from answer-format matching, so the choice of metric changes conclusions about which model is best."],"supporting_citations":[{"why":"Supplies the DriveLM dataset and QA structure that DriveBench re-samples, and provides the DriveLM-Agent model whose no-pixel and no-feature results motivate the text-only analysis.","marker":"[59]"},{"why":"Documents the dataset imbalance and majority 'Going Ahead' answers in open-loop driving benchmarks, which DriveBench uses to explain text-only performance.","marker":"[39]"},{"why":"GPT-4o is used to filter the benchmark to instances it can answer from a single frame and serves as the strongest evaluated model.","marker":"[2]"},{"why":"Provides the robustness-corruption generation algorithms that DriveBench applies to create its 15 corruption types.","marker":"[30]"},{"why":"Supplies the corruption simulation code for depth and perception robustness that the benchmark adapts.","marker":"[31]"},{"why":"Defines the camera crash and frame lost corruption settings used in the benchmark's sensor-failure conditions.","marker":"[71]"},{"why":"ROUGE-L is one of the language metrics the paper shows fails to reflect correctness, inflating format-matched answers.","marker":"[40]"},{"why":"BLEU is the other language metric shown to reward template matching over visual grounding in the evaluation analysis.","marker":"[54]"}],"fun_headline_variants":["VLMs answer driving questions blindfolded as well as with vision","DriveBench: text-only prompts match visual ones in VLM driving tests","Autonomous driving VLMs lean on language priors, not the road","VLM driving scores don't drop when images are removed—why?","Driving VLMs hallucinate road awareness from camera names and cues"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The benchmark selects test instances by keeping only those GPT-4o can answer correctly from a single frame, assuming that GPT-4o's success marks an instance where one frame carries enough visual information; if that filter inherits GPT-4o's own biases, then the measured differences between clean, corrupted, and text-only inputs do not cleanly measure visual grounding.","fun_headline_variants_meta":{"raw":{"variants":["VLMs answer driving questions blindfolded as well as with vision","DriveBench: text-only prompts match visual ones in VLM driving tests","Autonomous driving VLMs lean on language priors, not the road","VLM driving scores don't drop when images are removed—why?","Driving VLMs hallucinate road awareness from camera names and cues"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000305,"raw_usage":{"total_tokens":1758,"prompt_tokens":964,"completion_tokens":794,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":580,"completion_tokens_details":{"reasoning_tokens":700}},"tokens_in":580,"tokens_out":794,"duration_ms":8289,"temperature":1.0,"reasoning_tokens":700,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:41:16.413263+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the DriveBench perception questions and replace the text-only condition with images whose pixels are randomly permuted or inverted, so no scene information remains but the question text is unchanged; if VLM accuracy stays at the clean-image level on these inputs, the claim that models lean on language priors is supported, whereas a drop to chance would falsify it. Alternatively, compute the always-'Going Ahead' accuracy on the balanced DriveBench behavior MCQs: if that baseline is near chance rather than above 70%, the dataset-imbalance explanation would need revision.","supporting_citations":[{"cited_title":"Drivelm: Driving with graph visual question answering","cited_arxiv_id":null,"evidence_quote":"Supplies the DriveLM dataset and QA structure that DriveBench re-samples, and provides the DriveLM-Agent model whose no-pixel and no-feature results motivate the text-only analysis."},{"cited_title":"Is ego status all you need for open- loop end-to-end autonomous driving? In IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 14864–14873, 2024","cited_arxiv_id":null,"evidence_quote":"Documents the dataset imbalance and majority 'Going Ahead' answers in open-loop driving benchmarks, which DriveBench uses to explain text-only performance."},{"cited_title":"Robo3d: Towards robust and reliable 3d perception against corruptions","cited_arxiv_id":null,"evidence_quote":"Provides the robustness-corruption generation algorithms that DriveBench applies to create its 15 corruption types."},{"cited_title":"Robodepth: Robust out-of-distribution depth estimation under corruptions","cited_arxiv_id":null,"evidence_quote":"Supplies the corruption simulation code for depth and perception robustness that the benchmark adapts."},{"cited_title":"Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving","cited_arxiv_id":"2405.17426","evidence_quote":"Defines the camera crash and frame lost corruption settings used in the benchmark's sensor-failure conditions."},{"cited_title":"Rouge: A package for automatic evaluation of summaries","cited_arxiv_id":null,"evidence_quote":"ROUGE-L is one of the language metrics the paper shows fails to reflect correctness, inflating format-matched answers."},{"cited_title":"Bleu: a method for automatic evaluation of machine translation","cited_arxiv_id":null,"evidence_quote":"BLEU is the other language metric shown to reward template matching over visual grounding in the evaluation analysis."}],"review_version":1}