{"id":"e33d606d-3946-4432-848d-a1a37a7a9a59","arxiv_id":"2504.16134","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"A broad survey of MLLM applications in traffic safety, summarizing capabilities, datasets, and future directions, with mostly unsupported performance claims.","lead":"This paper is a review, not an experimental study. It surveys how multimodal large language models (MLLMs) are being applied to traffic safety, and it argues that MLLMs will improve scene understanding, decision-making, and robustness compared to traditional ADAS.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1's 'MLLM' improvements often come from non-LLM multimodal systems, so the key evidence does not test MLLMs even if the numbers are accurate.","rationale":"The reader's REJECT verdict is correct, and their concern about unsupported/mismatched numbers is real. But the more load-bearing issue is structural: the quantitative evidence in the comparison table and narrative is not only potentially inaccurate, it is largely measuring the wrong systems. Several cited sources (CAFuser for mIoU, an aerial object-detector attack for adversarial robustness) are not multimodal LLMs at all, so the improvements they report cannot be attributed to MLLMs. This means the central claim lacks empirical support even under the charitable assumption that every cited number is correctly transcribed. The review also fails as a survey because it gives no inclusion criteria, no search protocol, and no transparent metric-extraction method, making its comparative conclusions non-reproducible. My proposed test, a source-by-source traceability audit, would settle whether any valid MLLM evidence remains. Since the reader already rejected the paper and I agree with that outcome, the verdict is unchanged. I mark agreement as 'partial' because my primary concern differs from the reader's weakest-assumption framing: I focus on the conflation of non-LLM multimodal systems with MLLMs, rather than only the accuracy of quoted numbers.","tokens_in":10929,"tokens_out":3964,"duration_ms":34908,"concrete_test":"Build a traceability audit for every numeric claim in Table 1 and Section 2. For each claim, fetch the cited paper (via arXiv, DOI, or IEEE Xplore), and check three things: (a) Does the exact number appear in that paper? (b) Is the system evaluated actually an MLLM (i.e., has a language-model component enabling multimodal reasoning)? (c) Does the benchmark and condition match what the review states (e.g., KITTI under rain)? Remove every row that fails any check. If the remaining rows cannot support a consistent MLLM advantage, the REJECT verdict stands.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical comparison (Table 1 and Section 2) does not actually test the paper's central claim that MLLMs enhance traffic safety. The MLLM-side numbers are largely drawn from systems that are not MLLMs. The mIoU improvement (72.3% to 88.6%) is attributed to [13], CAFuser, a condition-aware multimodal fusion network without any LLM component; KITTI also has no standard 'rainy conditions' benchmark, so the number cannot be a KITTI rainy result. The adversarial robustness row cites [19], which attacks an aerial-imagery CNN detector, and [20], a paper on vision-language model robustness that does not report the claimed 'GPS + historical data cross-validation' reducing misclassifications from 89% to 12%. The zero-shot anomaly detection figure of 76.31% is attributed to [14], which is the same paper as [6] and [26] (R. Zhang et al.), cited for different claims. Thus the evidence base conflates multimodal fusion, vision-language robustness, and true MLLMs. Even if every number is accurately transcribed from its source, the numbers do not license the conclusion that MLLMs are superior to traditional ADAS. The review supplies no systematic search or metric-extraction protocol, so the comparison is not auditable. This is more fundamental than the reader's 'unsupported numbers' concern: it is not just that the figures are unverified, but that they are the wrong tests of the hypothesis.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript is a narrative review claiming that multimodal large language models (MLLMs) improve traffic safety relative to traditional ADAS by integrating vision, language, and environmental signals. It presents a quantitative comparison in Table 1 (accuracy, inference speed, energy, adversarial robustness), organizes MLLM applications into perception, decision-making, human-machine interaction, and safety-critical analysis, reviews several driving datasets, and concludes with future research directions. The central claim is that MLLMs provide holistic scene understanding that outperforms conventional, isolated-sensor systems.","tokens_in":11195,"tokens_out":7356,"duration_ms":63381,"significance":"The topic is timely, and the proposed taxonomy in Section 3 (perception, decision-making, interaction, safety-critical analysis) is a reasonable organizing structure for a survey. However, the manuscript's main empirical evidence is not reliable: multiple quantitative claims in Table 1 and Section 3 are attributed to papers that are not MLLM systems, do not address the stated scenarios, or do not contain the reported numbers. The paper also lacks a systematic search or extraction protocol, so the comparison is not auditable. If the quantitative comparisons were corrected or removed, the remaining narrative could serve as a position piece, but in its current form the paper does not provide a trustworthy foundation for the claim that MLLMs 'enhance' traffic safety.","major_comments":[{"comment":"The mIoU row claims an improvement from 72.3% (camera-only) to 88.6% (multimodal fusion) on KITTI under rainy conditions, but this evidence does not test the MLLM hypothesis. Source [13] is CAFuser, a condition-aware multimodal fusion network without any LLM component, and KITTI [12] has no standard rainy-condition benchmark. As written, this row neither supports the paper's central claim about MLLMs nor provides a verifiable baseline.","section":"Section 2, Table 1"},{"comment":"The adversarial robustness rows are unsupported by the cited sources. The 41%/79% accuracy figures cite [19], which studies physical adversarial attacks on an aerial-imagery object detector, not a camera-only ADAS or a LiDAR+thermal fusion system; the 89%/12% misclassification reduction cites [20], a paper on vision-language model robustness that does not report the claimed 'GPS + historical data cross-validation' experiment. These numbers therefore cannot be verified from the references given.","section":"Section 2, Table 1"},{"comment":"The claim that LLaVA-ST 'reduces localization errors by 27% on the ST-Align dataset (4.3M samples)' cites [25], which is a paper on image-gene alignment in spatial transcriptomics, not a driving dataset. Additionally, references [14], [6], and [26] all point to the same R. Zhang et al. preprint, yet the manuscript uses it for three different quantitative results (76.31% zero-shot anomaly detection, 35% improvement, and 40% latency reduction) without describing how these numbers were obtained. This is a load-bearing evidence problem for the review's empirical claims.","section":"Section 3, LLaVA-ST paragraph"},{"comment":"The '89% precision' attributed to AccidentGPT is cited to [30] without indicating whether the figure appears in that paper or is a reinterpretation. The subsequent sentence states that Gemini-Pro-Vision 1.5 and LLaVA 'reduce hallucination errors, improving zero-shot accuracy on the SHRP2 NDS dataset' and cites [35], which is a general survey on multimodal hallucination and does not report experiments on SHRP2 NDS. These unsupported causal claims should be removed or replaced with primary sources that actually contain the reported evaluations.","section":"Section 3, AccidentGPT and hallucination-fine-tuning"},{"comment":"The review does not disclose a systematic search strategy, inclusion/exclusion criteria, or a data-extraction protocol for the quantitative comparisons in Table 1 and the narrative in Section 3. Without such a protocol, the reader cannot distinguish a comprehensive, balanced survey from a selection of favorable results, and the specific numbers appear chosen to support the thesis rather than to represent the field. This undermines the paper's claim to be a 'comprehensive review'.","section":"Sections 2 and 3, overall methodology"}],"minor_comments":[{"comment":"The table heading 'Comparison of Existing Dataset for Safety Traffic' is grammatically incorrect, and the table compares ADAS and MLLM systems, not datasets; the same heading appears on Table 2.","section":"Table 1"},{"comment":"The NuScenes row lists '1K RGB images'; NuScenes actually contains 1,000 driving scenes with about 1.4 million camera images, so this entry is misleading.","section":"Table 2"},{"comment":"The 'Misclassification Rate 1K' header contains a stray '1K' that should be removed.","section":"Table 1"},{"comment":"The phrase 'different in-context learning (ICT) settings' should be 'in-context learning (ICL) settings'; the abbreviation appears to be a typo.","section":"Section 3"},{"comment":"The sentence 'we investigate other settings such as self-ensemble learning...' appears to be copied from the authors' prior work [5] and is out of place in a review; the pronoun 'we' is confusing in a survey context.","section":"Section 3, SeeUnsafe discussion"},{"comment":"Several references have malformed author lists (e.g., [27], [38], [39]) and reference [14] is a duplicate of [6] and [26]; these should be unified and formatted according to the journal's style.","section":"References"}],"recommendation":"reject","confidential_remarks":"The paper relies heavily on the authors' own preprints and self-published venues (at least seven of the 58 references), and several quantitative claims trace back to the same R. Zhang et al. preprint under three different reference numbers. The duplicated references and mismatched citations (e.g., the spatial-transcriptomics dataset cited for LLaVA-ST) suggest that the quantitative evidence was not carefully verified. If the authors wish to pursue publication, they would need to substantially revise the empirical sections or reframe the work as a position paper without unsupported quantitative comparisons."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about arXiv:2504.16134. First, it is a review of MLLMs for traffic safety with a bold claim—that MLLMs are a paradigm shift—but its load-bearing comparison (Table 1) is built on citations that do not support the numbers. Second, despite that, it is not empty: the taxonomy and dataset list are a reasonable starting point for someone new to the area.\n\nWhat the paper does well: the classification into perception, decision-making, human-machine interaction, and safety-critical analysis is a clean frame. The future directions (edge deployment, causality, human-AI, adversarial robustness) are standard talking points but not wrong. The dataset descriptions in Section 4 are useful as a quick reference.\n\nNow the soft spots, and they are serious. Table 1's claimed mIoU gain from 72.3% to 88.6% on 'KITTI rainy conditions' is attributed to CAFuser, which is a condition-aware multimodal fusion network, not an MLLM, and KITTI has no standard rainy benchmark. The adversarial robustness rows cite an aerial-imagery detector paper and a vision-language robustness paper, neither of which reports the claimed GPS/historical-data cross-validation numbers. In other words, the evidence does not test the hypothesis. The prose repeats these numbers without error bars or extraction protocol, so a reader cannot verify them. Also, the same paper (R. Zhang et al.) appears as refs [6], [14], and [26] with different claims, and the authors self-cite heavily without noting the overlap.\n\nThe conclusion calls MLLMs a 'paradigm shift'—that is an assertion, not an outcome of the review.\n\nWho should read this? Someone looking for a literature list on MLLMs in traffic safety could use it as a pointer, but only with caution. It is not a reliable review in its current state.\n\nFor peer review: I would send it back for major revision with a demand to either replace the comparative table with verifiable numbers from actual MLLM systems or delete the quantitative claims entirely. That is a fixable problem, so I would not desk-reject, but the current version is not publishable. If the authors can redo the evidence, it could become a serviceable survey.","headline":"A survey whose quantitative case is built on mismatched citations and non-MLLM systems; useful only as a literature pointer until the evidence is redone.","tokens_in":11757,"tokens_out":3861,"would_cite":false,"duration_ms":34099,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This review argues that multimodal large language models improve traffic safety by fusing vision, language, and sensor data, reporting accuracy gains and stronger defenses against adversarial attacks.","keywords":["multimodal large language models","traffic safety","advanced driver-assistance systems","adversarial robustness","autonomous driving","sensor fusion","edge deployment","contextual reasoning"],"falsifier":"A controlled replication would settle it: run the same camera-only and multimodal systems on the KITTI rainy split under identical conditions, and on an adversarial-patch test set, to see whether mIoU rises from 72.3% to 88.6% and misclassification falls from 89% to 12% as claimed.","tokens_in":10694,"feed_emoji":"🚗","tokens_out":5991,"duration_ms":47056,"temperature":0.7,"pith_summary":"This review sets out to establish that multimodal large language models (MLLMs), by integrating visual, linguistic, and environmental sensor data, can overcome the fragmented-perception and adversarial-vulnerability problems of conventional driver-assistance systems. It compares traditional ADAS with MLLM-based systems across accuracy, speed, energy, and robustness, reporting gains such as rainy-condition mIoU rising from 72.3% to 88.6% and adversarial misclassification falling from 89% to 12%. A sympathetic reader would care because these numbers point toward a concrete path from brittle single-sensor pipelines to context-aware, edge-deployable safety systems.","feed_headline":"Rainy-scene accuracy jumps from 72% to 89% with multimodal LLMs","feed_subtitle":"A review claims fusing vision, language, and sensors also cuts adversarial misclassification from 89% to 12%.","key_machinery":"The carrying mechanism is cross-modal fusion inside an LLM backbone: image, LiDAR, text, GPS, and weather signals are aligned into one representation so that scene understanding is contextual rather than per-sensor. Named systems do the work: LLaVA-ST aligns language and visual tokens in space and time for trajectory prediction; SeeUnsafe uses severity-based aggregation for interactive video analysis; AccidentGPT fuses accident reports with real-time sensor streams to flag collision-prone areas. The shared mechanism is that each modality can validate or disambiguate the others, which is what the paper credits for both accuracy gains and adversarial robustness.","core_discovery":"The paper's central claim is that MLLMs are a viable upgrade path for traffic safety, not just a research curiosity. It assembles comparative evidence that multimodal fusion and contextual reasoning beat isolated sensor processing: on KITTI rainy scenes, mIoU rises from 72.3% with camera-only input to 88.6% with multimodal fusion; near-miss anomaly detection reaches 76.31% accuracy versus 41.31% for rule-based baselines; adversarial misclassification falls from 89% to 12% when GPS and historical data are cross-referenced; and quantized MLLMs run at 25 FPS on edge devices, comparable to lightweight CNNs. The review also classifies MLLM applications into perception, decision-making, human interaction, and safety-critical analysis, and identifies datasets (KITTI, DRAMA, SHRP2 NDS, ML4RoadSafety, among others) that enable such systems.","pith_inferences":["If the reported gains come from cross-modal validation, then synchronized multi-sensor attacks — perturbing camera and LiDAR feeds together — become the critical next test, since the cited evidence only covers single-modal attacks.","The numbers as presented argue for a standardized public benchmark of MLLM traffic-safety systems, with fixed attacks, weather conditions, and metrics, before any of these figures can guide deployment decisions.","The paper's emphasis on hallucination reduction implies that false-positive hazard warnings, not just missed detections, should be a primary safety metric, because a system that cries wolf erodes driver trust.","Causality-driven reasoning is the natural extension of the paper's argument, but it would require new datasets annotated with causal structure, which none of the reviewed datasets currently provide."],"forward_implications":["Physical adversarial patches on road signs become far less effective: with cross-modal validation the reported misclassification rate drops from 89% to 12%, directly removing a known failure mode of camera-only detectors.","MLLMs can be deployed in vehicles rather than only in the cloud: quantized variants reach 25 FPS on edge hardware, comparable to lightweight convolutional baselines.","Interactive accident analysis becomes practical: MLLMs on the DRAMA dataset cut post-processing latency by 40% while answering natural-language queries about near-miss events.","Risk prediction becomes proactive: AccidentGPT-style systems report 89% precision in identifying collision-prone areas, enabling intervention before incidents occur.","Zero-shot capabilities reduce data dependence: near-miss detection at 76.31% accuracy works without task-specific training data, easing adoption in new traffic environments."],"supporting_citations":[{"why":"Supplies the camera-only baseline: KITTI dataset, cited for 72.3% mIoU in rainy conditions.","marker":"[12]"},{"why":"Supplies the multimodal result: condition-aware fusion raising mIoU to 88.6%.","marker":"[13]"},{"why":"Supplies the near-miss anomaly detection figures (76.31% vs 41.31% rule-based).","marker":"[14]"},{"why":"Supplies the edge-deployment claim: MobileVLM reaching 25 FPS after quantization.","marker":"[16]"},{"why":"Supplies the lightweight ADAS comparison point: MobileNet at 30 FPS.","marker":"[17]"},{"why":"Supplies the adversarial-robustness claim: misclassification dropping from 89% to 12% with cross-modal validation.","marker":"[20]"},{"why":"Supplies AccidentGPT's 89% precision in collision-prone area identification.","marker":"[30]"},{"why":"Supplies the DRAMA dataset used for interactive QA-based traffic video analysis.","marker":"[27]"},{"why":"Supplies the ML4RoadSafety dataset (9M accident records) used for risk prediction.","marker":"[40]"}],"fun_headline_variants":["Near-miss detection accuracy nearly doubles with multimodal LLMs","Adversarial misclassification drops from 89% to 12% with MLLMs","Quantized MLLMs run at 25 FPS on edge devices for safety","Multimodal LLMs boost near-miss detection from 41% to 76%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The review's comparative verdict rests on the assumption that the cited performance figures — 72.3% to 88.6% mIoU, 89% to 12% misclassification, 35% faster responses, 89% AccidentGPT precision — actually come from the cited studies and are reproducible, yet the text gives no experimental protocols, error bars, or primary-source verification for these numbers.","fun_headline_variants_meta":{"raw":{"variants":["Near-miss detection accuracy nearly doubles with multimodal LLMs","Adversarial misclassification drops from 89% to 12% with MLLMs","Quantized MLLMs run at 25 FPS on edge devices for safety","Multimodal LLMs boost near-miss detection from 41% to 76%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000752,"raw_usage":{"total_tokens":3331,"prompt_tokens":916,"completion_tokens":2415,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":532,"completion_tokens_details":{"reasoning_tokens":2329}},"tokens_in":532,"tokens_out":2415,"duration_ms":16024,"temperature":1.0,"reasoning_tokens":2329,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:26:33.082052+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled replication would settle it: run the same camera-only and multimodal systems on the KITTI rainy split under identical conditions, and on an adversarial-patch test set, to see whether mIoU rises from 72.3% to 88.6% and misclassification falls from 89% to 12% as claimed.","supporting_citations":[{"cited_title":"MobileVLM V2: Faster and Stronger Baseline for Vision Language Model,","cited_arxiv_id":null,"evidence_quote":"Supplies the edge-deployment claim: MobileVLM reaching 25 FPS after quantization."},{"cited_title":"MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,","cited_arxiv_id":null,"evidence_quote":"Supplies the lightweight ADAS comparison point: MobileNet at 30 FPS."},{"cited_title":"DRAMA: Joint Risk Localization and Captioning in Driving,","cited_arxiv_id":null,"evidence_quote":"Supplies the DRAMA dataset used for interactive QA-based traffic video analysis."},{"cited_title":"Graph neural networks for road safety modeling: datasets and evaluations for accident analysis,","cited_arxiv_id":null,"evidence_quote":"Supplies the ML4RoadSafety dataset (9M accident records) used for risk prediction."}],"review_version":1}