{"id":"0f3ffa0d-47c7-4ecd-a01f-78521f78afc0","arxiv_id":"2501.05081","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"Fine-tuning Mini-InternVL on DriveLM yields a CIDEr of 0.191 on driving QA, but the paper is a thin, under-specified application of prior work.","lead":"This workshop paper applies a small vision-language model, Mini-InternVL, to autonomous driving by fine-tuning it on the DriveLM dataset and reports benchmark scores. It shows a 2B-parameter model can reach moderate performance, but it provides little experimental detail and appears to present an existing model as its own contribution.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed 'excellent' DriveLM result is uninterpretable without a zero-shot baseline and the exact domain-mixing recipe; neither is reported, so the central claim is unsupported.","rationale":"The reader's rejection is well-founded. The paper's central evidence is a CIDEr score that is not contextualized by any baseline measurement, not accompanied by a reproducible recipe, and not supported by ablations or release of code. My stress test sharpens this into a single testable objection: the claimed contribution of domain adaptation cannot be separated from the base model's prior capabilities because no zero-shot evaluation is reported. This is an evidential gap, not a disagreement with consensus; the paper even presents Mini-InternVL as its own introduction even though it is an existing open-source model, which further weakens provenance but is secondary to the missing baseline. One concrete run of the original Mini-InternVL-2B on the val split would largely settle the matter. I therefore see no reason to change the reader's verdict.","tokens_in":5680,"tokens_out":2793,"duration_ms":28856,"concrete_test":"Run two evaluations on DriveLM-nuScenes 1.1 val with identical VQA prompts: (a) the original Mini-InternVL-2B checkpoint with no DriveLM fine-tuning, and (b) a fine-tuned model trained with the exact data ratio and hyperparameters if those are supplied. If zero-shot CIDEr is within 0.01 of 0.191, the adaptation adds nothing and the central claim fails; if it is materially lower, the absence of a reported baseline and recipe still leaves the result unverified until the general-data ratio and training configuration are published and an ablation reproduces CIDEr 0.191.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that a 2B model, adapted by reformatting DriveLM into VQA and mixing in general data, achieves competitive performance (CIDEr 0.191). The evidence in Section 3.3 is a single table row with no error bars, no evaluation protocol, and no pre-adaptation number. The weakest load-bearing assumption is in Section 3.2: that adding an undocumented 'balanced proportion of general multimodal data' to the domain data is sufficient domain transfer. Because the mixture ratio, prompt formatting, training hyperparameters, and checkpoint selection are all unspecified, CIDEr 0.191 cannot be separated from the base model's prior knowledge or from arbitrary implementation choices. In particular, Mini-InternVL is an existing open-source model; without a zero-shot run of the original checkpoint on DriveLM-val, one cannot tell whether fine-tuning contributes anything, and without ablations over the general-data ratio the recipe is not even defined. The result therefore does not establish the claimed generalization.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"Section 3.3 reports that a 2B-parameter Mini-InternVL model fine-tuned on DriveLM-nuScenes with a VQA reformatting and a mixture of general multimodal data achieves CIDEr 0.191 on DriveLM-nuScenes-val, which the authors call an excellent domain adaptation result. The paper describes the Mini-InternVL architecture, a general recipe for reformatting tasks into VQA format, and a table comparing the adapted models with MTMM, Team NVIDIA, and MMFM_AD. The title announces DriVLM, but the body and conclusion consistently refer to 'Mini-InternVL' as the introduced model.","tokens_in":6010,"tokens_out":5623,"duration_ms":51810,"significance":"If substantiated, the result would be notable: a small-scale open MLLM, adapted with a simple data-reformatting recipe, performing competitively on DriveLM would support the viability of low-resource domain adaptation for autonomous driving. However, the manuscript does not provide the evidence needed: no zero-shot baseline, no training hyperparameters, no mixing ratio, no ablations, no code or checkpoints, and no details of the evaluation split or metric computation. The paper also appears to claim credit for Mini-InternVL, an existing model. As submitted, the contribution cannot be independently verified, so the significance of the central claim is currently unestablished.","major_comments":[{"comment":"The domain-adaptation recipe is not specified: the text says 'integrating a balanced proportion of general multimodal data' but never gives the ratio or a procedure for choosing it, and no ablation over this mixture is reported. Because the amount of general data mixed into the DriveLM fine-tuning is a free parameter on which the claimed generalization depends, CIDEr 0.191 is not reproducible and cannot be attributed to the method.","section":"Section 3.2"},{"comment":"The central result is a single table row with no standard deviations, no number of runs, no training hyperparameters, no checkpoint selection rule, and no description of the evaluation protocol (e.g., which metric implementation, prompt template, or generation settings). More importantly, no zero-shot evaluation of the original Mini-InternVL-2B checkpoint on DriveLM-nuScenes-val is reported, so the reader cannot tell whether fine-tuning added anything over the base model's prior knowledge.","section":"Section 3.3"},{"comment":"The comparison in Table 1 does not support the label 'excellent.' The prior systems MTMM, Team NVIDIA, and MMFM_AD are listed without citations or descriptions, metrics are missing for some rows, and there is no statement that all entries use the same validation split, prompt format, and evaluation code. Without this context, the 0.191 CIDEr score is not interpretable as a competitive result.","section":"Section 3.3"},{"comment":"The manuscript's model identity is inconsistent: the title introduces DriVLM, but the body (Section 3.1) and conclusion say 'we introduce Mini-InternVL,' which is an existing open-source model family. The paper never defines what DriVLM is or how it differs from Mini-InternVL, making it unclear what the actual contribution is. This is a load-bearing issue because the central claim is about the adapted model's performance.","section":"Title and Section 4"}],"minor_comments":[{"comment":"References [1] and [14] are the same CLIP citation; [14] should be removed or replaced with the intended source.","section":"References"},{"comment":"Several reference entries contain stray page numbers (e.g., '[7] ... 1, 3, 8' and '[15] ... 2, 3'), apparently citation-manager artifacts.","section":"References"},{"comment":"The abstract is generic and does not mention the experimental setup, the DriveLM dataset, or the reported CIDEr score, making it impossible to judge the paper's contribution from the abstract.","section":"Abstract"},{"comment":"Section 3.2 never gives a concrete example of the VQA prompt used for DriveLM, such as the exact formatting of camera labels or the question templates; adding one would materially improve reproducibility.","section":"Section 3.2"}],"recommendation":"reject","confidential_remarks":"The manuscript is more of a workshop abstract than a complete research paper. The missing pieces are not cosmetic: a zero-shot baseline, the data-mixing ratio, ablations, and training details are required before the central claim can even be tested. The recurring statement that the paper 'introduces Mini-InternVL' is also problematic, since Mini-InternVL is an existing open-source model; if the authors' contribution is the domain-adapted variant, the paper must be rewritten to say so. On these grounds I recommend rejection, though I would look favorably on a substantially expanded revision with the missing evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a thin workshop write-up that takes the existing Mini-InternVL model (OpenGVLab's work, uncited) and fine-tunes it on DriveLM. The only new numbers are two rows in Table 1. The rest of the paper re-describes Mini-InternVL's architecture and two-stage training as if it were the authors' own contribution. The conclusion literally says 'we introduce Mini-InternVL.' That is a provenance problem, not a style problem.\n\nWhat's worth keeping: the paper does call attention to a genuinely useful data point — a 2B-parameter model can reach CIDEr 0.191 on DriveLM-nuScenes-val with a simple VQA-reformatting recipe, which is cheap compared to larger driving MLLMs. If the recipe were fully specified and the base checkpoint clearly identified, this would be a useful benchmark entry for the small-MLLM-for-driving crowd.\n\nWhere it falls apart: reproducibility. Section 3.3 gives exactly one table row per model, with no standard deviations, no training hyperparameters, no data split details, and no code. Section 3.2 mentions mixing 'a balanced proportion of general multimodal data' but doesn't say what that proportion is. Without that ratio, without the prompt templates, and without a zero-shot run of the original Mini-InternVL checkpoint on DriveLM-val, the 0.191 CIDEr is not interpretable as a domain-adaptation result — it could be mostly the base model's prior knowledge. The paper's own limitation sentence about 'constraints of the training data and evaluation metrics' is honest but doesn't fix the missing details.\n\nThe comparison table is also shaky: MTMM and Team NVIDIA numbers are taken from other papers with no protocol alignment, and the paper doesn't say whether its evaluation matches theirs.\n\nNet: the central claim ('excellent result') is unsupported as written. The paper is not ready for peer review in its current form; a serious editor would desk-reject it on the grounds that the method is prior art and the single new number is unverifiable. A revision could add the missing details, cite Mini-InternVL properly, and rename the model to DriVLM consistently — then it might be a legitimate workshop contribution. As is, I wouldn't cite it and wouldn't send it to referees.","headline":"Applies someone else's open-source model to DriveLM without citing it, and the single reported number is unverifiable as written; a desk reject rather than a referee assignment.","tokens_in":6383,"tokens_out":2520,"would_cite":false,"duration_ms":23024,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 2B-parameter vision-language model, adapted to autonomous driving purely by reformatting tasks as visual questions, achieves a 0.191 CIDEr score on the DriveLM benchmark.","keywords":["domain adaptation","vision-language models","small-scale multimodal LLMs","autonomous driving","visual question answering","instruction tuning","DriveLM","CIDEr evaluation"],"falsifier":"A reader could settle the claim by fine-tuning Mini-InternVL-2B on DriveLM in three variants: with the full VQA reformatting, with plain caption-style prompts, and with no general-data mixing; if the 0.191 CIDEr changes by a large margin across variants, the reported score is an artifact of undocumented recipe choices rather than evidence for the unified transfer scheme.","tokens_in":5489,"feed_emoji":"🚗","tokens_out":9141,"duration_ms":83857,"temperature":0.7,"pith_summary":"The paper sets out to show that a small-scale multimodal language model can be moved into autonomous driving without heavy architectural surgery, by re-expressing perception, prediction, and planning tasks as visual question answering and fine-tuning all parameters. It reports that the 2B-parameter Mini-InternVL variant scores 0.191 CIDEr, 0.763 accuracy, and 0.762 BLEU-1 on the DriveLM-nuScenes 1.1 validation set, a result the paper takes as evidence of successful domain adaptation. The 1B variant also adapts, reaching 0.167 CIDEr. The practical stake is that researchers and developers with limited compute can still apply multimodal language models to driving, provided the base model has a strong small vision encoder and a unified prompt-level transfer format.","feed_headline":"2B vision-language model scores 0.191 CIDEr on DriveLM","feed_subtitle":"Reformatting driving tasks as visual questions lets a 2B model match heavier specialist systems.","key_machinery":"The machinery is the VQA-style instruction-tuning format combined with the Mini-InternVL architecture. Classification becomes multiple-choice questions; localization uses <ref></ref> name tags and <box></box> coordinate tags; region-aware tasks put a box around the target region; and multi-view driving images are resized into 448x448 tiles, given a thumbnail for global context, and labeled with camera positions. The model itself is a compact vision-language stack whose 300M vision encoder is initialized from a web-scale image-text encoder and improved by distillation, connected through an MLP projector to a small language model, then fully fine-tuned on a mix of driving and general multimodal data. This unified reformatting is what carries the domain transfer, because it lets one training procedure handle perception, prediction, and planning without per-task architecture changes.","core_discovery":"On its own terms, the paper's central claim is that Mini-InternVL, a family of 1B, 2B, and 4B parameter multimodal models built from an InternViT-300M encoder, an MLP projector, and small pre-trained language backbones, transfers to autonomous driving through a single recipe: reformat every driving task into VQA prompts, give the model multi-view images with camera labels and dynamic resolution, and fully fine-tune with a balanced mix of general multimodal data. Evaluated on the DriveLM-nuScenes 1.1 validation split, the 2B model scores 0.763 accuracy, 0.762 BLEU-1, and 0.191 CIDEr, matching the 0.18 CIDEr of the listed MTMM entry and exceeding the accuracy of the listed MMFM_AD entry; the 1B variant reaches 0.167 CIDEr. The paper takes this as evidence that small-scale MLLMs can be competitive in a specialized domain when equipped with a strong small vision encoder and a unified instruction-tuning format.","pith_inferences":["My extension: hold the 2B checkpoint fixed and vary only the general-to-driving data ratio; a wide plateau of ratios reproducing 0.191 CIDEr would show the recipe is robust, while a sharp peak would mean the score depends on an undocumented setting.","My extension: the same VQA reformatting could be carried to other ego-centric domains such as robotics manipulation or navigation, where a single prompt scheme would avoid per-task architectural work.","My extension: if the scaling trend holds, the 4B Mini-InternVL variant, which the paper does not evaluate on DriveLM, should push CIDEr above 0.191 under the same fine-tuning setup; checking this would test the generality of the claimed adaptation."],"forward_implications":["The reported CIDEr of 0.191 for the 2B model tops the 0.18 CIDEr of the listed MTMM entry, suggesting a small model can match or exceed a specialized baseline on DriveLM.","The 1B variant also adapts successfully with 0.701 accuracy, 0.736 BLEU-1, and 0.167 CIDEr, indicating the recipe is not tied to one model size.","Because the transfer scheme is just prompt reformatting plus full fine-tuning, the same training run can cover perception, prediction, and planning without changing the model architecture.","The small scale and open-source framing lower the compute barrier to applying multimodal language models in autonomous driving research.","Mixing general multimodal data with driving data during fine-tuning is presented as necessary to preserve generalization while gaining domain-specific performance."],"supporting_citations":[{"why":"Supplies the web-scale image-text pretraining from which the paper's small vision encoder is initialized, establishing the encoder domain-gap problem the paper addresses.","marker":"[1]"},{"why":"Provides the small pre-trained language backbone used in the 1B Mini-InternVL variant, so it is load-bearing for the model composition.","marker":"[3]"},{"why":"Provides the language backbone for the larger Mini-InternVL variants, including the 2B model whose results carry the central claim.","marker":"[4]"},{"why":"General multimodal benchmark used to show that Mini-InternVL already performs well before driving-specific adaptation.","marker":"[5]"},{"why":"Example of domain-adapted vision-language modeling in remote sensing, against which the paper positions its unified transfer approach.","marker":"[16]"},{"why":"Example of domain-adapted vision-language modeling in medical imaging, used to illustrate that prior approaches need custom data and training procedures.","marker":"[17]"},{"why":"Representative autonomous-driving vision-language model that the paper contrasts with its architecture-free VQA reformatting approach.","marker":"[18]"}],"fun_headline_variants":["Small 2B VLM matches heavyweights on DriveLM","Tiny 2B model rivals larger MLLMs in driving","Reformatted as VQA, 2B VLM hits 0.191 CIDEr","Mini-InternVL 2B matches MTMM on DriveLM","Small-scale MLLMs go autonomous with VQA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that re-expressing all driving tasks as VQA prompts and fine-tuning with an unstated balanced mix of general and driving data is sufficient for domain transfer: nothing in the paper isolates the effect of the prompt format, the data ratio, or the base checkpoint.","fun_headline_variants_meta":{"raw":{"variants":["Small 2B VLM matches heavyweights on DriveLM","Tiny 2B model rivals larger MLLMs in driving","Reformatted as VQA, 2B VLM hits 0.191 CIDEr","Mini-InternVL 2B matches MTMM on DriveLM","Small-scale MLLMs go autonomous with VQA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00069,"raw_usage":{"total_tokens":3095,"prompt_tokens":882,"completion_tokens":2213,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":498,"completion_tokens_details":{"reasoning_tokens":2134}},"tokens_in":498,"tokens_out":2213,"duration_ms":16863,"temperature":1.0,"reasoning_tokens":2134,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:18:29.702660+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader could settle the claim by fine-tuning Mini-InternVL-2B on DriveLM in three variants: with the full VQA reformatting, with plain caption-style prompts, and with no general-data mixing; if the 0.191 CIDEr changes by a large margin across variants, the reported score is an artifact of undocumented recipe choices rather than evidence for the unified transfer scheme.","supporting_citations":[{"cited_title":"Learning transferable visual models f rom natural language supervision","cited_arxiv_id":null,"evidence_quote":"Supplies the web-scale image-text pretraining from which the paper's small vision encoder is initialized, establishing the encoder domain-gap problem the paper addresses."},{"cited_title":"Qwen 2 technical report","cited_arxiv_id":null,"evidence_quote":"Provides the small pre-trained language backbone used in the 1B Mini-InternVL variant, so it is load-bearing for the model composition."},{"cited_title":"Danish, Muzamm al Naseer, Abhijit Das, Salman Khan, and Fahad S","cited_arxiv_id":null,"evidence_quote":"Example of domain-adapted vision-language modeling in remote sensing, against which the paper positions its unified transfer approach."}],"review_version":1}