{"id":"5690898b-147a-433e-a483-8f4ce4134963","arxiv_id":"2412.07247","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Fine-tuning InternVL-1.5 on concatenated six-camera images with SAM-generated bounding boxes achieved 0.6002 and won the driving language challenge.","lead":"A team fine-tuned the open-source vision-language model InternVL-1.5 on driving question-answer data and won the Driving with Language track of the CVPR 2024 Autonomous Grand Challenge with a score of 0.6002. The report shows how image formatting and automatic box annotation let a general multimodal model perform well on autonomous-driving perception and reasoning tasks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported 0.6002 score is not independently checkable, and the largest-SAM-mask annotation assumption may add label noise that the paper never validates; a concrete reproducibility or analysis check is needed before the claim can be accepted.","rationale":"The reader's verdict was CONDITIONAL, and the weakest assumption identified was exactly the SAM-based bounding-box annotation pipeline. I independently locate the same concern in Section 2 and agree that it is the most load-bearing unvalidated assumption: the final score depends on these labels during full-parameter fine-tuning, and the paper provides no annotation-quality check or ablation. The claim is not internally inconsistent; it is just under-supported by the report. The paper gives real credit for using an external benchmark, which prevents circularity, and for describing the method clearly enough to reimplement approximately. However, the missing artifacts and missing validation justify keeping the verdict CONDITIONAL rather than ACCEPT or REJECT. My concrete test targets the annotation quality and reproducibility directly, matching the reader's recommendation to release artifacts and add ablations. No stronger, more specific weakness exists in this short technical report; the temporal-fusion section is admittedly preliminary, and the v1/v2 ensemble comment is only a passing observation, not the central claim. Honest non-finding does not apply because a concrete, load-bearing concern exists and is testable.","tokens_in":3109,"tokens_out":1491,"duration_ms":13172,"concrete_test":"Request the released code/weights and annotation outputs, or run the following independent check: take the DriveLM center-point annotations for a held-out subset, run the described SAM largest-mask pipeline, and compare the resulting boxes against human-annotated bounding boxes (when available in nuScenes). Compute mean IoU and the fraction of boxes with IoU < 0.5; if that fraction is small, the annotation concern is mitigated. In addition, re-fine-tune InternVL-1.5 with the released configuration on the training split and recompute the DriveLM leaderboard score; if the score cannot be reproduced within a small tolerance, the headline result should be treated as unverified.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central empirical claim is the leaderboard score of 0.6002 (Accuracy 0.7339, ChatGPT 65.2512) for InternVL4Drive-v2. The report provides no code, weights, annotation scripts, leaderboard link, or frame-level result dump, so the score cannot be independently verified or decomposed. The score itself is plausible for a strong open-source VLM fine-tuned on DriveLM, but the automatic annotation pipeline is the most load-bearing methodological assumption: Section 2 converts each object's center point into a bounding box by taking the largest SAM mask and deriving box coordinates from it. This assumes the center point lies on the object's main body and that the largest mask covers the complete object. The authors explicitly acknowledge failures for traffic lights, and they provide no validation of annotation quality (e.g., IoU against ground truth, human review, or ablation). Because DriveLM gates reasoning questions on correct perception, corrupted boxes could directly lower accuracy or ChatGPT scores; alternatively, if the boxes are almost always correct, the claim stands. The paper also lacks an ablation isolating the concatenation and bounding-box contributions, and the temporal model's failure is attributed to 'data format issues' without diagnosis. These omissions do not make the claim false, but they make it conditional on unpublished implementation details and unmeasured label quality.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports the authors' winning submission to the Driving with Language track of the CVPR 2024 Autonomous Grand Challenge. The method is based on InternVL-1.5, which is fine-tuned with full parameters on the DriveLM-nuScenes dataset. To handle the six surround-view camera images, the authors concatenate them into a 2x3 grid with orientation labels and resize the composite image. They also replace the center-point object representation in DriveLM with bounding boxes obtained by prompting Segment Anything with the center point and taking the largest resulting mask. Their best single model, InternVL4Drive-v2, is reported to achieve a final score of 0.6002 on the competition leaderboard, with Accuracy 0.7339 and ChatGPT score 65.2512, outperforming their v1 model. A preliminary temporal variant scored lower and is attributed to data format issues.","tokens_in":3294,"tokens_out":3147,"duration_ms":33208,"significance":"If the reported leaderboard score is accurate, the paper demonstrates that a general open-source vision-language model, combined with simple preprocessing and automatic annotation, can achieve a leading result on a driving-language benchmark. This is an interesting empirical data point for the community. The paper also honestly acknowledges the limitation of the SAM-based annotation pipeline for cases where the center point is not on the object's main body. However, the scientific significance is limited by the absence of reproducibility artifacts, ablation studies, and validation of the automatic annotation quality. The central claim is an externally evaluated competition score, but the paper does not provide enough evidence to verify the method's contribution beyond that score.","major_comments":[{"comment":"The automatic annotation strategy is a load-bearing component of the method, but its quality is not validated. The paper states that the largest SAM mask is consistently selected and that this \"works well in most cases,\" while acknowledging that traffic lights can produce incorrect boxes. Because the competition scoring gates reasoning questions on correct perception (as stated in the Introduction), corrupted bounding boxes could directly lower the reported accuracy and ChatGPT scores. The authors should provide quantitative validation of the annotation pipeline, for example by measuring IoU against a human-annotated subset, presenting a few qualitative examples, or performing an ablation that replaces the SAM boxes with the original center points or with manually corrected boxes.","section":"Section 2"},{"comment":"The paper attributes the good performance to the concatenation of multi-view images and the replacement of center points with bounding boxes, but no baseline or ablation is provided. Without comparing InternVL4Drive-v2 to a model trained on the original center-point representation, or to a model that processes the six views without concatenation, the reader cannot assess whether these design choices actually contribute to the reported score. The central claim of the paper is the leaderboard result, but the methodological claims are unsupported without these controls.","section":"Section 3"},{"comment":"The temporal model InternVL4Drive-T is reported to have a much lower final score (0.4600) and the failure is attributed to \"data format issues,\" but no diagnosis is provided. Including this result without any explanation of what the data format issue was, or how it was detected, does not advance the reader's understanding and weakens the credibility of the experimental section. Either the issue should be analyzed and resolved in the paper, or the temporal experiment should be clearly labeled as an inconclusive preliminary exploration.","section":"Table 2"},{"comment":"The manuscript provides no code, checkpoints, annotation scripts, leaderboard link, or frame-level result dump. The reported score of 0.6002 therefore cannot be independently verified or decomposed from the manuscript alone. For a journal publication, the authors should make available the essential implementation details and, if possible, the exact evaluation results or a link to the public leaderboard entry.","section":"General reproducibility"}],"minor_comments":[{"comment":"The title contains a typo: \"Oustanding\" should be \"Outstanding,\" and the abstract uses \"leadboard\" instead of \"leaderboard.\"","section":"Title and Abstract"},{"comment":"The sentence \"Based in this sub-dataset, our model actually achieves all higher score except on the ChatGPT score\" is grammatically unclear; it likely means \"Based on this sub-dataset, our model achieves higher scores on all metrics except the ChatGPT score.\"","section":"Section 4"},{"comment":"The heading \"T emporal F usion\" contains stray spaces and the section would benefit from a clearer separation of the temporal experiment from the main single-frame method.","section":"Section 3"},{"comment":"Figures 1 and 2 are not referenced in the text; the authors should explicitly refer to them in Sections 3 and 4 where the architecture and the concatenated image are described.","section":"Figures"}],"recommendation":"major_revision","confidential_remarks":"This manuscript is a competition technical report rather than a full research paper. Its main claim is a leaderboard score, which is externally validated but not independently verifiable from the text alone. The lack of ablations and annotation-quality validation makes the methodological claims conditional. If the journal does not normally publish short competition reports, the editor may consider whether the paper offers sufficient scientific contribution beyond the reported score. The authors should be encouraged to either add the missing validation and ablations or reframe the paper as a brief empirical note."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a short competition report, not a research contribution. What's new is the specific recipe — full fine-tuning of InternVL-1.5 on DriveLM with a 2x3 concatenated multi-view grid and SAM-derived bounding boxes — and the resulting leaderboard score of 0.6002. That score is the paper's only load-bearing claim, and it is externally measured on the DriveLM leaderboard, so it is not manufactured. I find the score plausible, though I cannot independently check it because the authors provide no code, weights, annotation scripts, or leaderboard link.\n\nWhat the paper does well: it is clear about what was done, gives the system prompt and key training details, and honestly flags a known failure mode of the automatic annotation for traffic lights. The citations to InternVL, DriveLM, SAM, and nuScenes are appropriate, and the writing is straightforward.\n\nSoft spots, in proportion to their seriousness: no artifacts are released, so the result is not reproducible as-is. There is no ablation isolating the two claimed contributions (the concatenation format and the bounding-box conversion), leaving the reader to guess which part matters. The temporal model's failure is attributed to \"data format issues\" with no diagnosis. The largest-SAM-mask assumption is the most load-bearing methodological choice; if it corrupts a meaningful fraction of training boxes, the perception-gated reasoning questions could be hurt. The authors acknowledge this limitation, but they provide no validation of annotation quality. These omissions are real but not fatal for a competition report: the score stands as an empirical result, and the method is described well enough for approximate reimplementation.\n\nWho this is for: challenge participants who want a strong practical baseline, and researchers interested in how a general VLM can be fine-tuned for driving QA. I would bring it to a reading group as an example of a clean engineering solution.\n\nRecommendation: this paper deserves a serious referee, not a desk reject, but I would make the decision conditional on artifact release. It is a legitimate champion report, and the claims are specific enough to verify. Just do not treat it as a methodology advance.","headline":"A solid, honestly-written competition report whose score is plausible but unverifiable; the SAM-box conversion is the only load-bearing assumption and it is acknowledged, not validated.","tokens_in":3874,"tokens_out":1869,"would_cite":true,"duration_ms":20542,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a general, open-source vision-language model, InternVL-1.5, wins the DriveLM driving-with-language track after full-parameter fine-tuning on multi-view concatenated images and SAM-generated bounding boxes, achieving…","keywords":["autonomous driving","vision-language model","DriveLM","InternVL-1.5","multi-view image concatenation","bounding box grounding","Segment Anything annotation","CVPR 2024 Autonomous Grand Challenge"],"falsifier":"Take a random sample of the training images, draw the SAM-derived bounding boxes for objects whose center points are on traffic lights and other small or partially occluded objects, and compare them with human-labeled boxes; if a substantial fraction have near-zero IoU, the reported score cannot be credited to the claimed clean perception labels, and replacing those boxes should change accuracy.","tokens_in":2839,"feed_emoji":"🚗","tokens_out":8388,"duration_ms":79355,"temperature":0.7,"pith_summary":"At its core, the paper tries to show that specialized driving-task architectures are not required to win a driving-with-language benchmark. A general open-source multimodal model, InternVL-1.5, is full-parameter fine-tuned on the DriveLM-nuScenes training set, with the six surround-view images rearranged into one labeled 2×3 composite. An automatic pipeline uses the Segment Anything model to turn object center points into bounding boxes, giving the language model a grounding-friendly visual representation. The resulting single model scores 0.6002 on the official leaderboard, with 0.7339 accuracy and a ChatGPT evaluation score of 65.2512. If this holds, it suggests that input formatting and label representation can matter as much as model architecture in multimodal driving QA.","feed_headline":"General VLM tops DriveLM at 0.6002 with stacked views","feed_subtitle":"Full fine-tuning of InternVL-1.5 on six labeled views with Segment Anything-derived boxes yields the top score.","key_machinery":"The load-bearing machinery is a 2-by-3 grid of resized 896x448 camera frames, each prefixed with a text label of the camera direction and then concatenated into one 2688x896 image; the grid is split into twelve 448x448 tiles plus a thumbnail and fed through InternVL. The second mechanism is the annotation conversion: a DriveLM center point is used as a point prompt for Segment Anything, the largest sampled mask is taken as the object, and its bounding box is computed and re-normalized to InternVL's integer 0-1000 coordinate space. The argument that this works rests on the model's pretrained ability to ground language in boxes, so the input format lets the model inherit that capability.","core_discovery":"The paper's central claim is that a general-purpose vision-language model can dominate an autonomous-driving reasoning benchmark without any task-specific perception head. The authors take InternVL-1.5 and fine-tune it end-to-end on DriveLM-nuScenes, feeding it a single concatenated image made from the six camera views, each labeled with its direction (FRONT LEFT, FRONT, FRONT RIGHT, BACK LEFT, BACK, BACK RIGHT). They replace the dataset's center-point object representation with bounding boxes extracted by prompting Segment Anything from the center point and taking the largest mask, then normalize coordinates to InternVL's 0-1000 token format. Their single best model, InternVL4Drive-v2, achieves a final leaderboard score of 0.6002; a version trained on only about 10% of the data already reaches 0.5862, and the authors state that ensembling the two would score higher.","pith_inferences":["A natural next experiment, not run in the paper, is to hand-validate the Segment Anything boxes against human annotations; if traffic-light boxes are bad, fixing those labels could push the ChatGPT and Match scores above the reported values.","The same recipe of labeled multi-view concatenation plus prompt-based box grounding could transfer to other multi-camera visual question answering benchmarks, since it does not depend on task-specific architecture.","The accuracy drop from v1 to v2 while the ChatGPT score rose suggests the full-data fine-tune trades some exact perception for more fluent or preferred answers; splitting the evaluation by question type would reveal where the trade happens."],"forward_implications":["Under the paper's claim, a task-general open vision-language model with simple input reformatting is enough to outperform purpose-built driving reasoning systems on the competition's metric.","Because the benchmark gates reasoning questions on correct perception, the SAM-derived bounding boxes directly determine how much reasoning score the model can reach, so better boxes should yield higher scores.","The paper's ensemble observation implies the final score can be raised further without changing the architecture, since the two versions have complementary accuracy and ChatGPT behavior.","The failed temporal variant shows that simply adding the previous keyframe does not exploit temporal context in this setup, leaving temporal fusion an open problem."],"supporting_citations":[{"why":"Supplies the DriveLM-nuScenes dataset, the question format with center-point object references, and the competition's evaluation protocol.","marker":"[SRC+23]"},{"why":"Supplies the nuScenes multi-view camera images and temporal keyframes that the model is trained on.","marker":"[CBL+20]"},{"why":"Defines InternVL-1.5, the pretrained architecture whose visual grounding and dynamic-resolution tiling are inherited by the fine-tuned model.","marker":"[CWW+23]"},{"why":"Provides the Segment Anything model used to convert center points into bounding-box training labels.","marker":"[KMR+23]"}],"fun_headline_variants":["InternVL tops DriveLM with stacked views and auto boxes","General VLM wins CVPR driving track at 0.6002","Fine-tuned InternVL-1.5 hits 0.6002 on DriveLM","Single model, no perception head, tops DriveLM at 0.6002","Auto-annotated boxes power InternVL to 0.6002 in drive"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The automatic labeling step assumes each dataset center point lies on the object's main body, so the largest automatically generated mask is the whole object and its bounding box is a clean training label, an assumption the paper itself notes fails for traffic lights.","fun_headline_variants_meta":{"raw":{"variants":["InternVL tops DriveLM with stacked views and auto boxes","General VLM wins CVPR driving track at 0.6002","Fine-tuned InternVL-1.5 hits 0.6002 on DriveLM","Single model, no perception head, tops DriveLM at 0.6002","Auto-annotated boxes power InternVL to 0.6002 in drive"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000285,"raw_usage":{"total_tokens":1653,"prompt_tokens":895,"completion_tokens":758,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":511,"completion_tokens_details":{"reasoning_tokens":657}},"tokens_in":511,"tokens_out":758,"duration_ms":8335,"temperature":1.0,"reasoning_tokens":657,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:56:15.386461+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of the training images, draw the SAM-derived bounding boxes for objects whose center points are on traffic lights and other small or partially occluded objects, and compare them with human-labeled boxes; if a substantial fraction have near-zero IoU, the reported score cannot be credited to the claimed clean perception labels, and replacing those boxes should change accuracy.","supporting_citations":[],"review_version":1}