{"id":"9ff96aae-a12b-40f1-aad2-070be35616fe","arxiv_id":"2608.09333","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"DH-VLM enables an infrastructure VLM to send latent reasoning guidance to a lightweight onboard VLM, reporting improved open-loop planning accuracy and collision rate with reduced communication and memory costs.","lead":"This paper presents DH-VLM, in which a powerful roadside computer reasons about a traffic scene and sends a compact internal 'thinking' vector to a small onboard model in the car, which then plans the path. The authors report lower path error and fewer collisions than earlier cooperative driving methods, with less data transmitted and lower memory use on the vehicle.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Because both the Section 3.4 cooperation-oriented QA dataset and the Section 4.2 planning evaluation are drawn from DAIR-V2X with no disclosed scene split, the reported SOTA margins may reflect train/test overlap rather than cooperative latent reasoning.","rationale":"The reader's weakest assumption already identified possible train/eval overlap, and I agree this is the single load-bearing point. The paper's own text makes the same DAIR-V2X the source for both the QA supervision and the planning benchmark, and the descriptor of approximately 100 scenes is small enough that overlap is not averaged away. The paper does disclose open-loop evaluation in the conclusion, which is a real limitation but not an internal contradiction; the overlap issue, if present, would invalidate the specific SOTA numbers. The proposed test is feasible with the authors' records: a list of scene IDs and a retraining run on disjoint scenes. Given the absence of code and split disclosure, I do not move the verdict; the paper remains acceptable only under the explicit condition that the split be disclosed and the table re-verified. This is consistent with the reader's CONDITIONAL verdict.","tokens_in":14399,"tokens_out":6713,"duration_ms":62983,"concrete_test":"Contact the authors for the DAIR-V2X scene IDs used to generate the Section 3.4 QA pairs and the scene IDs used for the Section 4.2 planning evaluation. If the intersection is nonempty, retrain the full three-stage pipeline with QA supervision restricted to a disjoint training split and recompute Table 1 on held-out scenes; if the margins over LangCoop and V2X-VLM shrink materially or vanish, the SOTA claim is a leakage artifact. Even if the lists are disjoint, repeat Table 1 under five random 80/20 scene-level splits and report mean plus standard deviation, to confirm the 14.6% and 26.9% improvements exceed run-to-run noise.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim in Table 1 rests on the infrastructure model producing guidance that transfers to scenes it was not optimized on. Section 3.4 states the cooperation-oriented QA dataset is built on DAIR-V2X with approximately 100 scenes across 28 urban intersections; Section 4.1 states the same DAIR-V2X is used to construct this QA dataset, and Section 4.2 reports planning metrics on DAIR-V2X. No sentence in the paper states that the scenes used for QA generation are disjoint from the scenes used for planning evaluation. If any planning-evaluation scene also supplied QA training pairs, then the HLA and IDLE modules are optimized with supervision derived from the exact images and annotations on which Table 1 is computed, so the 14.6% L2 and 26.9% collision margins would be inflated by memorization rather than achieved by latent-level cooperation. This is especially dangerous at the reported scale: with roughly 100 scenes, a small amount of overlap can shift the 0.19% collision and 1.87 m L2 averages substantially. The acknowledged open-loop setup amplifies the issue because open-loop imitation metrics are sensitive to seeing the same traffic geometry in training. No code, per-scene results, or train/eval split is provided, so the overlap cannot currently be ruled out.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DH-VLM, a cooperative driving framework with asymmetric latent-level reasoning between an infrastructure model (Qwen2.5-7B) and an ego vehicle model (Qwen2.5-0.5B). The infrastructure aggregates intermediate hidden states through a Hierarchical Latent Aggregation (HLA) module into a compact latent guidance, which the ego vehicle injects via an Infrastructure-Driven Latent Evolution (IDLE) module during a second forward pass. The authors construct a cooperation-oriented QA dataset of over 80k pairs from DAIR-V2X, train the infrastructure and ego models in three stages, and report open-loop planning results on DAIR-V2X with additional claimed validation on V2X-Sim. The headline results are an average L2 error of 1.87 m (14.6% better than LangCoop) and an average collision rate of 0.19% (26.9% better than V2X-VLM), with a 57.3% communication-cost reduction relative to UniV2X. The paper includes ablations showing that the fusion encoder, QA data components, HLA, and IDLE each contribute to performance, and a robustness analysis under packet loss and latency.","tokens_in":14729,"tokens_out":4428,"duration_ms":39008,"significance":"If the quantitative claims survive a clean evaluation, the latent-level fusion design is a valuable and original direction: it avoids the bandwidth cost of raw feature exchange, the error propagation of text-level advice, and the onboard compute burden of large VLMs, while preserving ego autonomy. The paper also contributes a reusable cooperation-oriented QA dataset and a careful component ablation (Tables 3, 5, 6) that supports the internal design choices. However, the headline SOTA claims currently rest on open-loop metrics over roughly 100 scenes with no disclosed train/eval split, no error bars or significance tests, and unreported V2X-Sim validation, so the empirical evidence is not yet commensurate with the strength of the claims. No code or per-scene results are provided, which limits reproducibility and makes the overlap risk impossible to rule out.","major_comments":[{"comment":"The cooperation-oriented QA dataset is constructed from DAIR-V2X (Section 3.4, over 80k QA pairs), and the planning metrics in Table 1 are also computed on DAIR-V2X (Section 4.2), but the manuscript never states that the scenes used for QA generation are disjoint from the scenes used for planning evaluation. Because the infrastructure model is trained on this QA data and the HLA/IDLE modules are jointly optimized, any scene overlap would allow the model to memorize the evaluation scenes and would inflate the reported 14.6% L2 and 26.9% collision-rate margins. The authors should disclose the exact scene-level split, report per-scene results, and ideally evaluate on held-out intersections.","section":"§3.4, §4.1, §4.2"},{"comment":"The headline comparison is not against a single previous state of the art: the 14.6% L2 improvement is relative to LangCoop, while the 26.9% collision-rate improvement is relative to V2X-VLM; no method is state of the art on both metrics simultaneously. With roughly 100 scenes, a small number of scenes can move the reported averages, and no error bars, confidence intervals, or significance tests are provided. Please report variance over scenes and seeds, run statistical tests, and compare all methods under identical data splits and training conditions.","section":"Table 1, §4.2"},{"comment":"Section 4.1 states, 'We further validate our framework on V2X-Sim', but no V2X-Sim experiments, tables, or figures appear anywhere in the paper. A validation that is not reported cannot be assessed, so either provide the V2X-Sim results or remove the claim.","section":"§4.1"},{"comment":"The abstract, introduction, and conclusion frame the collision-rate reduction and the 'robust paradigm' as evidence of safety and practical deployment, but all planning metrics are open-loop and computed on approximately 100 scenes; the conclusion itself acknowledges the open-loop limitation. Collision rates in open-loop imitation are not a closed-loop safety measure, so the safety claims should be tempered or supplemented with closed-loop evaluation (e.g., in simulation) before publication.","section":"Abstract, §1, §4.2, §5"}],"minor_comments":[{"comment":"The conclusion says the method 'reduces the 5s L2 planning error by 14.6%', but Table 1 shows 14.6% on the average L2 error (1.87 vs 2.19); the 5s L2 reduction relative to LangCoop is 18.8% (3.72 vs 4.58). Please correct this number.","section":"§5"},{"comment":"Several table entries have merged numerals due to missing spacing, e.g., '0.890.44' in Table 1, '1.730.44' in Table 3, '0.250.45' in Table 4, and '0.190.33' in Table 5. These should be separated and the tables rechecked.","section":"Tables 1, 3, 4, 5"},{"comment":"The definition of the '•, ◦, △' symbols is confusing: the caption says they indicate average 5s L2 error of infrastructure-generated trajectories, but they are used as row markers in an evaluation of ego planning. Please define the symbol assignment (e.g., text vs. latent guidance levels) explicitly in the caption.","section":"Table 4 caption"},{"comment":"The communication robustness plots show single curves with no error bars or multiple runs; given that the paper stresses robustness, please report variability across seeds or scene subsets.","section":"Fig. 5"}],"recommendation":"major_revision","confidential_remarks":"The train/eval overlap concern is the central risk in this manuscript. If the authors cannot demonstrate a disjoint scene-level split between the QA training data and the planning evaluation data, the headline quantitative claims should be withdrawn or substantially re-scoped. I also recommend checking the novelty claim 'first to explore latent-level fusion in VLM-based cooperative driving' against reference [44] and other concurrent latent-collaboration work before the final decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the core idea is good, the paper is clearly written, and the evidence section is the weak spot. The architecture is a sensible division of labor: the infrastructure compresses intermediate VLM hidden states through HLA into a compact latent vector, and the ego vehicle's IDLE module cross-attends to it before a second forward pass. Putting a 0.5B model onboard with sparse 5:1 communication is a practical target, and the ablation study does show HLA and IDLE each move the metrics. The ego-personalized QA dataset is also a useful contribution, especially the counterfactual and safety-aware reasoning pieces.\n\nThe problem is that the headline numbers are not yet evidential. Both the cooperation-oriented QA dataset (Section 3.4) and the planning evaluation (Section 4.2) come from DAIR-V2X, and no sentence states a train/eval scene split. With roughly 100 scenes, even partial overlap could inflate the reported 14.6% L2 and 26.9% collision margins substantially. The paper also gives no error bars, no per-scene breakdown, no released code or data, and only mentions V2X-Sim without reporting numbers. The conclusion honestly says the evaluation is open-loop, which is good, but it means the safety claim rests on imitation metrics, not closed-loop behavior.\n\nTwo smaller issues. First, the baselines are not a single SOTA model: LangCoop is best on L2, V2X-VLM is best on collision rate, so 'beats the previous SOTA' means beating two different methods on one metric each. That is worth stating explicitly. Second, Table 3 reports infrastructure-generated text outputs with 1.72 m L2 but lists the full latent system at 1.87 m, which looks odd and needs a footnote explaining what is being compared.\n\nFairness check: I do not see a methodological howler. The HLA/IDLE ablations move in the right directions, and the communication-robustness analysis is a plus. The stress-test concern about train/eval overlap is the load-bearing issue, and it is a real one. This deserves peer review rather than desk rejection, but the authors should be asked to disclose the split, add error bars, and show actual V2X-Sim results. I would bring it to reading group only after those numbers are out.","headline":"The latent-cooperation architecture is sensible and worth taking seriously, but the SOTA claim cannot be trusted until the authors disclose whether the QA training scenes overlap the planning evaluation scenes on DAIR-V2X.","tokens_in":15220,"tokens_out":2774,"would_cite":false,"duration_ms":27403,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Cooperative driving becomes safer and cheaper when vehicles exchange latent VLM states rather than text or raw data.","keywords":["autonomous driving","cooperative driving","vision-language models","latent reasoning","V2X communication","infrastructure-to-vehicle cooperation","L2 planning error","collision rate"],"falsifier":"Check whether any DAIR-V2X scene appears both in the 80k-pair QA training set and in the planning evaluation set; if the sets overlap, rerun the comparison on a disjoint split. Separately, deploy the trained ego policy closed-loop in a simulator with the same intersections and measure collision rates; the central claim fails if the 0.19% open-loop collision rate does not translate into fewer closed-loop collisions than the single-agent baseline.","tokens_in":14214,"feed_emoji":"🚗","tokens_out":7182,"duration_ms":57520,"temperature":0.7,"pith_summary":"The paper tries to show that the best way for an infrastructure system to help an autonomous vehicle plan is to send it a compact vector from the infrastructure's vision-language model's internal reasoning layers, not a text summary and not raw sensor data. The proposed framework, DH-VLM, uses a large infrastructure VLM to form global-reasoning latent guidance and a small 0.5B onboard VLM that refines its own latent state with that guidance via cross-attention. On the DAIR-V2X cooperative driving benchmark, the paper reports the best planning numbers among the compared methods: an average L2 error of 1.87 m and a collision rate of 0.19%, improving on the prior text-based and query-based cooperative methods by 14.6% and 26.9%, respectively, while reducing communication cost by 57.3%. A sympathetic reader would care because it suggests that semantic cooperation can be both more accurate and more practical than exchanging words or heavy features, and that a resource-constrained vehicle can benefit from a powerful infrastructure without losing local autonomy.","feed_headline":"Latent guidance cuts driving error 14.6% and crash rate 26.9%","feed_subtitle":"Sharing VLM hidden states between infrastructure and vehicle improves planning while cutting bandwidth 57.3%.","key_machinery":"Two modules carry the argument. Hierarchical Latent Aggregation (HLA) compresses the infrastructure VLM's selected intermediate-layer hidden states into a single temporal latent vector, using attention over tokens, learned layer weights, and a temporal window, so guidance captures global context rather than the final layer's token-prediction embedding. Infrastructure-Driven Latent Evolution (IDLE) is an attention-based fusion module on the ego side: it aligns the infrastructure latent with an MLP, applies cross-attention to the ego's own latent, and feeds the refined representation through a second VLM forward pass, so cooperative information is folded into planning while the ego can fall back to its first pass when guidance is missing. Together they implement the paper's dual-horizon split between global reasoning at the infrastructure and local planning at the ego vehicle.","core_discovery":"The central discovery is that latent-level fusion beats both result-level (text) and early-level (raw sensory or query) cooperation for VLM-based autonomous driving. The infrastructure model aggregates intermediate hidden states from a subset of transformer layers through a Hierarchical Latent Aggregation module, producing a compact global-reasoning latent guidance; the ego model, after an initial forward pass, aligns this guidance with an MLP adapter and injects it through cross-attention in an Infrastructure-Driven Latent Evolution module, then runs a second forward pass for planning. This lets the ego vehicle keep its own local perception as the source of control while using infrastructure latent states as a soft prior that can be ignored when communication fails. The paper reports that this design achieves the best planning numbers on DAIR-V2X: 1.87 m average L2 error and 0.19% average collision rate, with the largest gains at longer horizons, and attributes the gains to the latent reasoning mechanism rather than model scale, since the onboard model is only 0.5B parameters.","pith_inferences":["Because the paper's QA data and planning evaluation both come from DAIR-V2X, a natural next experiment is to test whether the reported gains survive a strict scene split; the paper does not report this.","If latent guidance is truly model-agnostic, the same infrastructure latent should transfer to a different onboard VLM after only adapter retraining; testing zero-shot transfer would clarify whether IDLE learns cooperation or overfits to Qwen2.5-0.5B.","The robustness study varies packet loss and latency but not the semantic content of guidance; a targeted test that corrupts latent guidance, for instance by encoding a fake pedestrian, would show whether cross-attention actually filters misinformation or just averages it in.","The dual-horizon split could generalize to other asymmetric robot systems, such as cloud-assisted manipulation or multi-robot planning, wherever a large server model guides a small onboard policy."],"forward_implications":["Ego vehicles can remain drivable when infrastructure communication degrades: under packet loss or latency, the model falls back to its first forward pass and still plans from local perception.","Latent guidance works best from intermediate transformer layers; transmitting only the final hidden layer, which is close to token decoding, loses visual semantic information and degrades planning.","A sparse 5:1 communication ratio retains most of the benefit of 1:1 guidance while cutting bandwidth, suggesting practical deployment ratios exist between 1:1 and 10:1.","Long-horizon planning benefits more than short-horizon planning: the reported 5-second L2 error and collision rate improve most relative to non-cooperative and query-based baselines, which supports the global-reasoning premise.","Cooperative QA supervision with ego-personalized counterfactual questions is necessary for infrastructure guidance; removing either fundamental scene understanding or ego-personalized QA raises the L2 error of infrastructure-generated text outputs."],"supporting_citations":[{"why":"DAIR-V2X is the benchmark from which the cooperation-oriented QA dataset is generated and on which the planning metrics are computed; the whole empirical case rests on it.","marker":"[37]"},{"why":"LangCoop is the text-based cooperative VLM baseline that DH-VLM is compared against for the 14.6% L2-error improvement.","marker":"[5]"},{"why":"V2X-VLM is the query-based cooperative VLM baseline that DH-VLM is compared against for the 26.9% collision-rate improvement.","marker":"[36]"},{"why":"UniV2X is the end-to-end query-fusion cooperative baseline used as the reference for the 57.3% communication-cost reduction.","marker":"[38]"},{"why":"Qwen2.5-7B and Qwen2.5-0.5B are the actual infrastructure and ego backbones; their hidden states are what HLA and IDLE operate on.","marker":"[1]"},{"why":"Continuous latent-space reasoning motivates the choice to reason in latent states rather than decoding text at every step.","marker":"[6]"},{"why":"Latent collaboration in multi-agent systems motivates exchanging hidden states instead of raw data or text.","marker":"[44]"},{"why":"V2X-Sim provides a simulated cooperative-driving benchmark used for additional validation.","marker":"[12]"}],"fun_headline_variants":["Latent cooperation cuts driving error 14.6%, crashes 26.9%","Sharing hidden states reduces driving errors and collisions","Infrastructure latent guidance boosts planning, slashes bandwidth","Cooperative VLM: 57% less bandwidth, 27% fewer crashes","Latent fusion: better driving with 57% less data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The weakest premise is that planning numbers measured open-loop on about 100 DAIR-V2X scenes, without proof that the scenes used to build the cooperation-oriented QA training set are disjoint from the evaluation scenes, predict real closed-loop safety gains for cooperative driving.","fun_headline_variants_meta":{"raw":{"variants":["Latent cooperation cuts driving error 14.6%, crashes 26.9%","Sharing hidden states reduces driving errors and collisions","Infrastructure latent guidance boosts planning, slashes bandwidth","Cooperative VLM: 57% less bandwidth, 27% fewer crashes","Latent fusion: better driving with 57% less data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000217,"raw_usage":{"total_tokens":1478,"prompt_tokens":1029,"completion_tokens":449,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":645,"completion_tokens_details":{"reasoning_tokens":359}},"tokens_in":645,"tokens_out":449,"duration_ms":4209,"temperature":1.0,"reasoning_tokens":359,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:25:11.902246+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Check whether any DAIR-V2X scene appears both in the 80k-pair QA training set and in the planning evaluation set; if the sets overlap, rerun the comparison on a disjoint split. Separately, deploy the trained ego policy closed-loop in a simulator with the same intersections and measure collision rates; the central claim fails if the 0.19% open-loop collision rate does not translate into fewer closed-loop collisions than the single-agent baseline.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"DAIR-V2X is the benchmark from which the cooperation-oriented QA dataset is generated and on which the planning metrics are computed; the whole empirical case rests on it."},{"cited_title":"In: Proceedings of the AAAI Confer- ence on Artificial Intelligence","cited_arxiv_id":null,"evidence_quote":"UniV2X is the end-to-end query-fusion cooperative baseline used as the reference for the 57.3% communication-cost reduction."}],"review_version":2}