{"id":"38fcf47c-b572-4502-b9b6-812c45fc4cf3","arxiv_id":"2507.21610","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"This paper summarizes the CVPR 2025 V2X cooperative driving challenge, its winning solutions, and the open research problems it reveals.","lead":"The organizers of the CVPR 2025 V2X cooperation challenge report the competition's design, its top solutions, SparseCoop for perception and MAP for planning, and the open problems it highlights. The paper is a useful map of where cooperative driving with shared sensors, cameras, and communication constraints currently stands.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's headline progress claims rest on unreported numbers: no leaderboard table, per-metric scores, or min-max normalization ranges are provided, so the 44.5% improvement and SOTA claims are not independently checkable.","rationale":"The reader's weakest_assumption identifies open-loop planning metrics as a proxy for real safety. That is a genuine limitation for deployment transfer, but it is explicitly acknowledged in §6 and does not by itself undermine the challenge's internal progress claim if the reported scores are accurate. The blocking issue is upstream: the manuscript does not contain the scores. A benchmark paper whose central quantitative contributions are a 44.5% improvement and a SOTA claim must publish the underlying leaderboard, normalization parameters, and evaluation code. The missing data also compounds the structural overlap between the paper's author list and the top solutions SparseCoop and MAP, making independent verification necessary. These are fixable in a revision, so the reader's conditional verdict remains appropriate without moving to rejection.","tokens_in":15713,"tokens_out":5807,"duration_ms":67567,"concrete_test":"Release the official leaderboard with raw per-metric values (Track 1: mAP, AMOTA; Track 2: L2 at 2.5/3.5/4.5 s, collision rate, off-road rate), the min-max reference ranges, and the evaluation script; independently rerun the UniV2X baseline and the SparseCoop/MAP checkpoints on the same test split and recompute the 44.5% normalized-score improvement and the detection/tracking SOTA comparison. If the recomputed values differ or cannot be reproduced, the central progress claim is unsubstantiated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim—that the challenge constitutes a unified benchmark and that top solutions demonstrate meaningful progress: SparseCoop is 'state-of-the-art' in detection/tracking (§5.1) and MAP improves the normalized score by 44.5% over UniV2X (§5.2)—is unverifiable from the manuscript. Section 3.1 defines the Track 2 final score as a weighted sum of min-max normalized error metrics (0.5·L2 + 0.25·collision + 0.25·off-road, averaged at 2.5s/3.5s/4.5s), but never states the 'predefined reference ranges' used for normalization, nor the raw per-metric values, nor the number of evaluation scenes. Because min-max normalization is an affine rescaling, the magnitude of any percentage improvement depends on the chosen bounds; without them, '44.5% improvement' is not a scale-invariant property of the method. Section 5.1 similarly provides no mAP/AMOTA numbers or comparison baselines to support the SOTA claim. With no leaderboard table, no error bars, and no evaluation code, the reported progress cannot be checked. This is more load-bearing than the open-loop-metric validity limitation, which the paper itself concedes in §6 and which would matter only after the numbers are known.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports on the organization and outcomes of the first End-to-End V2X Cooperation Challenge, held at the MEIS Workshop @ CVPR 2025. The challenge consists of two tracks: cooperative temporal perception (3D detection and multi-object tracking) and cooperative end-to-end planning, built on the UniV2X framework and the V2X-Seq-SPD dataset. The paper describes the task setup, evaluation metrics, participation statistics, technical challenges observed, and the top-performing solutions: SparseCoop (Track 1) and MAP (Track 2). The central claims are that the challenge establishes a unified benchmark for cooperative driving systems and that the top solutions demonstrate meaningful progress, exemplified by MAP's 44.5% improvement over the UniV2X baseline and SparseCoop's state-of-the-art detection/tracking performance. The paper also discusses future research directions including realistic communication modeling, bandwidth-adaptive fusion, generalization across heterogeneous agents, and language-based V2X communication.","tokens_in":15933,"tokens_out":3502,"duration_ms":41938,"significance":"If the quantitative claims were fully substantiated, this paper would document a useful community resource: a unified benchmark covering both cooperative perception and planning, with open-source baselines and external participation. The challenge addresses a real gap in V2X benchmarking, which has been dominated by perception-only tasks. The analysis of technical trends and the future-directions section are thoughtful and could help guide the field. However, the current manuscript does not provide the underlying leaderboard data, metric definitions in sufficient operational detail, or statistical significance assessments that would allow readers to verify the claimed progress. The main value at present is as a qualitative competition summary; the benchmark's quantitative evidence and the verification of SOTA claims require substantial additional reporting.","major_comments":[{"comment":"The Track 2 final score is defined as 0.5 × normalized L2 error + 0.25 × normalized collision rate + 0.25 × normalized off-road rate, where each metric is min-max normalized based on 'predefined reference ranges.' The reference ranges are never specified. Since min-max normalization is an affine transformation, the reported '44.5% improvement' in Section 5.2 is not scale-invariant and cannot be interpreted without these bounds. Please provide the reference ranges, the raw per-metric values for MAP and the UniV2X baseline, and the number of evaluation scenes used.","section":"§3.1 and §5.2"},{"comment":"The claim that SparseCoop achieves 'state-of-the-art detection and tracking performance' is not supported by any numerical results. No mAP or AMOTA values, no comparison against the UniV2X baseline, and no leaderboard table are given. Please include a full leaderboard for Track 1 with final scores and per-metric breakdowns for all ranked teams, and state whether the results are averaged over multiple runs with reported variance or statistical tests.","section":"§5.1"},{"comment":"The paper does not disclose that the SparseCoop team (Wang et al., Tsinghua University) includes co-authors of this paper (several authors are affiliated with Tsinghua University), and that the organizers themselves evaluated the challenge using their own dataset and baseline. This creates a potential conflict of interest that should be explicitly acknowledged. Additionally, the roles of the organizers in designing, running, and evaluating the methods should be clarified to assure readers of impartiality.","section":"§3.2 and §5.1"},{"comment":"Only 5 finalists achieved ranked results out of over 30 registered teams. The paper does not state how many teams actually submitted valid results, what criteria determined the finalists, or the evaluation conditions (e.g., allowed compute, communication bandwidth limits, test-time constraints). This information is necessary to assess whether the reported progress is representative of the broader participant pool and to understand the practical conditions of the benchmark.","section":"§3.2 and §5"},{"comment":"The paper concedes in Section 6 that closed-loop evaluation and robustness to weather, lighting, and sensor degradation were not tested. Given that all headline progress claims are based on open-loop planning metrics (L2 error, collision rate, off-road rate averaged at fixed future timestamps), the framing that the results demonstrate 'meaningful progress' toward deployable V2X systems is overstated. Please temper the claims accordingly, or add an explicit discussion of the limitations of open-loop metrics as a proxy for real-world safety.","section":"§6"}],"minor_comments":[{"comment":"The challenge is referred to inconsistently: 'End-to-End Autonomous Driving through V2X Cooperation Challenge' in the abstract and conclusion versus 'End-to-End V2X Cooperation Challenge' elsewhere. Please use a consistent name.","section":"Abstract and throughout"},{"comment":"The abbreviation 'NA VSIM' appears to be a typo for 'NAVSIM.' Also, the 'Competition' entries for V2X-Seq list 'CVPRW25 (Ours)' while other datasets list multiple years; consider formatting consistency.","section":"Table 1"},{"comment":"The text says 'On the DAIR-V2X-Seq-SPD benchmark,' but the dataset is called 'V2X-Seq-SPD' elsewhere. Please clarify whether these refer to the same benchmark and use a single name consistently.","section":"§5.2"},{"comment":"The paragraph beginning 'Multi-Agent Sensor Fusion under Bandwidth Constraints' is not visually separated from the following paragraph; add formatting or bullet structure to improve readability.","section":"§4"},{"comment":"For Track 1, the paper should specify the object class (merged 'Car'), the 3D IoU threshold used for mAP, and the exact formulation of AMOTA (e.g., standard MOTA with interpolation parameters). These details are needed for reproducibility.","section":"§3.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a competition report. The main weakness is the absence of numerical results needed to verify the central claims; this is not merely a presentation issue but affects the credibility of the benchmark's contribution. I suggest the editor require the authors to include a full leaderboard and protocol details before publication. Additionally, the undisclosed overlap between the SparseCoop team and the organizing authors is a conflict-of-interest concern that should be flagged during review."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: this is a competition summary, not a new technical contribution. The value is in the benchmark design and the taxonomy of open problems; the headline progress claims (SparseCoop \"state-of-the-art\", MAP's 44.5% improvement) are not verifiable from the manuscript as written.\n\nWhat is actually new: the paper documents the first End-to-End V2X Cooperation Challenge, with two tracks (cooperative temporal perception and cooperative end-to-end planning) built on UniV2X/V2X-Seq-SPD. It attracted over 30 registered teams and 5 finalists. The description of the task setup, evaluation protocol, and the comparison table of existing datasets is clear and useful. The paper also does a decent job of organizing open challenges—bandwidth-aware fusion, robust multi-agent planning, heterogeneous sensors, communication-aware design—and Section 6 is honest about what the benchmark did not test: closed-loop evaluation, weather/lighting robustness, and realistic communication degradation. That self-awareness is genuine.\n\nThe soft spots are mostly about transparency, and they are load-bearing. Track 2's final score is a weighted sum of min-max normalized L2 error, collision rate, and off-road rate, but the paper never gives the predefined reference ranges used for normalization, the per-metric raw scores, the number of evaluation scenes, or any error bars. Min-max normalization is an affine rescaling, so a 44.5% improvement is not a scale-invariant statement; without the bounds it cannot be checked. Track 1's SOTA claim for SparseCoop has no mAP/AMOTA numbers at all. There is no leaderboard table in the paper. On top of that, both highlighted top methods have co-authors on this paper (Jiahao Wang on SparseCoop, Yiming Kan on MAP), and that overlap is not disclosed. The evaluation is run by the same group that built the baseline and dataset; external teams did participate, which helps, but the conflict-of-interest issue should be addressed openly. Minor: Section 5.2 says \"DAIR-V2X-Seq-SPD benchmark\"—that name is inconsistent with the V2X-Seq-SPD dataset used elsewhere.\n\nThe central idea—that a unified two-track benchmark for V2X cooperation is needed—holds up. The paper is clearly written and the authors are honest about the benchmark's limitations. What it lacks is the data needed to assess whether the progress claims are real.\n\nWho this is for: people working on V2X cooperative perception and planning, especially those organizing or entering benchmarks. A serious referee could extract value from it, but only after the missing tables and disclosure are added. I'd recommend sending it to peer review with a request for major revision, not desk rejection.","headline":"A useful competition write-up whose headline numbers are not yet checkable; worth reviewing, but only after the authors release the leaderboard and normalization details.","tokens_in":16589,"tokens_out":2867,"would_cite":false,"duration_ms":30264,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The End-to-End V2X Cooperation Challenge establishes a unified two-track benchmark for cooperative perception and planning, with top solutions showing state-of-the-art results including a 44.5% planning-score improvement over the UniV2X…","keywords":["V2X cooperative driving","cooperative perception","cooperative planning","benchmark","sparse fusion","end-to-end autonomous driving","UniV2X","V2X-Seq-SPD"],"falsifier":"Run the MAP planner and the UniV2X baseline in a closed-loop simulator on the same V2X-Seq-SPD scenarios and compare actual collision rates and off-road events; if MAP's advantage over the baseline shrinks or reverses under closed-loop control or under injected communication packet loss, the benchmark's proxy claim is weakened.","tokens_in":15463,"feed_emoji":"🚗","tokens_out":4683,"duration_ms":47702,"temperature":0.7,"pith_summary":"This paper reports on the first End-to-End Autonomous Driving through V2X Cooperation Challenge, a two-track competition built on the UniV2X framework and the V2X-Seq-SPD dataset. The paper argues that the challenge provides a unified benchmark for evaluating cooperative perception and planning, and that the top submissions show measurable progress: SparseCoop achieves state-of-the-art detection and tracking, while MAP improves the planning score by 44.5% over the baseline. By analyzing submissions from over 30 teams, the paper identifies bandwidth-aware fusion, robust planning, and heterogeneous sensor integration as the field's central open problems. The value of the claim is that a standardized, reproducible platform now exists for comparing V2X cooperative driving systems.","feed_headline":"Cooperative driving challenge: top planner beats baseline by 44.5%","feed_subtitle":"A two-track V2X benchmark shows sparse fusion and semantic maps push cooperative driving forward.","key_machinery":"The load-bearing mechanism is the challenge benchmark itself: a two-track protocol on the UniV2X framework and the V2X-Seq-SPD dataset, with Track 1 scored as 0.5·mAP + 0.5·AMOTA and Track 2 scored as 0.5·normalized L2 error + 0.25·normalized collision rate + 0.25·normalized off-road rate, each metric averaged at 2.5s, 3.5s, and 4.5s. Within the top solutions, the key mechanisms are SparseCoop's anchor-aided instance queries (explicit geometric attributes per object enabling cross-agent fusion without dense BEV features) with cooperative instance denoising, and MAP's two-branch query generation (Ego-status-guided Planning plus Plan-enhancing Online Mapping) fused by a learned weight adapter that adaptively balances ego state and semantic priors.","core_discovery":"The central discovery, as the paper states it, is that a two-track benchmark can jointly evaluate cooperative temporal perception and cooperative end-to-end planning, and that top solutions under this benchmark demonstrate clear progress. SparseCoop, built on anchor-aided instance queries with cooperative instance denoising, achieves state-of-the-art detection and tracking performance under the V2X-Seq-SPD benchmark. MAP, which feeds semantic map information directly into planning via a learned adaptive fusion weight, improves the overall normalized planning score by 44.5% over the UniV2X baseline and ranks first on the planning leaderboard. From the submitted solutions, the paper identifies converging design trends: sparse query-based fusion, modular architectures, and temporal reasoning.","pith_inferences":["The 44.5% improvement is measured on open-loop planning metrics; if closed-loop simulation or real-world testing were applied, the ranking could change because collision rate and off-road rate are computed from planned trajectories, not actual control.","The challenge's emphasis on sparse fusion suggests that communication bandwidth, not perception accuracy alone, is the binding constraint for V2X deployment.","MAP's success implies that perception modules should be optimized for planning utility rather than for detection accuracy alone, a criterion the current benchmark only partially captures.","A direct test would be to re-run the same top solutions under injected packet loss or latency to see whether their advantage persists."],"forward_implications":["Sparse, query-based cooperative fusion can achieve top perception performance while keeping communication costs low, supporting bandwidth-aware deployment.","Explicitly using semantic map information as a planning input can substantially improve trajectory quality over ego-only or pure feature-fusion baselines.","A unified open benchmark for cooperative perception and planning is feasible and can attract diverse international participation, enabling reproducible comparison.","The identified research gaps — realistic communication modeling, bandwidth-adaptive fusion, and generalization across heterogeneous agents — define concrete next targets for the field.","Future challenge editions with closed-loop evaluation and adverse conditions are necessary to verify that benchmark progress transfers to deployment."],"supporting_citations":[{"why":"Provides the UniV2X framework and serves as the official baseline for both challenge tracks.","marker":"[87]"},{"why":"Supplies the V2X-Seq-SPD dataset that underpins the benchmark's cooperative perception and planning tasks.","marker":"[86]"},{"why":"Introduces the DAIR-V2X dataset lineage; the paper refers to the Track 2 benchmark as DAIR-V2X-Seq-SPD.","marker":"[84]"}],"fun_headline_variants":["V2X planning benchmark: semantic map boosts score by 44.5%","Cooperative driving: sparse fusion tops V2X planning benchmark","44.5% planning gain from semantic maps in V2X challenge","V2X challenge: sparse fusion and semantic maps drive planning gains","Top V2X planner beats baseline by 44.5% with semantic maps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The scoring protocol treats open-loop planning metrics — L2 error, collision rate, and off-road rate at three future timestamps — as a valid proxy for real driving safety and quality, and the paper itself notes that closed-loop evaluation and robustness to weather, lighting, and sensor degradation were not tested.","fun_headline_variants_meta":{"raw":{"variants":["V2X planning benchmark: semantic map boosts score by 44.5%","Cooperative driving: sparse fusion tops V2X planning benchmark","44.5% planning gain from semantic maps in V2X challenge","V2X challenge: sparse fusion and semantic maps drive planning gains","Top V2X planner beats baseline by 44.5% with semantic maps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000534,"raw_usage":{"total_tokens":2537,"prompt_tokens":884,"completion_tokens":1653,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":500,"completion_tokens_details":{"reasoning_tokens":1556}},"tokens_in":500,"tokens_out":1653,"duration_ms":14811,"temperature":1.0,"reasoning_tokens":1556,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T12:33:48.387119+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the MAP planner and the UniV2X baseline in a closed-loop simulator on the same V2X-Seq-SPD scenarios and compare actual collision rates and off-road events; if MAP's advantage over the baseline shrinks or reverses under closed-loop control or under injected communication packet loss, the benchmark's proxy claim is weakened.","supporting_citations":[{"cited_title":"End-to-end autonomous driving through v2x cooperation","cited_arxiv_id":null,"evidence_quote":"Provides the UniV2X framework and serves as the official baseline for both challenge tracks."},{"cited_title":"Dair-v2x: A large-scale dataset for vehicle- infrastructure cooperative 3d object detection","cited_arxiv_id":null,"evidence_quote":"Introduces the DAIR-V2X dataset lineage; the paper refers to the Track 2 benchmark as DAIR-V2X-Seq-SPD."}],"review_version":1}