{"id":"87a0e066-9493-4618-99f9-7ecabf8b1123","arxiv_id":"2608.09541","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Fusing collaborative sensor data at detection or tracking level beats fusing predicted trajectories, and a compressed LiDAR-sharing prototype improves forecasting over a single vehicle.","lead":"This paper proposes a framework called Co-P&P that lets connected vehicles share sensor data before predicting other road users' future motion. Experiments on a real vehicle-infrastructure dataset show that sharing data early, at detection or tracking level, improves forecasts and cuts bandwidth compared with sharing final predictions.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"A single confidence-averaging late-fusion baseline is load-bearing for the claim that prediction-level fusion is inferior; a learned trajectory-level fusion could reverse the ordering.","rationale":"The reader's weakest_assumption is the same one I would flag: the fusion-timing conclusion is an inductive generalization from a single, unlearned late-fusion baseline. I agree with that diagnosis. I do not see an internal contradiction in the reported tables strong enough to overturn the paper's baseline value: the detection/tracking fusion numbers are plausible, and the end-to-end prototype, while noisy, is honestly labeled. The reason this is the most load-bearing concern, rather than the missing error bars or the unvalidated CSC module, is that Sections 5.2 and 8 convert a narrow empirical comparison into a deployment rule ('fuse before motion prediction'). That rule is exactly what a systems designer would take away, and it is not justified by one confidence-weighted averaging baseline. A learned late-fusion baseline is the direct, inexpensive check. If it fails to beat tracking-level fusion, the paper's recommendation survives; if it succeeds, the paper must be reframed as a baseline-family-specific result. Either way, the current CONDITIONAL verdict is appropriate; no change is needed.","tokens_in":18243,"tokens_out":10744,"duration_ms":91512,"concrete_test":"Implement a learned late-fusion baseline for Section 5.1's prediction-level fusion: for each associated object, feed the top-K trajectory hypotheses (means, covariances, confidences) from each agent's MTR into a small cross-attention network, train it on the DAIR-V2X-Seq training split, and evaluate on the same test split used for Table 8, with 5 random seeds and mean±std. Settling criterion: if learned prediction-level fusion reaches or exceeds tracking-level fusion's 12.2 mAPf (or if the 9.7-vs-12.2 gap is within seed noise), then the paper's design rule in Sections 5.2 and 8 is not supported and the verdict should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The design rule in Sections 5.2 and 8 — fuse before motion prediction — is inferred from a single prediction-level fusion implementation: confidence-weighted averaging of per-agent predicted trajectories (Section 5.1). This baseline discards covariance and multimodality and is not trained, so it is the weakest of the three fusion methods compared. Table 8 puts prediction fusion at 9.7 mAPf versus 12.2 for tracking fusion, but Table 7 shows the same late-fusion rule without RPO already reaches 10.2 mAPf, so the gap is sensitive to small design choices. A learned trajectory-level fusion (e.g., cross-attention over each agent's top-K trajectory distributions) could plausibly close or reverse the 2.5-point gap. Because the conclusion is stated as a general deployment rule rather than as a result limited to this baseline family, the empirical comparison does not yet license it. This is the load-bearing point: if a stronger late-fusion baseline changes the ordering, the central recommendation fails even though all reported tables remain literally correct.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"Building on the authors' earlier conceptual Co-P&P paper, this manuscript contributes a modular baseline evaluation on DAIR-V2X-Seq that compares V2X fusion at the detection, tracking, and prediction stages, together with a minimal end-to-end prototype coupling RENO point-cloud compression with FutureDet. The central empirical claims are that prediction-level fusion (implemented as confidence-weighted averaging of per-agent trajectories) is less accurate and more bandwidth-hungry than detection- or tracking-level fusion, and that collaborative point-cloud sharing improves FutureDet forecasting over ego-only operation while neural compression preserves the benefit at about 34x lower bandwidth. The paper also argues for visibility-stratified recall as an evaluation methodology and surveys deployment challenges.","tokens_in":18454,"tokens_out":7758,"duration_ms":68728,"significance":"If the fusion-timing result holds as a general design rule, it would be a useful systems-level guideline for V2X perception-prediction pipelines. The RENO+FutureDet prototype is a concrete first instantiation of collaborative P&P with a quantified bandwidth-accuracy trade-off, and the visibility-stratified recall analysis is a principled step beyond plain mAP. The authors are honest about cases where gains are small or negative (Table 9 minFDE/MR, and the infrastructure-only comparisons in Table 8). However, the strength of the deployment rule currently exceeds what the evidence can support: the prediction-level fusion conclusion rests on a single untrained late-fusion baseline, and the end-to-end versus modular comparison is confounded by architecture and training differences. The paper has value as a controlled baseline study, but the general claims need to be re-scoped or supported by additional experiments.","major_comments":[{"comment":"The central design rule that fusion should be performed before motion prediction is inferred from a single prediction-level fusion implementation: confidence-weighted averaging of associated trajectories. That untrained rule discards covariance, multimodality, and per-agent uncertainty, making it structurally the weakest of the three fusion methods. The gap in Table 8 (mAPf 9.7 for prediction fusion vs 12.2 for tracking fusion) is of the same order as the effect of RPO in Table 7 (10.2 vs 9.7), which shows sensitivity to small design choices. A learned trajectory-level fusion, e.g., cross-attention over top-K trajectory distributions or covariance-aware ensembling, could plausibly close or reverse the ordering. Please add such a baseline or explicitly re-scope the abstract and Sections 5.2 and 8 to state that the conclusion applies to the implemented baseline family rather than to prediction-level fusion in general.","section":"§5.1, Table 8"},{"comment":"The sentence claiming that the end-to-end prototype 'isolates cascade error accumulation' as the dominant cause of the modular pipeline's weak forecasting is not supported by the evidence. The two systems differ in model architecture (FutureDet vs MTR), training objective (joint vs separate), and input representation (raw/fused point cloud vs tracked boxes), so the large mAPf gap (14.15 vs 4.9) could reflect a stronger forecaster rather than the absence of a cascade. To make the attribution, the authors should run a controlled comparison, for example feeding FutureDet's output to the modular tracker/predictor or evaluating MTR on FutureDet detections; otherwise the statement should be softened to a hypothesis.","section":"§5.2, Table 9"},{"comment":"All quantitative results are reported as single runs with no confidence intervals or significance tests, yet Section 5.2 states that some differences 'fall within the expected run-to-run variation.' This is not verifiable from the paper. Since the main fusion-timing ordering (10.3 vs 12.2 vs 9.7 mAPf in Table 8) is comparable in magnitude to the RPO-induced shift in Table 7, the ordering could plausibly change under seed variation. Please provide multiple seeds or bootstrap intervals for at least the fusion-timing comparison and the RENO quantization sweep.","section":"§5.2, Tables 4-9"},{"comment":"The abstract states that collaboration 'improves forecasting accuracy' without qualification, but Table 9 shows that minFDE and MR are worse in every collaborative condition than in the ego-only condition (e.g., minFDE 6.242 at RENO q256 vs 6.216 for ego; MR 0.452 vs 0.448). The improvement is metric-dependent, holding for mAPf and minADE but not for minFDE or MR. The abstract and Section 8 should name the metrics on which the improvement is claimed, or the claim should be weakened to 'improves some forecasting metrics.'","section":"Abstract and Table 9"}],"minor_comments":[{"comment":"The IDS entry for 'Infrastructure only' reads 2526, which is either a typo or an implausible value compared with 26 for vehicle-only and 12 for track fusion; please correct the value and ensure column alignment.","section":"Table 6"},{"comment":"The description of mAPf is confusing: it says mAPf penalizes trajectories with minFDEk < 4m for inaccurate forecasts, while the miss-rate definition uses minFDEk > 4m. Please clarify the threshold direction and the exact penalty rule.","section":"Table 2"},{"comment":"The sentence 'Gains over the stronger infrastructure-only baseline are smaller and, for some metrics, fall within the expected run-to-run variation' is inconsistent with Table 7, where late fusion with RPO (mAPf 9.7) is below infrastructure-only (9.8); please use neutral wording such as 'differences' and avoid implying consistent gains.","section":"§5.2, Table 7"},{"comment":"The phrase 'futility of prediction-level fusion' is too strong for a result based on one baseline; rephrase to 'the ineffectiveness of the implemented prediction-level fusion' or similar.","section":"§6"},{"comment":"Communication cost units are given inconsistently as 'Bytes', 'bytes per message', and 'per-frame payloads'; please state clearly that Table 8 reports bytes per message (or per frame) and derive the KB/s figures from that convention.","section":"§5.1/Table 8"},{"comment":"The paper calls the study a 'controlled, reproducible baseline' but does not provide code or model weights; please include a code repository or state explicitly what artifacts are available.","section":"§2.2"},{"comment":"Minor typographical issue: 'roughly34×' should read 'roughly 34×'.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the experimental tables are transparent, but the two load-bearing claims—that prediction-level fusion is generally inferior and that the end-to-end prototype isolates cascade error accumulation—need stronger support or careful re-scoping. I would be willing to accept a revised version that adds a stronger trajectory-level fusion baseline (or restricts the claim) and either de-confounds the end-to-end comparison or removes the causal attribution. No concerns about circularity or data fabrication were identified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: useful, honest baseline work. The fusion-timing comparison on DAIR-V2X-Seq — detection vs tracking vs prediction — is new, and the RENO+FutureDet prototype is a concrete first step toward compressed collaborative joint perception and prediction. The authors report numbers fairly, including cases where gains are small or negative, and they give a principled reason for reporting visibility-stratified recall rather than mAP. The direction of the core result, prediction-level fusion costs the most bandwidth and helps least, is solid.\n\nSoft spots are real but repairable. The abstract overstates the forecasting gain: in the prototype, collaboration improves mAPf and minADE but slightly worsens minFDE and MR, something the paper itself acknowledges later. The prediction-level fusion baseline is the weakest of the three: confidence-weighted averaging of predicted trajectories, no learned ensembling, no covariance. The design rule 'fuse before motion prediction' is therefore calibrated to that baseline family; a stronger late-fusion method could plausibly shift the accuracy ordering, though the bandwidth penalty would remain. The modular-vs-end-to-end comparison is also confounded: MTR vs FutureDet differ in model and training, so claiming it 'isolates cascade error accumulation' overreaches. No error bars, and no code or data release, which limits how much weight the absolute numbers can carry.\n\nWho should read it: anyone working on V2X perception or collaborative forecasting. It is a useful anchor for a subfield with very few public baselines, and the compression-aware prototype is worth citing. Recommendation: send it to peer review, and ask the authors to strengthen the late-fusion baseline, add error bars or release code, and tone down the abstract and the causal claims.","headline":"Useful, honest baseline on fusion timing for collaborative perception, but the headline design rule leans on a weak late-fusion baseline and the abstract overclaims forecasting gains.","tokens_in":18962,"tokens_out":3170,"would_cite":true,"duration_ms":28632,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Collaborative fusion should happen before motion prediction, not after.","keywords":["collaborative perception","motion prediction","joint perception and prediction","V2X communication","fusion strategy","point cloud compression","occlusion recovery","DAIR-V2X-Seq"],"falsifier":"Run the same fusion-stage comparison on DAIR-V2X-Seq with a late-fusion method that learns to combine full trajectory distributions instead of confidence-weighted averaging; if prediction-level fusion then matches or exceeds detection-level fusion in forecast accuracy, the paper's central ordering fails. Alternatively, repeat the three-stage comparison on another cooperative dataset with the same pipeline; a reversal of the ordering there would show the result is dataset-specific.","tokens_in":18039,"feed_emoji":"🚗","tokens_out":7579,"duration_ms":59813,"temperature":0.7,"pith_summary":"This paper tries to establish where in a connected-vehicle perception pipeline collaborative information should be fused, and whether an end-to-end collaborative perception-and-prediction system can work under realistic communication limits. On the DAIR-V2X-Seq vehicle-to-infrastructure dataset, it compares fusing at detection, tracking, and prediction stages using the same detection, tracking, and prediction models. It finds that fusing predicted trajectories underperforms detection-level and tracking-level fusion and barely matches the stronger single-agent baseline, while costing roughly an order of magnitude more bandwidth. It then builds a minimal end-to-end prototype in which a collaborating vehicle's LiDAR sweep is compressed by the RENO neural codec and fed into the FutureDet detection-forecasting model; collaboration improves forecasting accuracy, and the strongest compression preserves that gain at about 34x lower bandwidth. If correct, the paper gives system designers a concrete rule: fuse before motion prediction, and share compressed scene geometry rather than final forecasts.","feed_headline":"Late fusion of trajectories hurts connected-car forecasting","feed_subtitle":"On a real V2X dataset, prediction-level fusion fails while earlier fusion helps; compression keeps the gain at 34x lower bandwidth.","key_machinery":"The load-bearing device is a controlled fusion-stage ablation. One modular pipeline—PointPillars for 3D object detection, AB3DMOT for multi-object tracking, MTR for motion prediction—is trained once, and only the point of V2X fusion changes: detection-level fusion merges per-agent 3D boxes by non-maximum suppression; tracking-level fusion associates tracks by Hungarian matching and combines them with covariance-weighted track-to-track fusion; prediction-level fusion averages per-agent trajectories by confidence. Relative pose optimization, an iterative closest-point registration, corrects residual misalignment between the two agents' coordinate frames before any fusion. The end-to-end prototype replaces the cascade with one trainable model: the RENO neural codec compresses the collaborating agent's LiDAR sweep, the ego side decodes and merges it, and FutureDet outputs current detections together with future trajectories in a single forward pass. These two mechanisms let the paper separate the question of when to fuse from the question of what to share.","core_discovery":"The paper's central claim is that prediction-level fusion—merging the final predicted trajectories of two agents by confidence-weighted averaging—degrades overall system performance relative to detection-level or tracking-level fusion, and that the reason is cascade error accumulation. In the modular pipeline (PointPillars detection, AB3DMOT tracking, MTR prediction), detection and tracking fusion lift recall on occluded objects and improve forecast accuracy, while prediction fusion does not and requires 4003.9 bytes per message versus 346.3 for detection fusion and 406.3 for tracking fusion. The authors attribute this to tracker fragmentation: identity switches and broken trajectories cannot be repaired once the forecast has been computed. The end-to-end prototype supports the same story: a jointly trained FutureDet model on shared, RENO-compressed point clouds reaches minADE around 3.5 and minFDE around 6.2, far below the modular pipeline's 14–19 and 26–38 on the same data, isolating inter-stage error accumulation rather than the dataset or forecaster as the dominant cause of weak modular forecasting.","pith_inferences":["Beyond the paper: a learned late-fusion method that merges full trajectory distributions rather than confidence-weighted point estimates could in principle overturn the ranking; the design rule is established only against the paper's baseline family.","Beyond the paper: if cascade error accumulation is the dominant bottleneck, then end-to-end collaborative models should show a larger accuracy gain from collaboration than modular pipelines do, which is a direct testable prediction of the paper's interpretation.","Beyond the paper: the 34x compression result suggests the transmitted representation is highly redundant for forecasting; task-aware feature compression or learned scene completion may preserve the same forecasting benefit at even lower bitrates.","Beyond the paper: the visibility-stratified recall metric could be extended to a precision-aware visibility mAP by attributing false positives to visibility groups through nearest-neighbor association, an open problem the paper explicitly leaves."],"forward_implications":["For practical deployment, collaborative information should be fused at the detection or tracking stage rather than after motion prediction.","Prediction-level fusion is dominated on both axes: it fails to improve accuracy and consumes roughly 40 KB/s at 10 Hz, about ten times the bandwidth of detection-level fusion.","A jointly trained detection-forecasting model avoids the inter-stage error accumulation that cripples modular forecasting, so end-to-end Co-P&P is the more promising route to accurate predictions.","Neural point-cloud compression at quantization level q=256 cuts the transmitted payload from 682,712 to 19,898 bytes per frame while slightly improving forecast accuracy over raw transmission, showing that aggressive compression preserves task-relevant geometry.","Visibility-stratified recall should be part of collaborative perception evaluation because it directly measures the occlusion-recovery benefit that collaboration provides."],"supporting_citations":[{"why":"Supplies the prior conceptual Co-P&P framework that this article extends and benchmarks.","marker":"[1]"},{"why":"Provides the RENO neural point-cloud codec used to compress the shared LiDAR sweep in the end-to-end prototype.","marker":"[21]"},{"why":"Provides FutureDet, the jointly trained detection-forecasting model that consumes the merged point cloud in the prototype.","marker":"[22]"},{"why":"Supplies the DAIR-V2X-Seq real-world V2I dataset with LiDAR, tracking, trajectory, and HD-map annotations used for all experiments.","marker":"[51]"},{"why":"Provides PointPillars, the 3D detector used as the first stage of the modular pipeline.","marker":"[53]"},{"why":"Provides AB3DMOT, the multi-object tracker used as the second stage of the modular pipeline.","marker":"[54]"},{"why":"Provides MTR, the motion predictor used as the final stage of the modular pipeline.","marker":"[55]"},{"why":"Supplies the covariance-weighted track-to-track fusion rule used for tracking-level fusion.","marker":"[58]"}],"fun_headline_variants":["Early fusion beats trajectory merging in connected-car forecasting","Prediction-level fusion fails, earlier fusion wins for CAVs","Cascade errors hurt late fusion; compression keeps gain at 34x","Joint perception-prediction: early fusion cuts bandwidth 34x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The conclusion that prediction-level fusion is worse rests on a single late-fusion baseline: confidence-weighted averaging of predicted trajectories; a stronger method for merging trajectory distributions could change the ordering, so the paper's design rule is calibrated to its own baseline family.","fun_headline_variants_meta":{"raw":{"variants":["Early fusion beats trajectory merging in connected-car forecasting","Prediction-level fusion fails, earlier fusion wins for CAVs","Cascade errors hurt late fusion; compression keeps gain at 34x","Joint perception-prediction: early fusion cuts bandwidth 34x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00019,"raw_usage":{"total_tokens":1348,"prompt_tokens":966,"completion_tokens":382,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":582,"completion_tokens_details":{"reasoning_tokens":311}},"tokens_in":582,"tokens_out":382,"duration_ms":3899,"temperature":1.0,"reasoning_tokens":311,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:24:33.144162+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same fusion-stage comparison on DAIR-V2X-Seq with a late-fusion method that learns to combine full trajectory distributions instead of confidence-weighted averaging; if prediction-level fusion then matches or exceeds detection-level fusion in forecast accuracy, the paper's central ordering fails. Alternatively, repeat the three-stage comparison on another cooperative dataset with the same pipeline; a reversal of the ordering there would show the result is dataset-specific.","supporting_citations":[{"cited_title":"In: Proceedings of the 11th International Conference on Vehicle Technology and Intelligent Transport Sys- tems - VEHITS, pp","cited_arxiv_id":null,"evidence_quote":"Supplies the prior conceptual Co-P&P framework that this article extends and benchmarks."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on 24 Computer Vision and Pattern Recognition, pp","cited_arxiv_id":null,"evidence_quote":"Provides the RENO neural point-cloud codec used to compress the shared LiDAR sweep in the end-to-end prototype."},{"cited_title":"In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pp","cited_arxiv_id":null,"evidence_quote":"Provides FutureDet, the jointly trained detection-forecasting model that consumes the merged point cloud in the prototype."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp","cited_arxiv_id":null,"evidence_quote":"Supplies the DAIR-V2X-Seq real-world V2I dataset with LiDAR, tracking, trajectory, and HD-map annotations used for all experiments."},{"cited_title":"In: Proceedings of 27 the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp","cited_arxiv_id":null,"evidence_quote":"Provides PointPillars, the 3D detector used as the first stage of the modular pipeline."},{"cited_title":"IEEE Transactions on Aerospace and Electronic Systems33(4), 1271–1276 (1997) https: //doi.org/10.1109/7.625124","cited_arxiv_id":null,"evidence_quote":"Supplies the covariance-weighted track-to-track fusion rule used for tracking-level fusion."}],"review_version":1}