pith. sign in

arxiv: 2605.00595 · v1 · submitted 2026-05-01 · 💻 cs.CV · cs.RO

Robust Fusion of Object-Level V2X for Learned 3D Object Detection

Lukas Ostendorf , Lennart Reiher , Onn Haran , Lutz Eckstein This is my paper

Pith reviewed 2026-05-09 18:57 UTC · model grok-4.3

classification 💻 cs.CV cs.RO
keywords 3D object detectionV2X communicationcooperative perceptionnoise robustnessBEV fusionautomated drivingnuScenes
0
0 comments X

The pith

Noise-aware training with confidence encoding keeps object-level V2X fusion effective for 3D detection under realistic imperfections.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines how object-level messages shared through V2X communication can supplement onboard sensors for 3D object detection in automated driving. It creates realistic test conditions on the nuScenes dataset by adding controlled noise and dropping objects from ground-truth data to simulate latency, localization errors, and low participation rates. Standard training on clean V2X data produces models that perform well only when messages are perfect but collapse when imperfections appear. Adding noise during training and explicitly encoding message confidence produces detectors that retain most of the performance benefit even when V2X data is severely degraded or scarce.

Core claim

Object-level V2X messages converted to a dedicated bird's-eye-view input and fused inside a BEVFusion-style detector raise 3D detection performance to an NDS of 0.80 when conditions are favorable. Models trained exclusively on idealized messages become fragile and over-dependent on V2X. A noise-aware training regimen that injects realistic imperfections together with explicit confidence encoding preserves the accuracy gains across wide ranges of noise severity and reduced V2X penetration rates.

What carries the argument

The noise-aware training strategy that injects controlled noise and object dropout into emulated V2X messages while encoding message confidence as an additional input channel to the BEV fusion detector.

If this is right

  • V2X object data raises NDS from baseline onboard-only levels to 0.80 when messages arrive with low noise.
  • Training without noise injection produces models that lose most gains once V2X messages contain realistic errors or dropouts.
  • The noise-aware strategy with confidence encoding maintains measurable detection improvement even at high noise levels and low V2X participation rates.
  • The same BEV fusion architecture accepts both ideal and degraded V2X inputs once trained with the proposed method.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same training approach could be applied to other cooperative inputs such as infrastructure sensor data without requiring perfect synchronization.
  • Tolerating higher message error rates may allow lower-bandwidth or lower-frequency V2X transmissions while still delivering useful detection gains.
  • Real deployments could use live estimates of message quality to modulate the weight given to each V2X object rather than relying on a static encoding.
  • The method suggests that multi-vehicle scenes with uneven participation rates remain workable provided the detector has seen similar dropout patterns during training.

Load-bearing premise

The controlled injection of noise and object dropout into ground-truth-derived V2X messages accurately emulates real-world imperfections such as latency, localization errors, and low penetration rates.

What would settle it

Running the trained detector on a dataset of actual V2X messages collected from real vehicles and infrastructure would show whether performance under emulated noise matches performance under physical communication errors.

Figures

Figures reproduced from arXiv: 2605.00595 by Lennart Reiher, Lukas Ostendorf, Lutz Eckstein, Onn Haran.

Figure 1
Figure 1. Figure 1: Comparison of perception modalities: (left) onboard sensors with [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Architecture overview: BEVFusion [6] with an added V2X branch and gating-based fusion at the BEV feature level. The V2X BEV input is encoded by a ResNet [20] backbone, aligned with camera and radar features, and modulated by a learnable gate before entering the detection head to weight V2X contributions by reliability. absent, making our evaluation conservative. This setup allows systematic variation of lo… view at source ↗
Figure 3
Figure 3. Figure 3: BEV input visualization: (left) ground-truth 3D boxes in the ego [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Noise robustness for Experiments A–C: NDS versus noise for camera+V2X (top, panels (a)–(e)) and radar+camera+V2X (bottom, panels (f)–(j)). [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Per-class AP under medium-high noise comparing: camera-only [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Robustness to partial V2X penetration: NDS versus V2X penetration [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
read the original abstract

Perception for automated driving is largely based on onboard environmental sensors, such as cameras and radar, which are cost-effective but limited by line-of-sight and field-of-view constraints. These inherent limitations may cause onboard perception to fail under occlusions or poor visibility conditions. In parallel, cooperative awareness via vehicle-to-everything (V2X) communication is becoming increasingly available, enabling vehicles and infrastructure to share their own state as object-level information that complements onboard perception. In this work, we study how such V2X information can be integrated into 3D object detection and how robust the resulting system is to realistic V2X imperfections. Using the nuScenes dataset, we emulate object-level cooperative awareness messages from ground truth, injecting controlled noise and object dropout to mimic real-world conditions such as latency, localization errors, and low V2X penetration rates. We convert these messages into a dedicated bird's-eye view (BEV) input and fuse them into a BEVFusion-style detector. Our results demonstrate that while object-level cooperative information can substantially improve detection performance, achieving an NDS of 0.80 under favorable conditions, models trained on idealized data become fragile and over-reliant on V2X. Conversely, our proposed noise-aware training strategy, coupled with explicit confidence encoding, enhances robustness, maintaining performance gains even under severe noise and reduced V2X penetration.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper studies fusion of object-level V2X cooperative awareness messages into a BEVFusion-style 3D object detector for automated driving. On nuScenes, V2X inputs are emulated by perturbing ground-truth boxes with controlled noise (latency, localization) and random dropout (penetration rate). The authors introduce noise-aware training together with explicit confidence encoding in the BEV input and report that this yields an NDS of 0.80 under clean conditions while preserving gains under severe synthetic noise and low penetration, in contrast to models trained on idealized data that become fragile.

Significance. If the robustness result transfers beyond the synthetic emulation, the work would provide a concrete, trainable recipe for making object-level V2X fusion reliable under realistic imperfections, addressing a key barrier to cooperative perception. The use of a public dataset and explicit noise-injection protocol makes the empirical claims reproducible in principle and supplies a useful baseline for future studies that replace the emulation with real detector outputs.

major comments (2)
  1. [Methods (V2X emulation)] Methods / V2X emulation paragraph: the noise model is generated exclusively by adding controlled perturbations and dropout to nuScenes ground-truth boxes. No comparison or calibration against real V2X traces, real detector outputs from transmitting agents, or measured latency distributions is reported. Because the central robustness claim rests on this emulation accurately capturing false positives, misses, classification errors, and correlated sensor noise, the absence of such validation is load-bearing for the transferability of the reported gains.
  2. [Results (ablation studies)] Results / ablation tables: while the abstract states that noise-aware training plus confidence encoding maintains performance under severe noise, the manuscript does not appear to contain a full factorial ablation that isolates the contribution of each component (noise injection schedule, confidence channel, training regime) across the full range of penetration rates. Without these tables it is difficult to determine whether the robustness is attributable to the proposed strategy or to other unstated design choices.
minor comments (2)
  1. [Methods] Notation for the BEV input tensor and the confidence encoding channel should be defined once in a single equation or table rather than scattered across text and figures.
  2. [Methods] The exact functional form of the noise injection (Gaussian parameters, dropout probability schedule) is described only qualitatively; a compact table listing the ranges used for each experiment would improve reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback on our manuscript. We have carefully reviewed each major comment and provide point-by-point responses below. We are prepared to revise the paper accordingly to strengthen the presentation and address the concerns.

read point-by-point responses

  1. Referee: Methods / V2X emulation paragraph: the noise model is generated exclusively by adding controlled perturbations and dropout to nuScenes ground-truth boxes. No comparison or calibration against real V2X traces, real detector outputs from transmitting agents, or measured latency distributions is reported. Because the central robustness claim rests on this emulation accurately capturing false positives, misses, classification errors, and correlated sensor noise, the absence of such validation is load-bearing for the transferability of the reported gains.

    Authors: We acknowledge that the emulation relies on synthetic perturbations to ground-truth boxes rather than direct calibration against real V2X traces or detector outputs, which limits direct claims about transfer to field deployments. This choice was made to enable fully reproducible, controlled experiments on a public dataset where real V2X data is unavailable. In the revised manuscript we will expand the Methods section with additional justification for the chosen noise parameters (drawing from published V2X latency and localization statistics), include a dedicated limitations paragraph discussing the synthetic nature of the emulation, and clarify that the reported robustness gains are with respect to the modeled imperfections rather than claiming equivalence to real-world V2X. revision: partial

  2. Referee: Results / ablation tables: while the abstract states that noise-aware training plus confidence encoding maintains performance under severe noise, the manuscript does not appear to contain a full factorial ablation that isolates the contribution of each component (noise injection schedule, confidence channel, training regime) across the full range of penetration rates. Without these tables it is difficult to determine whether the robustness is attributable to the proposed strategy or to other unstated design choices.

    Authors: We appreciate the request for clearer isolation of components. The current manuscript already reports performance for the full proposed method versus idealized training across multiple noise levels and penetration rates, but does not present an exhaustive factorial design. In the revision we will add a new ablation table that systematically varies the presence/absence of noise-aware training and the confidence encoding channel, evaluated at representative penetration rates (10 %, 50 %, 100 %). This will make the individual and synergistic contributions explicit. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical robustness claims rest on synthetic perturbations of public dataset, not self-referential definitions or fits

full rationale

The paper describes an empirical pipeline: ground-truth boxes from nuScenes are perturbed with controlled noise/dropout to create synthetic V2X inputs, which are then encoded as BEV features and fused into a BEVFusion-style detector. Models are trained with a noise-aware strategy plus confidence encoding and evaluated on held-out synthetic test cases. No equations, self-citations, or ansatzes are invoked that reduce the reported NDS gains or robustness metrics to quantities defined or fitted inside the same experiment. The evaluation is externally benchmarked against the public nuScenes split and standard detection metrics; the central claim therefore remains falsifiable outside the training loop and does not collapse by construction.

Axiom & Free-Parameter Ledger

2 free parameters · 1 axioms · 0 invented entities

The central claim rests on the fidelity of the synthetic V2X emulation and on the assumption that the BEVFusion backbone can be extended with an additional BEV channel without architectural changes that would invalidate the robustness findings.

free parameters (2)
  • noise injection parameters
    Specific distributions for latency, localization error, and object dropout chosen by the authors to emulate real conditions.
  • V2X penetration rate schedule
    Fraction of vehicles sending messages varied across experiments.
axioms (1)
  • domain assumption Ground-truth annotations in nuScenes can be treated as perfect object-level V2X messages before noise injection.
    Used to generate the cooperative input stream.

pith-pipeline@v0.9.0 · 5554 in / 1280 out tokens · 50102 ms · 2026-05-09T18:57:17.328261+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

20 extracted references · 20 canonical work pages

  1. [1]

    Global status report on road safety 2023,

    World Health Organization, “Global status report on road safety 2023,” World Health Organization, Geneva, Switzerland, Tech. Rep., 2023

    work page 2023

  2. [2]

    E. D. Kaplan and C. J. Hegarty,Understanding GPS/GNSS: Principles and Applications, 3rd ed. Norwood, MA: Artech House, 2017

    work page 2017

  3. [3]

    Shadow matching: A new gnss positioning technique for urban canyons,

    P. D. Groves, “Shadow matching: A new gnss positioning technique for urban canyons,”Journal of Navigation, vol. 64, no. 3, 2011

    work page 2011

  4. [4]

    End-to-end v2x latency modeling and analysis in 5g networks,

    B. Coll-Perales, M. C. Lucas-Est ˜an, T. Shimizu, J. Gozalvez, T. Higuchi, S. Avedisov, O. Altintas, and M. Sepulcre, “End-to-end v2x latency modeling and analysis in 5g networks,”IEEE Transactions on Vehicular Technology, vol. 72, no. 4, 2023

    work page 2023

  5. [5]

    nuscenes: A multimodal dataset for autonomous driving,

    H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,”Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020

    work page 2020

  6. [6]

    Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation,

    Z. Liu, H. Tang, A. Amini, X. Yang, H. Mao, D. Rus, and S. Han, “Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation,” inIEEE International Conference on Robotics and Automation (ICRA), 2023

    work page 2023

  7. [7]

    Mec-assisted end-to- end latency evaluations for c-v2x communications,

    M. Emara, M. C. Filippou, and D. Sabella, “Mec-assisted end-to- end latency evaluations for c-v2x communications,” in2018 European Conference on Networks and Communications (EuCNC), 2018

    work page 2018

  8. [8]

    V2aix: A multi- modal real-world dataset of etsi its v2x messages in public road traffic,

    G. K ¨uppers, J.-P. Busch, L. Reiher, and L. Eckstein, “V2aix: A multi- modal real-world dataset of etsi its v2x messages in public road traffic,” in2024 IEEE International Conference on Intelligent Transportation Systems (ITSC), Edmonton, Canada, 2024

    work page 2024

  9. [9]

    Cooper: Cooperative perception for connected autonomous vehicles based on 3d point clouds,

    Q. Chen, S. Tang, Q. Yang, and S. Fu, “Cooper: Cooperative perception for connected autonomous vehicles based on 3d point clouds,” inIEEE 39th International Conference on Distributed Computing Systems, 2019

    work page 2019

  10. [10]

    F-cooper: Feature based cooperative perception for autonomous vehicle edge computing system using 3d point clouds,

    Q. Chen, X. Ma, S. Tang, J. Guo, Q. Yang, and S. Fu, “F-cooper: Feature based cooperative perception for autonomous vehicle edge computing system using 3d point clouds,” inProceedings of the 4th ACM/IEEE Symposium on Edge Computing. ACM, 2019

    work page 2019

  11. [11]

    V2x-vit: Vehicle-to-everything cooperative perception with vision transformer,

    R. Xu, H. Xiang, Z. Tu, X. Xia, M.-H. Yang, and J. Ma, “V2x-vit: Vehicle-to-everything cooperative perception with vision transformer,” inComputer Vision – ECCV Workshops. Springer, 2022

    work page 2022

  12. [12]

    Keypoints-based deep feature fusion for cooperative vehicle detection of autonomous driving,

    Y . Yuan, H. Cheng, and M. Sester, “Keypoints-based deep feature fusion for cooperative vehicle detection of autonomous driving,”IEEE Robotics and Automation Letters, vol. 7, no. 2, 2022

    work page 2022

  13. [13]

    Dair-v2x: A large-scale dataset for vehicle- infrastructure cooperative 3d object detection,

    H. Yu, Y . Luo, M. Shu, Y . Huo, Z. Yang, Y . Shi, Z. Guo, H. Li, X. Hu, J. Yuan, and Z. Nie, “Dair-v2x: A large-scale dataset for vehicle- infrastructure cooperative 3d object detection,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    work page 2022

  14. [14]

    Opv2v: An open benchmark dataset and fusion pipeline for perception with vehicle-to- vehicle communication,

    R. Xu, H. Xiang, X. Xia, X. Han, J. Li, and J. Ma, “Opv2v: An open benchmark dataset and fusion pipeline for perception with vehicle-to- vehicle communication,” in2022 IEEE International Conference on Robotics and Automation (ICRA), 2022

    work page 2022

  15. [15]

    A novel probabilistic v2x data fusion framework for cooperative perception,

    M. Shan, K. Narula, S. Worrall, Y . Wong, J. S. B. Perez, P. Gray, and E. Nebot, “A novel probabilistic v2x data fusion framework for cooperative perception,” inIEEE 25th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2022

    work page 2022

  16. [16]

    Monocular 3d object detection for autonomous driving,

    X. Chen, K. Kundu, Z. Zhang, H. Ma, S. Fidler, and R. Urtasun, “Monocular 3d object detection for autonomous driving,” inIEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2016

    work page 2016

  17. [17]

    Learn- ing depth-guided convolutions for monocular 3d object detection,

    M. Ding, Y . Huo, H. Yi, Z. Wang, J. Shi, Z. Lu, and P. Luo, “Learn- ing depth-guided convolutions for monocular 3d object detection,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Jun. 2020

    work page 2020

  18. [18]

    Bevdepth: Acquisition of reliable depth for multi-view 3d object detection,

    Y . Li, Z. Ge, G. Yu, J. Yang, Z. Wang, Y . Shi, J. Sun, and Z. Li, “Bevdepth: Acquisition of reliable depth for multi-view 3d object detection,” inProceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence, 2023

    work page 2023

  19. [19]

    Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers,

    Z. Li, W. Wang, H. Li, E. Xie, C. Sima, T. Lu, Y . Qiao, and J. Dai, “Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers,” inComputer Vision – ECCV Lecture Notes in Computer Science, vol. 13669. Springer, 2022

    work page 2022

  20. [20]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inIEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016

    work page 2016