Pith. sign in

REVIEW 54 references

ELMAR: Enhancing LiDAR Detection with 4D Radar Motion Awareness and Cross-modal Uncertainty

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2506.17958 v1 pith:PDOWPJ4E submitted 2025-06-22 cs.CV

Xiangyuan Peng , Miao Tang , Huawei Sun , Bierzynski Kay , Lorenzo Servadei , Robert Wille This is my paper
classification cs.CV
keywords radarlidarcross-modaldetectiondrivingenhanceinformationmethod
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

LiDAR and 4D radar are widely used in autonomous driving and robotics. While LiDAR provides rich spatial information, 4D radar offers velocity measurement and remains robust under adverse conditions. As a result, increasing studies have focused on the 4D radar-LiDAR fusion method to enhance the perception. However, the misalignment between different modalities is often overlooked. To address this challenge and leverage the strengths of both modalities, we propose a LiDAR detection framework enhanced by 4D radar motion status and cross-modal uncertainty. The object movement information from 4D radar is first captured using a Dynamic Motion-Aware Encoding module during feature extraction to enhance 4D radar predictions. Subsequently, the instance-wise uncertainties of bounding boxes are estimated to mitigate the cross-modal misalignment and refine the final LiDAR predictions. Extensive experiments on the View-of-Delft (VoD) dataset highlight the effectiveness of our method, achieving state-of-the-art performance with the mAP of 74.89% in the entire area and 88.70% within the driving corridor while maintaining a real-time inference speed of 30.02 FPS.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

54 extracted references · 3 canonical work pages

  1. [1]

    Robustness-aware 3d object detection in autonomous driving: A review and outlook,

    Z. Song, L. Liu, F. Jia, Y . Luo, C. Jia, G. Zhang, L. Yang, and L. Wang, “Robustness-aware 3d object detection in autonomous driving: A review and outlook,” IEEE Transactions on Intelligent Transportation Systems, 2024

    2024

  2. [2]

    4d mmwave radar for autonomous driving perception: a comprehensive survey,

    L. Fan, J. Wang, Y . Chang, Y . Li, Y . Wang, and D. Cao, “4d mmwave radar for autonomous driving perception: a comprehensive survey,” IEEE Transactions on Intelligent Vehicles , 2024

    2024

  3. [3]

    Event cameras in automotive sensing: A review,

    W. Shariff, M. S. Dilmaghani, P. Kielty, M. Moustafa, J. Lemley, and P. Corcoran, “Event cameras in automotive sensing: A review,” IEEE Access, 2024

    2024

  4. [4]

    Deep learning for lidar point clouds in autonomous driving: A review,

    Y . Li, L. Ma, Z. Zhong, F. Liu, M. A. Chapman, D. Cao, and J. Li, “Deep learning for lidar point clouds in autonomous driving: A review,” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 8, pp. 3412–3432, 2020

    2020

  5. [5]

    Object detection in adverse weather condition for autonomous vehicles,

    E. O. Appiah and S. Mensah, “Object detection in adverse weather condition for autonomous vehicles,” Multimedia Tools and Applica- tions, vol. 83, no. 9, pp. 28 235–28 261, 2024

    2024

  6. [6]

    A new wave in robotics: Survey on recent mmwave radar applications in robotics,

    K. Harlow, H. Jang, T. D. Barfoot, A. Kim, and C. Heckman, “A new wave in robotics: Survey on recent mmwave radar applications in robotics,” IEEE Transactions on Robotics , 2024

    2024

  7. [7]

    Multi- class road user detection with 3+ 1d radar in the view-of-delft dataset,

    A. Palffy, E. Pool, S. Baratam, J. F. Kooij, and D. M. Gavrila, “Multi- class road user detection with 3+ 1d radar in the view-of-delft dataset,” IEEE Robotics and Automation Letters , vol. 7, no. 2, pp. 4961–4968, 2022

    2022

  8. [8]

    Interfusion: Interaction-based 4d radar and lidar fusion for 3d object detection,

    L. Wang, X. Zhang, B. Xv, J. Zhang, R. Fu, X. Wang, L. Zhu, H. Ren, P. Lu, J. Li et al., “Interfusion: Interaction-based 4d radar and lidar fusion for 3d object detection,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2022, pp. 12 247–12 253

    2022

Show all 54 references
  1. [9]

    Multi-modal and multi-scale fusion 3d object detection of 4d radar and lidar for autonomous driving,

    L. Wang, X. Zhang, J. Li, B. Xv, R. Fu, H. Chen, L. Yang, D. Jin, and L. Zhao, “Multi-modal and multi-scale fusion 3d object detection of 4d radar and lidar for autonomous driving,” IEEE Transactions on Vehicular Technology, vol. 72, no. 5, pp. 5628–5641, 2022

    2022

  2. [10]

    Robust 3d object detection from lidar-radar point clouds via cross-modal feature augmentation,

    J. Deng, G. Chan, H. Zhong, and C. X. Lu, “Robust 3d object detection from lidar-radar point clouds via cross-modal feature augmentation,” in 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 6585–6591

    2024

  3. [11]

    Joint scene flow estimation and moving object segmentation on rotational lidar data,

    X. Chen, J. Cui, Y . Liu, X. Zhang, J. Sun, R. Ai, W. Gu, J. Xu, and H. Lu, “Joint scene flow estimation and moving object segmentation on rotational lidar data,” IEEE Transactions on Intelligent Transportation Systems, 2024

    2024

  4. [12]

    Mambamos: Lidar-based 3d moving object segmentation with motion-aware state space model,

    K. Zeng, H. Shi, J. Lin, S. Li, J. Cheng, K. Wang, Z. Li, and K. Yang, “Mambamos: Lidar-based 3d moving object segmentation with motion-aware state space model,” in Proceedings of the 32nd ACM International Conference on Multimedia , 2024, pp. 1505–1513

    2024

  5. [13]

    Mf-mos: A motion-focused model for moving object segmentation,

    J. Cheng, K. Zeng, Z. Huang, X. Tang, J. Wu, C. Zhang, X. Chen, and R. Fan, “Mf-mos: A motion-focused model for moving object segmentation,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) , 2024, pp. 12 499–12 505

    2024

  6. [14]

    Mutualforce: Mutual-aware enhancement for 4d radar-lidar 3d object detection,

    X. Peng, H. Sun, K. Bierzynski, A. Fischbacher, L. Servadei, and R. Wille, “Mutualforce: Mutual-aware enhancement for 4d radar-lidar 3d object detection,” arXiv preprint arXiv:2501.10266 , 2025

    2025 arXiv

  7. [15]

    Radar velocity transformer: Single-scan moving object segmentation in noisy radar point clouds,

    M. Zeller, V . S. Sandhu, B. Mersch, J. Behley, M. Heidingsfeld, and C. Stachniss, “Radar velocity transformer: Single-scan moving object segmentation in noisy radar point clouds,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) , 2023, pp. 7054– 7061

    2023

  8. [16]

    L4dr: Lidar-4dradar fusion for weather-robust 3d object detection,

    X. Huang, Z. Xu, H. Wu, J. Wang, Q. Xia, Y . Xia, J. Li, K. Gao, C. Wen, and C. Wang, “L4dr: Lidar-4dradar fusion for weather-robust 3d object detection,” arXiv preprint arXiv:2408.03677 , 2024

    2024 arXiv

  9. [17]

    Towards robust 3d object detection with lidar and 4d radar fusion in various weather conditions,

    Y . Chae, H. Kim, and K.-J. Yoon, “Towards robust 3d object detection with lidar and 4d radar fusion in various weather conditions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 15 162–15 172

    2024

  10. [18]

    Bi-lrfusion: Bi-directional lidar-radar fusion for 3d dy- namic object detection,

    Y . Wang, J. Deng, Y . Li, J. Hu, C. Liu, Y . Zhang, J. Ji, W. Ouyang, and Y . Zhang, “Bi-lrfusion: Bi-directional lidar-radar fusion for 3d dy- namic object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 13 394–13 403

    2023

  11. [19]

    Pointpillars: Fast encoders for object detection from point clouds,

    A. H. Lang, S. V ora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for object detection from point clouds,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 12 697–12 705

    2019

  12. [20]

    Swiftpillars: High- efficiency pillar encoder for lidar-based 3d detection,

    X. Jin, K. Liu, C. Ma, R. Yang, F. Hui, and W. Wu, “Swiftpillars: High- efficiency pillar encoder for lidar-based 3d detection,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 3, 2024, pp. 2625–2633

    2024

  13. [21]

    V oxelnext: Fully sparse voxelnet for 3d object detection and tracking,

    Y . Chen, J. Liu, X. Zhang, X. Qi, and J. Jia, “V oxelnext: Fully sparse voxelnet for 3d object detection and tracking,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 21 674–21 683

    2023

  14. [22]

    3dssd: Point-based 3d single stage object detector,

    Z. Yang, Y . Sun, S. Liu, and J. Jia, “3dssd: Point-based 3d single stage object detector,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 11 040–11 048

    2020

  15. [23]

    Psns-ssd: Pixel-level suppressed nonsalient semantic and multicoupled channel enhance- ment attention for 3d object detection,

    X. Song, Z. Zhou, L. Zhang, X. Lu, and X. Hei, “Psns-ssd: Pixel-level suppressed nonsalient semantic and multicoupled channel enhance- ment attention for 3d object detection,” IEEE Robotics and Automation Letters, vol. 9, no. 1, pp. 603–610, 2023

    2023

  16. [24]

    Not all points are equal: Learning highly efficient point-based detectors for 3d lidar point clouds,

    Y . Zhang, Q. Hu, G. Xu, Y . Ma, J. Wan, and Y . Guo, “Not all points are equal: Learning highly efficient point-based detectors for 3d lidar point clouds,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 18 953–18 962

    2022

  17. [25]

    Pv-rcnn: Point-voxel feature set abstraction for 3d object detection,

    S. Shi, C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li, “Pv-rcnn: Point-voxel feature set abstraction for 3d object detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 10 529–10 538

    2020

  18. [26]

    Pv-rcnn++: Point-voxel feature set abstraction with local vector representation for 3d object detection,

    S. Shi, L. Jiang, J. Deng, Z. Wang, C. Guo, J. Shi, X. Wang, and H. Li, “Pv-rcnn++: Point-voxel feature set abstraction with local vector representation for 3d object detection,” International Journal of Computer Vision , vol. 131, no. 2, pp. 531–551, 2023

    2023

  19. [27]

    Dpa-rcnn: Dual position aware 3d object detector for point cloud,

    Y . Jiang, Q. Xie, J. Li, J. Xu, Y . Liu, and Y . Ma, “Dpa-rcnn: Dual position aware 3d object detector for point cloud,” in 2024 International Joint Conference on Neural Networks (IJCNN) . IEEE, 2024, pp. 1–9

    2024

  20. [28]

    Full waveform lidar for adverse weather conditions,

    A. M. Wallace, A. Halimi, and G. S. Buller, “Full waveform lidar for adverse weather conditions,” IEEE transactions on vehicular technology, vol. 69, no. 7, pp. 7064–7077, 2020

    2020

  21. [29]

    Smurf: Spatial multi-representation fusion for 3d object detection with 4d imaging radar,

    J. Liu, Q. Zhao, W. Xiong, T. Huang, Q.-L. Han, and B. Zhu, “Smurf: Spatial multi-representation fusion for 3d object detection with 4d imaging radar,”IEEE Transactions on Intelligent Vehicles, vol. 9, no. 1, pp. 799–812, 2024

    2024

  22. [30]

    Mvfan: Multi-view feature assisted network for 4d radar object detection,

    Q. Yan and Y . Wang, “Mvfan: Multi-view feature assisted network for 4d radar object detection,” in International Conference on Neural Information Processing. Springer, 2023, pp. 493–511

    2023

  23. [31]

    Mufasa: Multi-view fusion and adaptation network with spatial awareness for radar object detection,

    X. Peng, M. Tang, H. Sun, K. Bierzynski, L. Servadei, and R. Wille, “Mufasa: Multi-view fusion and adaptation network with spatial awareness for radar object detection,” in International Conference on Artificial Neural Networks . Springer, 2024, pp. 168–184

    2024

  24. [32]

    Sparsein- teraction: Sparse semantic guidance for radar and camera 3d object detection,

    S. Jiang, S. Xu, L. Liu, Z. Song, Y . Bo, Z.-X. Yang et al., “Sparsein- teraction: Sparse semantic guidance for radar and camera 3d object detection,” in ACM Multimedia 2024

    2024

  25. [33]

    Rectifying pseudo label learning via un- certainty estimation for domain adaptive semantic segmentation,

    Z. Zheng and Y . Yang, “Rectifying pseudo label learning via un- certainty estimation for domain adaptive semantic segmentation,” International Journal of Computer Vision , vol. 129, no. 4, pp. 1106– 1120, 2021

    2021

  26. [34]

    Augmenting 3d object detection through data uncertainty-driven auxiliary framework,

    J. Wang, S. Zhao, and S. Liang, “Augmenting 3d object detection through data uncertainty-driven auxiliary framework,” IEEE Transac- tions on Instrumentation and Measurement , 2024

    2024

  27. [35]

    Towards maximizing the rep- resentation gap between in-domain & out-of-distribution examples,

    J. Nandy, W. Hsu, and M. L. Lee, “Towards maximizing the rep- resentation gap between in-domain & out-of-distribution examples,” Advances in neural information processing systems, vol. 33, pp. 9239– 9250, 2020

    2020

  28. [36]

    Gradients as a measure of uncertainty in neural networks,

    J. Lee and G. AlRegib, “Gradients as a measure of uncertainty in neural networks,” in 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 2020, pp. 2416–2420

    2020

  29. [37]

    Dropconnect is effective in modeling uncertainty of bayesian deep networks,

    A. Mobiny, P. Yuan, S. K. Moulik, N. Garg, C. C. Wu, and H. Van Nguyen, “Dropconnect is effective in modeling uncertainty of bayesian deep networks,” Scientific reports, vol. 11, no. 1, p. 5458, 2021

    2021

  30. [38]

    Pasco: Urban 3d panoptic scene completion with uncertainty awareness,

    A.-Q. Cao, A. Dai, and R. de Charette, “Pasco: Urban 3d panoptic scene completion with uncertainty awareness,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 14 554–14 564

    2024

  31. [39]

    Uncertainty-aware ab3dmot by vari- ational 3d object detection,

    I. Oleksiienko and A. Iosifidis, “Uncertainty-aware ab3dmot by vari- ational 3d object detection,” in 2024 IEEE International Conference on Image Processing (ICIP) , 2024, pp. 3389–3395

    2024

  32. [40]

    Spsnet: Boosting 3d point-based object detectors with stable point sampling,

    A. Liang, H. Zhang, H. Hua, W. Chen, and H. Zhao, “Spsnet: Boosting 3d point-based object detectors with stable point sampling,” Engineering Applications of Artificial Intelligence, vol. 126, p. 106807, 2023

    2023

  33. [41]

    Glenet: Boosting 3d object detectors with generative label uncertainty estimation,

    Y . Zhang, Q. Zhang, Z. Zhu, J. Hou, and Y . Yuan, “Glenet: Boosting 3d object detectors with generative label uncertainty estimation,” International Journal of Computer Vision, vol. 131, no. 12, pp. 3332– 3352, 2023

    2023

  34. [42]

    Uncertainty-encoded multi-modal fusion for robust object detection in autonomous driving,

    Y . Lou, Q. Song, Q. Xu, R. Tan, and J. Wang, “Uncertainty-encoded multi-modal fusion for robust object detection in autonomous driving,” in ECAI 2023. IOS Press, 2023, pp. 1593–1600

    2023

  35. [43]

    Cocoon: Robust multi-modal perception with uncertainty- aware sensor fusion,

    M. Cho, Y . Cao, J. Sun, Q. Zhang, M. Pavone, J. J. Park, H. Yang, and Z. M. Mao, “Cocoon: Robust multi-modal perception with uncertainty- aware sensor fusion,” arXiv preprint arXiv:2410.12592 , 2024

    2024 arXiv

  36. [44]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space,

    C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” Advances in neural information processing systems , vol. 30, 2017

    2017

  37. [45]

    Harnessing uncertainty- aware bounding boxes for unsupervised 3d object detection,

    R. Zhang, H. Zhang, H. Yu, and Z. Zheng, “Harnessing uncertainty- aware bounding boxes for unsupervised 3d object detection,” arXiv preprint arXiv:2408.00619, 2024

    2024 arXiv

  38. [46]

    The hungarian method for the assignment problem,

    H. W. Kuhn, “The hungarian method for the assignment problem,” Naval research logistics quarterly , vol. 2, no. 1-2, pp. 83–97, 1955

    1955

  39. [47]

    Openpcdet: An open-source toolbox for 3d object detection from point clouds,

    O. Team et al. , “Openpcdet: An open-source toolbox for 3d object detection from point clouds,” OD Team, 2020

    2020

  40. [48]

    Lxl: Lidar excluded lean 3d object detection with 4d imaging radar and camera fusion,

    W. Xiong, J. Liu, T. Huang, Q.-L. Han, Y . Xia, and B. Zhu, “Lxl: Lidar excluded lean 3d object detection with 4d imaging radar and camera fusion,” IEEE Transactions on Intelligent Vehicles , vol. 9, no. 1, pp. 79–92, 2024

    2024

  41. [49]

    Bevfusion: Multi-task multi-sensor fusion with unified bird’s- eye view representation,

    Z. Liu, H. Tang, A. Amini, X. Yang, H. Mao, D. L. Rus, and S. Han, “Bevfusion: Multi-task multi-sensor fusion with unified bird’s- eye view representation,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) , 2023, pp. 2774–2781

    2023

  42. [50]

    Rcfusion: Fusing 4-d radar and camera with bird’s- eye view features for 3-d object detection,

    L. Zheng, S. Li, B. Tan, L. Yang, S. Chen, L. Huang, J. Bai, X. Zhu, and Z. Ma, “Rcfusion: Fusing 4-d radar and camera with bird’s- eye view features for 3-d object detection,” IEEE Transactions on Instrumentation and Measurement , vol. 72, pp. 1–14, 2023

    2023

  43. [51]

    Rcbevdet: Radar-camera fusion in bird’s eye view for 3d object detection,

    Z. Lin, Z. Liu, Z. Xia, X. Wang, Y . Wang, S. Qi, Y . Dong, N. Dong, L. Zhang, and C. Zhu, “Rcbevdet: Radar-camera fusion in bird’s eye view for 3d object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 14 928–14 937

    2024

  44. [52]

    Lxl: Lidar excluded lean 3d object detection with 4d imaging radar and camera fusion,

    W. Xiong, J. Liu, T. Huang, Q.-L. Han, Y . Xia, and B. Zhu, “Lxl: Lidar excluded lean 3d object detection with 4d imaging radar and camera fusion,” IEEE Transactions on Intelligent Vehicles , 2023

    2023

  45. [53]

    Radarocc: Robust 3d occupancy prediction with 4d imaging radar,

    F. Ding, X. Wen, Y . Zhu, Y . Li, and C. X. Lu, “Radarocc: Robust 3d occupancy prediction with 4d imaging radar,” Advances in Neural Information Processing Systems , vol. 37, pp. 101 589–101 617, 2025

    2025

  46. [54]

    Sgdet3d: Semantics and geometry fusion for 3d object detection using 4d radar and camera,

    X. Bai, Z. Yu, L. Zheng, X. Zhang, Z. Zhou, X. Zhang, F. Wang, J. Bai, and H.-L. Shen, “Sgdet3d: Semantics and geometry fusion for 3d object detection using 4d radar and camera,” IEEE Robotics and Automation Letters, vol. 10, no. 1, pp. 828–835, 2025

    2025

Pith tools