pith. sign in

arxiv: 2207.07228 · v2 · submitted 2022-07-14 · 📡 eess.SP

Multi-FEAT: Multi-Feature Edge Alignment for Targetless Camera-LiDAR Calibration

Bichi Zhang , Holger Caesar , Raj Thilak Rajan This is my paper

Pith reviewed 2026-05-24 11:41 UTC · model grok-4.3

classification 📡 eess.SP
keywords targetless calibrationcamera-LiDARextrinsic calibrationedge alignmentcylindrical projectionfeature matchingsensor fusion
0
0 comments X

The pith

Multi-FEAT achieves reliable targetless camera-LiDAR calibration by aligning multi-feature edges extracted from cylindrical LiDAR panoramas with camera edges.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Multi-FEAT as a targetless approach to determine the extrinsic parameters between a camera and LiDAR sensor. It projects the 3D LiDAR point cloud into a 2D cylindrical panorama to draw on diverse feature information that supplements the sparse boundaries in the original point cloud, then matches those features against camera edges obtained from segmentation to drive parameter optimization. A sympathetic reader would care because multi-agent systems such as automobiles and UAVs require precise sensor alignment for accurate environmental perception, and the method reports more reliable outcomes than several prior targetless techniques when tested on standard data.

Core claim

Multi-FEAT encodes the 3D LiDAR point cloud into a 2D panorama via the cylindrical projection model and exploits diverse LiDAR feature information in the panoramic images to supplement the sparse LiDAR point cloud boundaries. Camera edges are extracted using off-the-shelf segmentation solutions, after which a feature-matching function is used to optimize the calibration parameters, producing more reliable results than several existing targetless calibration methods.

What carries the argument

The feature-matching function that optimizes extrinsic parameters by aligning multi-feature edges between cylindrical LiDAR panoramas and camera images.

If this is right

  • Targetless calibration becomes feasible for in-field use in automobiles and UAVs without special calibration objects.
  • Sparse LiDAR boundaries can be made usable for alignment by supplementing them with panoramic feature information.
  • The resulting extrinsic parameters support more accurate sensor fusion than those from several prior targetless methods.
  • The approach supplies a concrete starting point for further development of feature-based calibration routines.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same cylindrical-projection step could be tested with other feature types or with radar data to check whether the edge-alignment benefit generalizes.
  • Performance may vary with the choice of segmentation tool for camera edges, suggesting an ablation on that component would be informative.
  • If the matching function converges quickly, the method could support periodic re-calibration during vehicle operation rather than only at setup time.

Load-bearing premise

Diverse LiDAR feature information extracted from cylindrical panoramic images can sufficiently supplement the sparse point cloud boundaries to support reliable edge alignment and parameter optimization.

What would settle it

A side-by-side evaluation on new sensor data in which the parameters returned by the Multi-FEAT matching function produce higher calibration error than one or more of the compared targetless baselines.

Figures

Figures reproduced from arXiv: 2207.07228 by Bichi Zhang, Holger Caesar, Raj Thilak Rajan.

Figure 1
Figure 1. Figure 1: Visualization of the proposed Multi-FEAT algorithm. We cover [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The proposed Multi-FEAT workflow. ψ is the input point cloud, while I is the camera image. The output of the workflow is the estimated calibration parameter θˆ. Parameters d, r, o represent depth, reflectivity and object features, respectively. Coordinates i, j shows the 3D-2D conversion after cylindrical projection. extrinsic calibration parameters θ = [rx, ry, rz, tx, ty, tz] T . In the following section… view at source ↗
Figure 3
Figure 3. Figure 3: Illustrations of image processing: The details in the Sobel edge map of the original image in (a) are not clearly visible, while those details are mostly [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Effect of RANSAC and DBSCAN algorithms: (a) The Birds’ eye view (BEV) of the raw LiDAR point cloud ( [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: An example of the occlusion effect and its influence on the edges. The [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The dense maps of depth, reflectivity, and object features, were [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The edge maps of depth, reflectivity, foreground objects and the [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Scenario 1 (Urban Road) shows a typical example of an road junction, where there are pedestrians, vehicles, cyclists and buildings in the backdrop, all within the field of view of the experimental vehicle. while the rests need maximization. For the sake of clarity, we normalize the cost functions into interval [0, 1]. A summary of all the hyper-parameters used in the Multi-FEAT optimization problem is summ… view at source ↗
Figure 9
Figure 9. Figure 9: Scenario 2 (Highway) presents an example of highway scenario. LiDAR cannot see through the near-field vehicles around, therefore providing little information from farther objects. degrade the solution. In comparison to the other methods, the proposed Multi-FEAT shows a smoother curve for almost all the parameters, with a peak almost centering at zero, which is promising. 2) Scenario 2: Highway: The second … view at source ↗
Figure 11
Figure 11. Figure 11: The box plots show the error residuals {i} 30 i=1 of calibration parameters estimated using the Multi-FEAT algorithm over 30 frames, where the true calibration parameters are constant. B. Multi-frame evaluation In the previous sections, we investigated the performance of the proposed Multi-FEAT against using various single￾frame scenarios of the KITTI dataset. In this section, we arbitrarily select multi… view at source ↗
read the original abstract

Multi-agent systems, e.g., automobiles and UAVs (Unmanned Ariel Vehicles), rely on the precision of onboard sensors to accurately perceive their environment, which in turn depends on the precision of onboard sensors and reliable in-field calibration. This paper introduces a novel targetless camera-LiDAR extrinsic calibration approach called Multi-FEAT (Multi-Feature Edge AlignmenT). Multi-FEAT uses the cylindrical projection model to encode the 3D LiDAR point cloud into a 2D panorama and exploits diverse LiDAR feature information in panoramic images to supplement the sparse LiDAR point cloud boundaries. Furthermore, camera edges are extracted using off-the-shelf segmentation solutions. In addition, a feature-matching function is designed to optimize the calibration parameters. The performance of the proposed Multi-FEAT algorithm is evaluated using the KITTI dataset, and our approach shows more reliable results than several existing targetless calibration methods. We conclude our analysis with directions for future work.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The manuscript introduces Multi-FEAT, a targetless camera-LiDAR extrinsic calibration method. It encodes the 3D LiDAR point cloud via cylindrical projection into a 2D panorama, extracts diverse LiDAR features to supplement sparse boundaries, detects camera edges with off-the-shelf segmentation, and optimizes extrinsic parameters with a designed feature-matching function. The approach is evaluated on the KITTI dataset and claimed to yield more reliable results than existing targetless methods.

Significance. If the empirical claims are substantiated with quantitative metrics, the work could provide a practical advance in targetless calibration for multi-agent systems by addressing LiDAR sparsity through multi-feature augmentation in panoramic projections. The pipeline is internally coherent with no evident circularity or unsupported logical steps.

major comments (1)
  1. Abstract: the central claim that the method 'shows more reliable results than several existing targetless calibration methods' on KITTI supplies no quantitative metrics, error statistics, baseline comparisons, or optimization details, rendering the performance assertion unverifiable from the text and load-bearing for the contribution.
minor comments (1)
  1. Abstract: 'Unmanned Ariel Vehicles' is a typographical error and should read 'Unmanned Aerial Vehicles'.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the detailed review and constructive comment. We address the single major comment point-by-point below.

read point-by-point responses

  1. Referee: Abstract: the central claim that the method 'shows more reliable results than several existing targetless calibration methods' on KITTI supplies no quantitative metrics, error statistics, baseline comparisons, or optimization details, rendering the performance assertion unverifiable from the text and load-bearing for the contribution.

    Authors: We agree that the abstract, as currently written, states the performance claim without accompanying quantitative support, which limits immediate verifiability. The body of the manuscript contains the full experimental results on KITTI (including error statistics, baseline comparisons, and optimization details), but the abstract does not summarize them. We will revise the abstract to incorporate concise quantitative metrics (e.g., mean rotation/translation errors and comparisons) that substantiate the claim while remaining within length constraints. revision: yes

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The paper describes a calibration pipeline (cylindrical LiDAR projection to panorama, multi-feature extraction to augment boundaries, off-the-shelf camera edge detection, and a designed matching function for extrinsic optimization) followed by empirical evaluation on KITTI. No equations, derivations, fitted parameters presented as predictions, or self-citation chains appear in the abstract or described method. The central performance claim is a comparative empirical assertion against existing methods, not a reduction to inputs by construction. The derivation chain is therefore self-contained and non-circular.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no information on free parameters, axioms, or invented entities; all such elements are unknown.

pith-pipeline@v0.9.0 · 5701 in / 1049 out tokens · 40416 ms · 2026-05-24T11:41:29.995956+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 36 canonical work pages

  1. [1]

    Extrinsic calibration of a camera and laser range finder (improves camera calibration),

    Q. Zhang and R. Pless, “Extrinsic calibration of a camera and laser range finder (improves camera calibration),” in 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566), vol. 3, 2004, pp. 2301–2306 vol.3

    work page 2004

  2. [2]

    Automatic calibration of a range sensor and camera system,

    H. Alismail, D. Baker, and B. Browning, “Automatic calibration of a range sensor and camera system,” 10 2012

    work page 2012

  3. [3]

    Self calibration of multiple lidars and cameras on autonomous vehicles,

    M. Pereira, D. Silva, V . Santos, and P. Dias, “Self calibration of multiple lidars and cameras on autonomous vehicles,” Robotics and Autonomous Systems, vol. 83, 05 2016

    work page 2016

  4. [4]

    Automatic camera and range sensor calibration using a single shot,

    A. Geiger, F. Moosmann, O. Car, and B. Schuster, “Automatic camera and range sensor calibration using a single shot,” in 2012 IEEE Interna- tional Conference on Robotics and Automation , 2012, pp. 3936–3943. 11

    work page 2012

  5. [5]

    A novel method for the extrinsic calibration of a 2d laser rangefinder and a camera,

    W. Dong and V . Isler, “A novel method for the extrinsic calibration of a 2d laser rangefinder and a camera,” IEEE Sensors Journal , vol. 18, no. 10, p. 4200–4211, May 2018. [Online]. Available: http://dx.doi.org/10.1109/JSEN.2018.2819082

    work page doi:10.1109/jsen.2018.2819082 2018

  6. [6]

    Montoya, O

    O. Montoya, O. Icasio, and J. Salas, COUPLED: Calibration of a LiDAR and Camera Rig Using Automatic Plane Detection , 06 2020, pp. 209– 218

    work page 2020

  7. [7]

    Are we ready for autonomous driving? the KITTI vision benchmark suite,

    A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the KITTI vision benchmark suite,” in Conference on Computer Vision and Pattern Recognition (CVPR) , 2012

    work page 2012

  8. [8]

    Fast extrinsic calibration of a laser rangefinder to a camera,

    R. Unnikrishnan and M. Hebert, “Fast extrinsic calibration of a laser rangefinder to a camera,” 01 2005

    work page 2005

  9. [9]

    Automatic camera and range sensor calibration using a single shot,

    A. Geiger, F. Moosmann, O. Car, and B. Schuster, “Automatic camera and range sensor calibration using a single shot,” in 2012 IEEE Interna- tional Conference on Robotics and Automation , 2012, pp. 3936–3943

    work page 2012

  10. [10]

    Extrinsic calibration between a multi-layer lidar and a camera,

    S. A. Rodriguez F., V . Fremont, and P. Bonnifait, “Extrinsic calibration between a multi-layer lidar and a camera,” in 2008 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Sys- tems, 2008, pp. 214–219

    work page 2008

  11. [11]

    3d lidar-camera extrinsic calibration using an arbitrary trihedron,

    X. Gong, Y . Lin, and J. Liu, “3d lidar-camera extrinsic calibration using an arbitrary trihedron,” Sensors (Basel, Switzerland), vol. 13, pp. 1902– 18, 02 2013

    work page 1902

  12. [12]

    Indirect correspondence-based robust extrinsic calibration of lidar and camera,

    S. Sim, J. Sock, and K. Kwak, “Indirect correspondence-based robust extrinsic calibration of lidar and camera,” Sensors, vol. 16, p. 933, 06 2016

    work page 2016

  13. [13]

    Accurate calibration of lidar-camera systems using ordinary boxes,

    Z. Pusztai and L. Hajder, “Accurate calibration of lidar-camera systems using ordinary boxes,” in 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) , 2017, pp. 394–402

    work page 2017

  14. [14]

    Sensor and sensor fusion technology in autonomous vehicles: A review,

    D. J. Yeong, G. Velasco-Hernandez, J. Barry, and J. Walsh, “Sensor and sensor fusion technology in autonomous vehicles: A review,” Sensors, vol. 21, no. 6, 2021. [Online]. Available: https://www.mdpi.com/1424-8220/21/6/2140

    work page 2021

  15. [15]

    Autonomy for mars rovers: Past, present, and future,

    M. Bajracharya, M. W. Maimone, and D. Helmick, “Autonomy for mars rovers: Past, present, and future,” Computer, vol. 41, no. 12, pp. 44–50, 2008

    work page 2008

  16. [16]

    Online camera-lidar calibration with sensor semantic information,

    Y . Zhu, C. Li, and Y . Zhang, “Online camera-lidar calibration with sensor semantic information,” in 2020 IEEE International Conference on Robotics and Automation (ICRA) , 2020, pp. 4970–4976

    work page 2020

  17. [17]

    Crlf: Automatic calibration and refinement based on line feature for lidar and camera in road scenes,

    T. Ma, Z. Liu, G. Yan, and Y . Li, “Crlf: Automatic calibration and refinement based on line feature for lidar and camera in road scenes,” 2021

    work page 2021

  18. [18]

    Regnet: Multimodal sensor registration using deep neural networks,

    N. Schneider, F. Piewak, C. Stiller, and U. Franke, “Regnet: Multimodal sensor registration using deep neural networks,” 07 2017

    work page 2017

  19. [19]

    Calibnet: Geo- metrically supervised extrinsic calibration using 3d spatial transformer networks,

    G. Iyer, R. Ram, K. Jatavallabhula, and M. Krishna, “Calibnet: Geo- metrically supervised extrinsic calibration using 3d spatial transformer networks,” 10 2018, pp. 1110–1117

    work page 2018

  20. [20]

    Automatic targetless extrinsic calibration of a 3D Lidar and camera by maximizing mutual information,

    G. Pandey, J. R. McBride, S. Savarese, and R. M. Eustice, “Automatic targetless extrinsic calibration of a 3D Lidar and camera by maximizing mutual information,” in AAAI, 2012. [Online]. Available: http://www.aaai.org/ocs/index.php/AAAI/AAAI12/paper/view/5029

    work page 2012

  21. [21]

    Automatic online calibration of cameras and lasers

    J. Levinson and S. Thrun, “Automatic online calibration of cameras and lasers.” in Robotics: Science and Systems , vol. 2, 2013, p. 7

    work page 2013

  22. [22]

    Target-less camera-lidar extrinsic calibration using a bagged dependence estimator,

    K. Irie, M. Sugiyama, and M. Tomono, “Target-less camera-lidar extrinsic calibration using a bagged dependence estimator,” in 2016 IEEE International Conference on Automation Science and Engineering (CASE), 2016, pp. 1340–1347

    work page 2016

  23. [23]

    Automatic calibration of multi- modal sensor systems using a gradient orientation measure,

    Z. Taylor, J. Nieto, and D. Johnson, “Automatic calibration of multi- modal sensor systems using a gradient orientation measure,” in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems , 2013, pp. 1293–1300

    work page 2013

  24. [24]

    Autocalibration of li- dar and optical cameras via edge alignment,

    J. Castorena, U. S. Kamilov, and P. T. Boufounos, “Autocalibration of li- dar and optical cameras via edge alignment,” in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2016, pp. 2862–2866

    work page 2016

  25. [25]

    Image analysis and mathematical morphol-ogy,

    J. Serra, “Image analysis and mathematical morphol-ogy,” 1982

    work page 1982

  26. [26]

    High-resolution lidar-based depth mapping using bilateral filter,

    C. Premebida, L. Garrote, A. Asvadi, A. P. Ribeiro, and U. Nunes, “High-resolution lidar-based depth mapping using bilateral filter,” in 2016 IEEE 19th international conference on intelligent transportation systems (ITSC). IEEE, 2016, pp. 2469–2474

    work page 2016

  27. [27]

    Communications of the ACM 24, 381–395

    M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM , vol. 24, no. 6, p. 381–395, Jun. 1981. [Online]. Available: https://doi.org/10.1145/358669.358692

    work page doi:10.1145/358669.358692 1981

  28. [28]

    A noise-removal approach for lidar intensity images using anisotropic diffusion filtering to preserve object shape characteristics,

    R. Nobrega, J. Quintanilha, and C. O’Hara, “A noise-removal approach for lidar intensity images using anisotropic diffusion filtering to preserve object shape characteristics,” American Society for Photogrammetry and Remote Sensing - ASPRS Annual Conference 2007: Identifying Geospatial Solutions, vol. 2, pp. 471–481, 01 2007

    work page 2007

  29. [29]

    Computation at the edge of chaos: Phase transitions and emergent computation,

    L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D: Nonlinear Phenomena , vol. 60, no. 1, pp. 259–268, 1992. [Online]. Available: https: //www.sciencedirect.com/science/article/pii/016727899290242F

    work page arXiv 1992

  30. [30]

    Fast gradient-based algorithms for con- strained total variation image denoising and deblurring problems,

    A. Beck and M. Teboulle, “Fast gradient-based algorithms for con- strained total variation image denoising and deblurring problems,” IEEE Transactions on Image Processing, vol. 18, no. 11, pp. 2419–2434, 2009

    work page 2009

  31. [31]

    The split bregman method for l1-regularized problems,

    T. Goldstein and S. Osher, “The split bregman method for l1-regularized problems,” SIAM J. Imaging Sciences , vol. 2, pp. 323–343, 01 2009

    work page 2009

  32. [32]

    Pyunlocbox: Optimization by proximal splitting,

    E. L. Laboratory, “Pyunlocbox: Optimization by proximal splitting,” https://pyunlocbox.readthedocs.io/en/stable/index.html#

    work page

  33. [33]

    A computational approach to edge detection,

    J. Canny, “A computational approach to edge detection,” IEEE Transac- tions on Pattern Analysis and Machine Intelligence , vol. PAMI-8, no. 6, pp. 679–698, 1986

    work page 1986

  34. [34]

    Two-point step size gradient methods,

    J. Barzilai and J. M. Borwein, “Two-point step size gradient methods,” IMA Journal of Numerical Analysis , vol. 8, no. 1, pp. 141–148, 01

    work page

  35. [35]

    IMA Journal of Numerical Analysis8(1), 141–148 (1988)

    [Online]. Available: https://doi.org/10.1093/imanum/8.1.141

    work page doi:10.1093/imanum/8.1.141

  36. [36]

    Vision meets robotics: The KITTI dataset,

    A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, “Vision meets robotics: The KITTI dataset,” International Journal of Robotics Research (IJRR) , 2013

    work page 2013