REVIEW 4 major objections 8 minor 29 references
Real Time Semantic Segmentation of High Resolution Automotive LiDAR Scans
T0 review · 4 major / 8 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read This paper establishes that a range-view CNN with surface-normal input channels can segment a full 128-layer automotive LiDAR scan in as little as 8 ms while matching or beating slower baselines in accuracy.
desk verdict A useful 128-layer LiDAR dataset wrapped in an oversold method paper; the normals-feature claim does not survive the noise floor, but the dataset deserves referee time. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a range-view CNN on a staggered spherical image: each LiDAR point is projected to a pixel via a pinhole-style matrix $K$ with angular spacings $\Delta\phi, \Delta\theta$, giving channels for $x$, $y$, $z$, range, and reflectivity. Surface normals are computed by taking horizontal and vertical neighbor differences in the spherical image and forming the normalized cross product $\frac{\overrightarrow{P_cP_b} \times \overrightarrow{P_cP_a}}{\lVert \overrightarrow{P_cP_b} \times \overrightarrow{P_cP_a} \rVert_2}$. These normal channels, along with $xyz$, are injected at multiple backbone scales, then multi-scale features are combined with additive self-attention and upsampled by deconvolution to full resolution. The normal channel is what carries the geometric orientation information that helps distinguish buildings, curbs, and objects from ground.
What would settle it
Train the six reported models on SemanticTHAB twice, once with the true normal channels and once with the same channels replaced by random noise matched in scale; if the mIoU gap between those conditions is not close to the reported +4.24%, the normal-feature benefit is not geometry-specific. Independently, hand-check a few dynamic-object frames from sequence 0006 against GPS/IMU ground truth to see whether ego-motion drift has smeared labels near moving vehicles and pedestrians.
Extended reading notes
Core claim
The central claim is that the bottleneck for LiDAR semantic segmentation is no longer the sensor or the runtime but the input representation: with a staggered spherical projection and surface normals as input channels, a compact CNN can segment a full 128-layer scan in single-digit milliseconds at accuracy close to much heavier 3D models. The authors show that surface normals, computed from neighboring pixels in the spherical image, improve mIoU by 4.24% over the same network without them, and that the combination of multiscale injection of xyz and normal features, additive self-attention, and pretraining on a related public dataset yields 54.66 mIoU on their validation sequence. They report that all six backbone variants outperform CENet and FIDNet in both speed and mIoU on SemanticTHAB, with runtimes of 8–31 ms versus 63–66 ms.
Load-bearing premise
The whole pipeline—labels and normal channels alike—assumes the ego-motion estimates used to align scans are unbiased, so that neither the ground-truth labels nor the surface-normal input features are systematically smeared at object boundaries; if that alignment drifts, both dataset quality and the measured benefit of normals suffer.
Editorial extensions
If this is right
- A full 128-layer scan can be segmented in 8–31 ms on a single RTX 3090, leaving ample budget for the rest of an autonomous-driving perception stack.
- Surface normals from spherical projection are a cheap, effective feature: they add no learned parameters and improve mIoU by about 4 points.
- Pretraining on an older 64-layer dataset transfers to a new 128-layer sensor, suggesting models do not need to be retrained from scratch when sensor resolution increases.
- The 53–56 mIoU range on a harder urban 20-class task is achieved with generic ResNet and ShuffleNet backbones, so the approach is easy to reproduce and deploy.
- The released dataset and code let other groups compare methods on high-resolution LiDAR rather than only on low-resolution benchmarks.
Reading between the lines
- If the normal-channel gain holds across more scenes, a testable extension is to feed normals at even higher resolution or to derive curvature features from the same difference vectors, which could help with small classes such as poles and traffic indicators.
- The 8 ms ResNet18 result suggests that frame rates above 100 FPS are within reach on embedded GPUs, making the method usable not only for perception but for labeling assistance or online mapping.
- The dataset's 4,750 scans are small relative to established benchmarks, so cross-validation results may underestimate variance; a larger version with more cyclists and motorcyclists would be needed to confirm the ranking of backbones.
- Because the method relies on a clean spherical projection, its accuracy probably depends on the sensor's uniform azimuth sampling; transferring the architecture to non-uniform or mechanically scanning LiDAR might require re-projection or angular padding.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces SemanticTHAB, a new semantic segmentation dataset recorded with an automotive Ouster OS2-128 (Rev7) 128-layer LiDAR in urban traffic scenes, comprising 4,750 annotated scans across nine sequences. The authors also propose a range-view CNN segmentation method that concatenates spherical-image channels (reflectivity, range, x/y/z, and surface normals), incorporates multiscale 3D features and attention, and uses SemanticKITTI pretraining. They report per-class IoU and mIoU on one test sequence (0006), a leave-one-out cross-validation study over all sequences, an ablation isolating the contribution of normals, multiscale features, attention, and pretraining, and a ROS2 deployment on a research vehicle. The main claims are that surface normals are strong input features and that the method bridges the gap between state-of-the-art accuracy and real-time automotive deployment.
Significance. If the central claims are supported, the paper would make a useful practical contribution: a public high-resolution LiDAR dataset with SemanticKITTI-compatible labels, a simple and fast range-view architecture, and a public ROS2 implementation. The authors should be credited for releasing dataset and code, for using leave-one-out cross-validation rather than a single split, and for designing an ablation that separates the contributions of normals, multiscale features, attention, and pretraining. However, the evidence for the headline claim about surface normals is currently fragile: the isolated gains are small, are measured on a single sequence without variance estimates, and sit within the large sequence-to-sequence variation shown in the cross-validation plots. The baseline comparisons also lack stated training protocols, and the runtime comparisons mix GPUs and resolutions. The significance of the paper therefore depends on additional statistical support and clearer experimental reporting.
major comments (4)
- [Section IV-C, Table V, Figure 7] The central claim that surface normals are 'strong input features' is not supported by the reported evidence. The isolated effect of normals is +1.71 mIoU (47.86 to 49.57) without pretraining and +1.01 (53.65 to 54.66) with pretraining, measured on a single test sequence (0006) with no standard deviations, multiple seeds, or per-fold results. The cross-validation results in Figure 7 show that mIoU varies by roughly 10–15 points across the nine test sequences, so a 1–2 point difference on one sequence is well within the noise floor. In addition, the 'Increase' column in Table V is cumulative relative to the 47.55 baseline rather than incremental: the +4.24% for the normals row includes the contribution of the multiscale features, so the text's statement that 'the use of surface normals increases the performance to 49.57 (+4.24%)' overstates the isolated effect. Please report leave-one-out means and standard deviations for the ablation configurations, ideally with multiple seeds, and state the incremental gain attributable to normals.
- [Section IV-B, Table IV] The baseline comparison is fragile and the baseline training protocol is missing. The paper does not state whether CENet and FIDNet were trained on SemanticTHAB, what resolution, loss, pretraining, or number of epochs were used, or whether the reported runtimes were measured under the same conditions. Moreover, the claim that 'Our models outperform both CENet and FIDNet' is not true for all variants in Table IV: ResNet18 (53.57) is essentially tied with FIDNet (53.58), and ShuffleNet1.0 (53.52) is below it. Please specify the exact training and evaluation protocol for the baselines and either report all variants' results against the baselines with variance information or qualify the claim to the variants that do outperform them.
- [Section IV-B, Table II] The SemanticKITTI comparison in Table II mixes hardware and resolution. 'Our @64×512' reports FPS measured on an RTX 2080 Ti, while Table IV and Figure 6 use an RTX 3090 at a much higher resolution (128×2048). The text acknowledges that published FPS figures are lower at high resolution but provides no measurement of the baselines under the same high-resolution, same-hardware conditions. Because the real-time claim is central to the paper, please report runtime and mIoU for all compared methods under identical hardware, resolution, and input-channel settings, and clearly separate the SemanticKITTI and SemanticTHAB evaluations.
- [Section II and Section III-B] The annotation and normal-estimation pipelines have unstated failure modes. The ego-motion estimate from KISS-ICP in the Ouster SDK is used to align scans for labeling, but no validation of this estimate is provided (for example, comparison against GNSS/INS or loop-closure quality). The surface-normal computation in Eqs. (3)–(4) uses neighboring pixels with no stated handling of invalid measurements, range discontinuities, or image-border pixels. If ego-motion is biased or normals are noisy at object boundaries, both the labels and the normal input channels could be systematically wrong, which would affect the dataset quality and the measured benefit of normals. Please state how invalid and out-of-bounds pixels are masked and how the ego-motion estimates used for annotation were verified.
minor comments (8)
- [Abstract] Typo: 'Additionaly' should be 'Additionally'; the phrase 'bridging the gap between cutting-edge research and practical automotive applications' is stronger than the evidence in Section IV supports.
- [Section III-B] Typo: 'homogenity' should be 'homogeneity'.
- [Figure 5 caption] Typo: 'violett' should be 'violet'.
- [Section VI] Typo: 'approch' should be 'approach'.
- [Section IV-A] Grammar: 'Our models was trained' should be 'Our models were trained'.
- [Section II] The mapping from SemanticKITTI's 28 classes to the 20 SemanticTHAB classes, including the merging of traffic signs and lane markings into 'traffic-indicator', is not described; this mapping is needed to understand the SemanticKITTI pretraining and fine-tuning experiments.
- [Section V] The claim that an RTX 3090 offers 'comparable inference performance' to an Nvidia DRIVE Orin is not supported by any measurement or reference; please either provide evidence or rephrase as an assumption.
- [Table I] Minor grammar: 'Dataset that share' should be 'Datasets that share'; the entry for SemanticSTF lists '✗' under FPS without explanation.
Circularity Check
No significant circularity: the paper's claims rest on held-out empirical evaluations and a fully specified normal computation, not on a derivation that reduces to its inputs.
full rationale
The paper's central contributions are a new LiDAR dataset and an empirical semantic-segmentation method that uses surface normals as additional input features. The performance claims in Table IV and the ablation in Table V are measured mIoU results on held-out sequence 0006, not quantities derived from fitted parameters or from the method's own definitions. The surface-normal features are computed by explicit cross-product formulas (Eqs. 3-4) from the spherical image; although the paper cites prior work by overlapping authors ([20], [21]) for projection and normal estimation, those citations are not load-bearing evidence for the new result because the computation is fully specified in the paper and is a standard preprocessing step. No uniqueness theorem, ansatz, or fitted parameter is imported from a self-citation to force the paper's outcome. The pretraining on SemanticKITTI followed by fine-tuning on SemanticTHAB is conventional transfer learning, and the comparisons to CENet and FIDNet are external baselines. Concerns about the small 1.7-point ablation gain, lack of reported variance, and cross-validation swings are legitimate threats to the strength of the empirical evidence, but they are statistical-validity concerns, not circularity. No claim reduces by construction to its inputs, and no fitted value is renamed as a prediction. Therefore the circularity score is 0.
Assumptions & free parameters
free parameters (4)
- Learning rate =
0.001
- Training schedule =
50 epochs on SemanticKITTI, then 30 epochs fine-tuning
- Loss weights for cross-entropy and Tversky =
not reported
- Spherical image resolution =
128x2048; 64x512 for the SemanticKITTI benchmark
assumptions (5)
- domain assumption Spherical projection maps neighboring pixels to adjacent LiDAR rays
- domain assumption Surface normals estimated from finite differences in the spherical image are meaningful for all valid pixels
- domain assumption SemanticKITTI label definitions transfer to the Ouster sensor, with traffic signs and lane markings merged into traffic-indicator
- domain assumption Pretraining on SemanticKITTI (Velodyne HDL-64) transfers to the Ouster OS2-128
- domain assumption RTX 3090 inference time approximates automotive embedded hardware such as Nvidia DRIVE Orin
Cite this review
Pith. "Pith review of Real Time Semantic Segmentation of High Resolution Automotive LiDAR Scans." pith.science (2026). https://pith.science/paper/NOFSXQNA
@misc{pith2026250421602,
author = {Pith},
title = {Pith review of: Real Time Semantic Segmentation of High Resolution Automotive LiDAR Scans},
year = {2026},
howpublished = {\url{https://pith.science/paper/NOFSXQNA}},
note = {Machine review of arXiv:2504.21602}
}
read the original abstract
In recent studies, numerous previous works emphasize the importance of semantic segmentation of LiDAR data as a critical component to the development of driver-assistance systems and autonomous vehicles. However, many state-of-the-art methods are tested on outdated, lower-resolution LiDAR sensors and struggle with real-time constraints. This study introduces a novel semantic segmentation framework tailored for modern high-resolution LiDAR sensors that addresses both accuracy and real-time processing demands. We propose a novel LiDAR dataset collected by a cutting-edge automotive 128 layer LiDAR in urban traffic scenes. Furthermore, we propose a semantic segmentation method utilizing surface normals as strong input features. Our approach is bridging the gap between cutting-edge research and practical automotive applications. Additionaly, we provide a Robot Operating System (ROS2) implementation that we operate on our research vehicle. Our dataset and code are publicly available: https://github.com/kav-institute/SemanticLiDAR.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
SemanticKITTI: A Dataset for Semantic Scene Under- standing of LiDAR Sequences,
J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, and J. Gall, “SemanticKITTI: A Dataset for Semantic Scene Under- standing of LiDAR Sequences,” inProc. of the IEEE/CVF International Conf. on Computer Vision (ICCV), 2019. 1, 2
work page 2019
-
[2]
Semanticposs: A point cloud dataset with large quantity of dynamic instances,
Y . Pan, B. Gao, J. Mei, S. Geng, C. Li, and H. Zhao, “Semanticposs: A point cloud dataset with large quantity of dynamic instances,” 2020. 1
work page 2020
-
[3]
Lidarnet: A boundary-aware domain adapta- tion model for lidar point cloud semantic,
P. Jiang and S. Saripalli, “Lidarnet: A boundary-aware domain adapta- tion model for lidar point cloud semantic,” 2020. 1
work page 2020
-
[4]
Panoptic nuscenes: A large-scale benchmark for lidar panoptic segmentation and tracking,
W. K. Fong, R. Mohan, J. V . Hurtado, L. Zhou, H. Caesar, O. Beijbom, and A. Valada, “Panoptic nuscenes: A large-scale benchmark for lidar panoptic segmentation and tracking,”arXiv preprint arXiv:2109.03805,
-
[5]
Scalability in perception for autonomous driving: Waymo open dataset,
P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V . Patnaik, P. Tsui, J. Guo, Y . Zhou, Y . Chai, B. Caine, V . Vasudevan, W. Han, J. Ngiam, H. Zhao, A. Timofeev, S. Ettinger, M. Krivokon, A. Gao, A. Joshi, Y . Zhang, J. Shlens, Z. Chen, and D. Anguelov, “Scalability in perception for autonomous driving: Waymo open dataset,” inProceedings of the IEEE/CV...
work page 2020
-
[6]
3D Semantic Segmentation in the Wild: Learning Generalized Models for Adverse-Condition Point Clouds
A. Xiao, J. Huang, W. Xuan, R. Ren, K. Liu, D. Guan, A. E. Saddik, S. Lu, and E. Xing, “3d semantic segmentation in the wild: Learning generalized models for adverse-condition point clouds,”arXiv preprint arXiv:2304.00690, 2023. 1
work page Pith review arXiv 2023
-
[7]
Frnet: Frustum-range networks for scalable lidar segmentation,
X. Xu, L. Kong, H. Shuai, and Q. Liu, “Frnet: Frustum-range networks for scalable lidar segmentation,”ArXiv, vol. abs/2312.04484, 2023. 1, 2
arXiv 2023
-
[8]
Fidnet: Lidar point cloud seman- tic segmentation with fully interpolation decoding,
Y . Zhao, L. Bai, and X. Huang, “Fidnet: Lidar point cloud seman- tic segmentation with fully interpolation decoding,”2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4453–4458, 2021. 1, 2, 5
work page 2021
Show all 29 references
-
[9]
Cenet: Toward concise and efficient lidar semantic segmentation for autonomous driving,
H.-X. Cheng, X.-F. Han, and G.-Q. Xiao, “Cenet: Toward concise and efficient lidar semantic segmentation for autonomous driving,” in 2022 IEEE International Conference on Multimedia and Expo (ICME), pp. 01–06, IEEE, 2022. 1, 2, 5
2022
-
[10]
Rethinking range view representation for lidar segmentation,
L. Kong, Y . Liu, R. Chen, Y . Ma, X. Zhu, Y . Li, Y . Hou, Y . Qiao, and Z. Liu, “Rethinking range view representation for lidar segmentation,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 228–240, 2023. 2
2023
-
[11]
Rangevit: Towards vision transformers for 3d semantic segmentation in autonomous driving,
A. Ando, S. Gidaris, A. Bursuc, G. Puy, A. Boulch, and R. Marlet, “Rangevit: Towards vision transformers for 3d semantic segmentation in autonomous driving,” inCVPR, 2023. 2
2023
-
[12]
Spherical transformer for lidar-based 3d recognition,
X. Lai, Y . Chen, F. Lu, J. Liu, and J. Jia, “Spherical transformer for lidar-based 3d recognition,” inCVPR, 2023. 2
2023
-
[13]
Cylindrical and asymmetrical 3d convolution networks for lidar segmentation,
X. Zhu, H. Zhou, T. Wang, F. Hong, Y . Ma, W. Li, H. Li, and D. Lin, “Cylindrical and asymmetrical 3d convolution networks for lidar segmentation,”arXiv preprint arXiv:2011.10033, 2020. 2
2011 arXiv
-
[14]
Search- ing efficient 3d architectures with sparse point-voxel convolution,
H. Tang, Z. Liu, S. Zhao, Y . Lin, J. Lin, H. Wang, and S. Han, “Search- ing efficient 3d architectures with sparse point-voxel convolution,” in European Conference on Computer Vision, 2020. 2
2020
-
[15]
Point transformer v3: Simpler, faster, stronger,
X. Wu, L. Jiang, P.-S. Wang, Z. Liu, X. Liu, Y . Qiao, W. Ouyang, T. He, and H. Zhao, “Point transformer v3: Simpler, faster, stronger,” inCVPR,
-
[16]
Using a waffle iron for automotive point cloud semantic segmentation,
G. Puy, A. Boulch, and R. Marlet, “Using a waffle iron for automotive point cloud semantic segmentation,” inIEEE/CVF International Con- ference on Computer Vision, pp. 3379–3389, 2023. 2
2023
-
[17]
Rpvnet: A deep and efficient range-point-voxel fusion network for lidar point cloud segmentation,
J. Xu, R. Zhang, J. Dou, Y . Zhu, J. Sun, and S. Pu, “Rpvnet: A deep and efficient range-point-voxel fusion network for lidar point cloud segmentation,” in2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 16004–16013, 2021. 2
2021
-
[18]
Point-to-voxel knowledge distillation for lidar semantic segmentation,
Y . Hou, X. Zhu, Y . Ma, C. C. Loy, and Y . Li, “Point-to-voxel knowledge distillation for lidar semantic segmentation,” inIEEE Conference on Computer Vision and Pattern Recognition, pp. 8479–8488, 2022. 2
2022
-
[19]
KISS-ICP: In Defense of Point-to-Point ICP – Simple, Accurate, and Robust Registration If Done the Right Way,
I. Vizzo, T. Guadagnino, B. Mersch, L. Wiesmann, J. Behley, and C. Stachniss, “KISS-ICP: In Defense of Point-to-Point ICP – Simple, Accurate, and Robust Registration If Done the Right Way,”IEEE Robotics and Automation Letters (RA-L), vol. 8, no. 2, pp. 1029–1036,
-
[20]
Sensor equivariance by lidar projection images,
H. Reichert, M. Hetzel, S. Schreck, K. Doll, and B. Sick, “Sensor equivariance by lidar projection images,” in2023 IEEE Intelligent Vehicles Symposium (IV), pp. 1–6, 2023. 3
2023
-
[21]
Height change feature based free space detection,
S. Schreck, H. Reichert, M. Hetzel, K. Doll, and B. Sick, “Height change feature based free space detection,” in2023 11th International Confer- ence on Control, Mechatronics and Automation (ICCMA), pp. 171–176,
-
[22]
Deep residual learning for image recognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,”2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778, 2015. 4
2016
-
[23]
Shufflenet: An extremely efficient convolutional neural network for mobile devices,
X. Zhang, X. Zhou, M. Lin, and J. Sun, “Shufflenet: An extremely efficient convolutional neural network for mobile devices,” in2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6848–6856, 2018. 4
2018
-
[24]
Torchvision: Pytorch’s computer vision library
T. maintainers and contributors, “Torchvision: Pytorch’s computer vision library.” https://github.com/pytorch/vision, 2016. 4
2016
-
[25]
3d-mininet: Learning a 2d representation from point clouds for fast and efficient 3d lidar semantic segmentation,
I. Alonso, L. Riazuelo, L. Montesano, and A. C. Murillo, “3d-mininet: Learning a 2d representation from point clouds for fast and efficient 3d lidar semantic segmentation,”IEEE Robotics and Automation Letters, vol. 5, pp. 5432–5439, 2020. 4
2020
-
[26]
Tversky as a loss function for highly unbalanced image segmentation using 3d fully convolutional deep networks,
S. R. Hashemi, S. S. M. Salehi, D. Erdo ˘gmu¸ s, S. P. Prabhu, S. Warfield, and A. Gholipour, “Tversky as a loss function for highly unbalanced image segmentation using 3d fully convolutional deep networks,”ArXiv, vol. abs/1803.11078, 2018. 4
2018 arXiv
-
[27]
Adam: A method for stochastic optimization,
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” CoRR, vol. abs/1412.6980, 2014. 4
2014 arXiv
-
[28]
Robotic operating system
Stanford Artificial Intelligence Laboratory et al., “Robotic operating system.” 5
-
[29]
Clustering is back: Reaching state-of-the-art LiDAR instance segmentation without training,
C. Sautier, G. Puy, A. Boulch, R. Marlet, and V . Lepetit, “Clustering is back: Reaching state-of-the-art LiDAR instance segmentation without training,”arxiv, 2025. 5
2025
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.