Pith. sign in

REVIEW 3 major objections 5 minor 17 references

Automatic Labelling & Semantic Segmentation with 4D Radar Tensors

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that 4D radar tensors alone can be semantically segmented into scenario objects, vehicles, and vulnerable road users, using automatically generated labels from LiDAR and camera, reaching over 65% of LiDAR-based detection…

desk verdict Useful auto-labelling contribution for 4D radar, but the segmentation evaluation is circular and the headline numbers mix configurations. read the letter →

arxiv 2501.11351 v1 pith:AKOF7VEG submitted 2025-01-20 cs.CV eess.SP

Botao Sun , Ignacio Roldan , Francesco Fioranelli This is my paper · ORCID
classification cs.CVeess.SP
keywords 4DautomotiveradarsemanticsegmentationautomaticlabellingLiDAR-camerafusionRAEDtensorRaDelftdatasetvoxelclassificationpointcloud
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles two gaps at once: automotive radar lacks labelled data, and 4D radar tensors are rarely used for semantic segmentation. It proposes an automatic labelling pipeline that combines a LiDAR-only 3D detector, camera semantic segmentation, and density-based clustering to create point-by-point labels for the RaDelft dataset. Those labels train a voxel-wise segmentation network that maps a range-azimuth-elevation-Doppler radar tensor to class probabilities per voxel. The central claim is that radar alone, with automatically generated labels, can recover over 65% of the LiDAR detection performance and nearly 50% of vehicle detection, while beating two literature-inspired variants on detection probability and Chamfer distance.

What carries the argument

The load-bearing mechanism is a two-stage pipeline. First, automatic labelling: a pretrained LiDAR detector (Part-A2) proposes bounding boxes, Patchwork++ removes ground points, the OneFormer camera model re-labels close points within 25 m, and DBSCAN enforces label consistency by majority voting; the labels are then voxelized into a 500 x 240 x 34 range-azimuth-elevation cube. Second, segmentation: the radar tensor is projected to the same cube, a 2D FPN with ResNet-18 backbone produces an occupancy latent space and a class latent space, broadcasting merges them into a 3D semantic latent space, and a five-layer 3D U-Net outputs per-voxel class probabilities. The paper's numbers stand or fall on this division of labour: the labelling must be trustworthy enough to serve as ground truth, and the voxelized radar power must preserve the class-discriminative information.

What would settle it

Manually label the two unseen test scenes (or reuse the 50-frame manual set) and recompute detection probability and Chamfer distance against those manual labels; if the numbers drop materially below 65.1%, 47.9%, and 1.77 m, the reported segmentation performance is an artefact of auto-label error propagation rather than a radar capability.

Watch

Extended reading notes

Core claim

The central discovery is that a 4D radar tensor, converted to a range-azimuth-elevation power cube, contains enough information for voxel-level semantic segmentation when paired with labels auto-generated from LiDAR and camera. The proposed network uses a two-branch 2D backbone to build occupancy and class latent spaces, combines them by broadcasting, and refines the result with a 3D U-Net. On the RaDelft test scenes, the best configuration (with pedestrians and bicycles merged into a vulnerable-road-user class) achieves 65.1% detection probability for all points, 47.9% for vehicles, and 1.77 m Chamfer distance, outperforming the compared variants by 13.2 percentage points in vehicle detection and by 0.54 m in Chamfer distance.

Load-bearing premise

The automatically generated labels are accurate enough to serve as ground truth for training and evaluating the segmentation network, despite F1-scores of only 0.69 for pedestrians and 0.83 for bicycles on 50 manually labelled frames.

Editorial extensions

If this is right

  • Semantic segmentation of 4D radar is feasible without manual annotation, since the automatic labelling pipeline replaces human labelling for the RaDelft dataset.
  • Merging pedestrians and bicycles into a vulnerable-road-user class improves detection probability for all points and reduces Chamfer distance, so class granularity trades against segmentation accuracy.
  • Radar's Doppler information helps segment dynamic targets, with moving vehicles among the best-segmented objects in the qualitative results.
  • The publicly released dataset with generated labels enables future radar-only perception work that does not need LiDAR at inference time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the evaluation metrics in Table II are computed against the same automatically generated labels used for training, the reported 65.1% detection probability is best read as an upper bound; an evaluation against manual labels on the same test frames would likely be lower, especially for pedestrians, whose auto labels have only 0.69 F1-score.
  • The labelling pipeline could transfer to other radar datasets if LiDAR and camera extrinsics are available, but the 25 m camera calibration range and the KITTI-trained detector will limit label quality for small, distant, or closely packed objects.
  • Doppler is discarded when the RAED tensor is averaged into the RAE cube; keeping Doppler as an extra channel could plausibly improve VRU detection, since the paper's own qualitative result credits Doppler for dynamic vehicles.
  • The comparison against literature variants is indirect because those variants were adapted from detection networks; a direct comparison on a common labelled benchmark would clarify whether the gain comes from the two-branch architecture or from the training labels.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents two contributions on the RaDelft dataset: an automatic labelling pipeline that fuses LiDAR (pre-trained Part-A2 detections, Patchwork++ ground removal, OneFormer camera calibration, and DBSCAN smoothing) and voxelizes the result into a 500×240×34 grid; and a semantic segmentation network for 4D radar tensors, consisting of two 2D FPN branches (occupancy and class latent spaces) combined and fed to a 3D U-Net. The labelling is evaluated on 50 manually labelled frames (Table I: F1 = 0.99 for scenario objects, 0.88 for vehicles, 0.83 for bicycles, 0.69 for pedestrians). The segmentation is evaluated with detection probability, false alarm rate, and Chamfer distance against the auto-generated LiDAR labels, with the best configuration at Pd All = 65.1%, Pd Vehicles = 47.9%, and CDAll = 1.77 m, compared with two literature-adapted variants and an enhanced baseline.

Significance. If validated, the automatic labelling pipeline would be a practical asset for radar-centric datasets lacking labels, and the public release of the generated labels on 4TU.ResearchData is a genuinely useful contribution. The proposed network is also among the first attempts at direct voxel-level semantic segmentation from range-azimuth-elevation-Doppler radar tensors. However, the central quantitative claim about segmentation quality is undercut by a circular evaluation: the auto-generated labels serve as both training targets and test ground truth, and the labelling step's own pedestrian F1 of 0.69 shows substantial label noise. Hence the significance of the segmentation results cannot be assessed from the reported numbers alone.

major comments (3)
  1. [III.B / Table II and Fig. 4] The segmentation evaluation is circular: the 'LiDAR ground truth' in Fig. 4 is the output of the automatic labelling pipeline of Fig. 1, and these same auto-generated voxel labels are used as the training targets for the network (Section III.A). Table I shows the auto labels have only 0.69 F1 for pedestrians and 0.83 for bicycles against 50 manual frames, so label noise propagates directly into every metric in Table II. A network that learned the label-generation pipeline's systematic errors would score higher on these metrics without being semantically better. Since the 50 manually labelled frames are used only for labelling evaluation and not for scoring the segmentation, the claimed 'over 65% of LiDAR detection performance' is not established against independent ground truth. Please recompute Table II on the manually labelled frames (or on a held-out clean subset) and report both sets of numbers.
  2. [Abstract and Section IV] The headline numbers in the abstract and conclusion are not reproducible from Table II. The '13.2% improvement in vehicle detection probability' is the difference between Baseline (VRU) at 47.9% and Baseline at 34.8%, i.e., the effect of merging pedestrian and bicycle classes within the authors' own baseline, not a comparison against the literature variants. The '0.54 m reduction in Chamfer distance' is the difference between Variant 1 (2.31 m) and Baseline + Res (VRU) (1.77 m) in CDAll, which compares a single variant to a different proposed configuration. Also, the best Pd All (65.1%) comes from Baseline (VRU), while the best CDAll (1.77 m) comes from Baseline + Res (VRU); no single configuration attains all the abstract's claims. Please state exactly which configuration each reported number refers to and use a consistent comparison.
  3. [III.B.2] The phrase 'over 65% of the LiDAR detection performance' is undefined as stated. Pd All is the detection probability of radar voxels with respect to the LiDAR-derived labels; it is not a ratio to LiDAR's own detection performance. Please either define the quantity precisely or rephrase the claim.
minor comments (5)
  1. [Fig. 1 and Section II.A.3] The heading 'Tranformation and voxelization' and the figure label 'Tranformation' contain a typo: it should be 'Transformation'.
  2. [III.A] Training details such as learning rate, batch size, number of epochs, and the weights of the wCE and SDice losses are not reported; these are needed for reproducibility.
  3. [Table II] Since the test set comprises only two scenes and each configuration is run once, the absence of error bars or per-frame variance makes it difficult to judge whether the differences between configurations are meaningful; adding such information would strengthen the comparison.
  4. [II.A.2] The calibration distance threshold (25 m) and DBSCAN parameters (epsilon = 0.6, minPts = 100) are presented without sensitivity analysis; a brief discussion of how these choices affect the labelling quality would help.
  5. [III.B.2] The CDTargets metric is said to be high because static and dynamic targets are not distinguished; please clarify how 'scenario objects' relates to parked vehicles and whether the high CDTargets is partly an artifact of the label definition.

Circularity Check

1 steps flagged · score 6.0 of 10

Segmentation metrics are scored against the same auto-generated labels used as training targets; manual labels never enter the segmentation evaluation.

  1. self definitional [Abstract; Section II.B.2; Section III.B.2 (Fig. 4, Table II)]
    "The generated labels are then used as ground truth with the corresponding 4D radar data as inputs to a proposed semantic segmentation network... For a quantitative evaluation of the proposed automatic labelling, we randomly selected 50 frames from 7 different scenes for manual expert labelling... The 3D semantic segmentation outputs are transformed into Cartesian coordinates to compare with LiDAR ground truth visually."

    The 'LiDAR ground truth' used for the segmentation metrics in Fig. 4 and Table II is the output of the automatic labelling pipeline, not independent manual annotation. The same pipeline produces the training targets for the segmentation network. Manual labels are collected for only 50 frames and are used solely to evaluate the labelling process in Table I; they are never used to re-evaluate the segmentation network. Therefore Pd, Pfa, and Chamfer distance in Table II measure how well the network reproduces the auto-label generation algorithm, including its systematic errors (pedestrian F1=0.69, bicycle F1=0.83).

full rationale

The paper contains one substantive circular element: the semantic segmentation network is trained on automatically generated labels and evaluated against the same automatic labelling pipeline's output, while manual labels are confined to the labelling-quality assessment in Table I. Since the automatic labels have documented errors, especially for pedestrians (F1=0.69) and bicycles (F1=0.83), the reported detection probabilities and Chamfer distances in Table II are not independent measurements of semantic segmentation quality; they reflect agreement with the label-generation algorithm. The relative improvement claims (13.2% Pd Vehicles, 0.54 m CDAll) compare network variants on the same noisy labels, so they may be less affected, but the absolute 'over 65% of LiDAR detection performance' statement is not supported by an independent reference. The rest of the derivation chain is self-contained: the label-generation pipeline uses external pre-trained models (Part-A2, OneFormer, Patchwork++), the network architecture is described concretely, and the comparisons to literature variants are implemented as stated. The self-citations in the paper ([6], [7], [16]) are not load-bearing in a way that forces the central result; they provide dataset details, thesis details, and a comparison method. No equation-level circularity or imported uniqueness theorem was found. The score reflects the self-referential evaluation protocol at the core of the empirical claims.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claims rest on several hand-chosen thresholds and on the assumption that the auto-labelling pipeline produces reliable ground truth; the segmentation evaluation is therefore at least partially self-referential.

free parameters (5)
  • confidence threshold for Part-A2 detections = 0.5
    Set in Section II.A.1 to filter bounding boxes; high-confidence threshold trades precision vs recall of initial labels.
  • camera calibration distance threshold = 25 m
    Section II.A.2: points within 25 m are relabelled from camera semantic segmentation; closer points assumed more reliable.
  • DBSCAN epsilon = 0.6
    Section II.A.2: neighborhood radius for clustering; chosen to detect small clusters.
  • DBSCAN minPts = 100
    Section II.A.2: minimum points per cluster; chosen to differentiate closely located targets.
  • loss weights for wCE and SDice
    Section III.A mentions a combination of weighted cross-entropy and soft-dice loss but does not give the weights; these must be tuned.
assumptions (4)
  • domain assumption The pre-trained Part-A2 detector and OneFormer segmenter transfer to RaDelft data despite being trained on KITTI and generic images.
    Section II.A.1 and II.A.2 rely on these pre-trained models without fine-tuning on RaDelft.
  • domain assumption The auto-generated LiDAR labels are a valid ground truth for evaluating radar semantic segmentation.
    Section III-B uses these labels as ground truth; manual labels validate only the labelling step, not the segmentation.
  • domain assumption The RAE tensor formed by averaging power over Doppler and channel retains enough information for semantic segmentation.
    Section III-A transforms the RAED tensor to RAE by averaging power; this discards Doppler information.
  • domain assumption The class latent space of shape C x NR x NA can be broadcast across elevation without loss.
    Section III-A: the class branch has no elevation dimension; this assumes class does not vary with elevation for a given range-azimuth cell.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Automatic Labelling & Semantic Segmentation with 4D Radar Tensors." pith.science (2026). https://pith.science/paper/AKOF7VEG

@misc{pith2026250111351,
  author       = {Pith},
  title        = {Pith review of: Automatic Labelling & Semantic Segmentation with 4D Radar Tensors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AKOF7VEG}},
  note         = {Machine review of arXiv:2501.11351}
}
read the original abstract

In this paper, an automatic labelling process is presented for automotive datasets, leveraging on complementary information from LiDAR and camera. The generated labels are then used as ground truth with the corresponding 4D radar data as inputs to a proposed semantic segmentation network, to associate a class label to each spatial voxel. Promising results are shown by applying both approaches to the publicly shared RaDelft dataset, with the proposed network achieving over 65% of the LiDAR detection performance, improving 13.2% in vehicle detection probability, and reducing 0.54 m in terms of Chamfer distance, compared to variants inspired from the literature.

Figures

Figures reproduced from arXiv: 2501.11351 by the authors.

Figure 1
Figure 1. Block diagram of the proposed automatic labelling process using LiDAR point clouds (PCs) and RGB images [7]. First, preliminary labels on [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Automatic vs manual labelling results for a complex scene, with a reference camera image provided. The color bar distinguishes the 4 labelled classes. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Proposed radar semantic segmentation approach [7]. A radar tensor [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Generated radar PCs with class information and LiDAR ground truth are presented for a complex scene. The corresponding RGB image is provided [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 9 canonical work pages

  1. [1]

    Multi-sensor fusion in automated driving: A survey,

    Z. Wang, Y . Wu, and Q. Niu, “Multi-sensor fusion in automated driving: A survey,” Ieee Access, vol. 8, pp. 2847–2868, 2019

    work page 2019

  2. [2]

    Rethinking atrous convolution for semantic image segmentation,

    L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam, “Rethinking atrous convolution for semantic image segmentation,” arXiv preprint arXiv:1706.05587, 2017

    arXiv 2017

  3. [3]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020

    arXiv 2010

  4. [4]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space,

    C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” Advances in neural information processing systems , vol. 30, 2017

    2017

  5. [5]

    Point transformer,

    H. Zhao, L. Jiang, J. Jia, P. H. Torr, and V . Koltun, “Point transformer,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 16 259–16 268

    2021

  6. [6]

    A deep automotive radar detector using the radelft dataset,

    I. Roldan, A. Palffy, J. F. P. Kooij, D. M. Gavrila, F. Fioranelli, and A. Yarovoy, “A deep automotive radar detector using the radelft dataset,” IEEE Transactions on Radar Systems , vol. 2, pp. 1062–1075, 2024

    work page 2024

  7. [7]

    Autolabeling & semantic segmentation with 4d radar tensors,

    B. Sun, “Autolabeling & semantic segmentation with 4d radar tensors,” TU Delft MSc Thesis , 2024. [Online]. Available: https://repository. tudelft.nl/record/uuid:f01462b1-0446-481e-9333-9b8d3a488f14

    work page 2024

  8. [8]

    From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network,

    S. Shi, Z. Wang, J. Shi, X. Wang, and H. Li, “From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network,” IEEE transactions on pattern analysis and machine intelli- gence, vol. 43, no. 8, pp. 2647–2664, 2020

    work page 2020

Show all 17 references
  1. [9]

    Vision meets robotics: The kitti dataset,

    A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, “Vision meets robotics: The kitti dataset,” International Journal of Robotics Research (IJRR) , 2013

    2013

  2. [10]

    Patchwork++: Fast and robust ground segmentation solving partial under-segmentation using 3D point cloud,

    S. Lee, H. Lim, and H. Myung, “Patchwork++: Fast and robust ground segmentation solving partial under-segmentation using 3D point cloud,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. , 2022, pp. 13 276– 13 283

    2022

  3. [11]

    Oneformer: One transformer to rule universal image segmentation,

    J. Jain, J. Li, M. T. Chiu, A. Hassani, N. Orlov, and H. Shi, “Oneformer: One transformer to rule universal image segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2023, pp. 2989–2998

    2023

  4. [12]

    A joint extrinsic calibration tool for radar, camera and lidar,

    J. Domhof, J. F. Kooij, and D. M. Gavrila, “A joint extrinsic calibration tool for radar, camera and lidar,” IEEE Transactions on Intelligent Vehicles, vol. 6, no. 3, pp. 571–582, 2021

    2021

  5. [13]

    A density-based algorithm for discovering clusters in large spatial databases with noise,

    M. Ester, H.-P. Kriegel, J. Sander, X. Xu et al. , “A density-based algorithm for discovering clusters in large spatial databases with noise,” in kdd, vol. 96, no. 34, 1996, pp. 226–231

    1996

  6. [14]

    Feature pyramid networks for object detection,

    T.-Y . Lin, P. Doll´ar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 2117–2125

    2017

  7. [15]

    3d u-net: learning dense volumetric segmentation from sparse annotation,

    ¨O. C ¸ ic ¸ek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ron- neberger, “3d u-net: learning dense volumetric segmentation from sparse annotation,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece, Oc...

    2016

  8. [16]

    See further than cfar: a data-driven radar detector trained by lidar,

    I. Roldan, A. Palffy, J. F. Kooij, D. M. Gavrila, F. Fioranelli, and A. Yarovoy, “See further than cfar: a data-driven radar detector trained by lidar,” in 2024 IEEE Radar Conference (RadarConf24). IEEE, 2024, pp. 1–6

    2024

  9. [17]

    K-radar: 4d radar object detection for autonomous driving in various weather conditions,

    D.-H. Paek, S.-H. Kong, and K. T. Wijaya, “K-radar: 4d radar object detection for autonomous driving in various weather conditions,” Ad- vances in Neural Information Processing Systems , vol. 35, pp. 3819– 3829, 2022

    2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.