REVIEW 4 major objections 7 minor 21 references
SAFERad: A Framework to Enable Radar Data for Safety-Relevant Perception Tasks
T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Criticality-aware radar filtering rescues 76% of critical echoes
desk verdict Point-level criticality gating is a good idea; the evaluation is in-sample and needs independent ground truth before the numbers can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a per-point criticality evaluation built on a discrete drive tube: the planned path is represented by $N=15$ states $S_k$ with position, velocity, acceleration, steering angle, and yaw angle over a $3.0\,\mathrm{s}$ horizon. For each radar point the algorithm finds the perpendicular projection onto that tube, obtaining the travelled distance $d_{\mathrm{dist}}$, the lateral tube distance $d_{\mathrm{tube}}$, and the speed $v_{pt}$ of the nearest state; these feed three components—velocity criticality $\operatorname{crit}_{\mathrm{vel}}=v_{\mathrm{coll}}^2/v_{\max,\mathrm{domain}}^2$, tube criticality $\operatorname{crit}_{\mathrm{tube}}$ (1 inside the tube, third-degree polynomial decay over a $2.0\,\mathrm{m}$ insecurity zone), and distance criticality $\operatorname{crit}_{\mathrm{ttc}}$—whose product $\operatorname{crit}_p$ is bounded by 1 and vanishes if any component vanishes. Around points exceeding the criticality threshold, the posteriori method defines regions for the next $N_{\mathrm{posterior}}$ measurement cycles, removes the RCS filter threshold inside them, and grows the region radius from $0.2\,\mathrm{m}$ to $1.0\,\mathrm{m}$ over a lifetime of five cycles. Clustering then runs as usual DBSCAN with $\epsilon=1.0$ and $\mathrm{minpts}=4$, so the framework's guarantee rests on getting these weak points into the clustering step rather than changing the clusterer.
What would settle it
Record real safety-critical scenes in which the automated vehicle's planner actually produces a trajectory toward a pedestrian or cyclist, label the radar points that reflect that vulnerable road user, and run the pipeline: if recall of those independently labeled critical points is not near 95% and the fraction of non-clustered critical points does not drop by roughly three quarters relative to a moderate RCS filter, the framework's headline numbers will not generalize.
Extended reading notes
Core claim
The paper's central claim is that safety-relevant radar perception can keep weak reflections of vulnerable road users without sacrificing noise robustness by making the filter decision depend on where the vehicle intends to drive. Each radar point receives a criticality score $\operatorname{crit}_p = \operatorname{crit}_{\mathrm{vel}} \cdot \operatorname{crit}_{\mathrm{tube}} \cdot \operatorname{crit}_{\mathrm{ttc}}$, where the velocity component grows quadratically with collision speed, the tube component is 1 inside a driving tube of half the vehicle width plus a safety margin and decays smoothly outside it, and the distance component reflects the stopping distance. Points above a threshold are "critical"; around them, the pipeline opens criticality regions in which the RCS filter is switched off for subsequent point clouds, with region radii growing over a five-cycle lifetime. On the RadarScenes data set with artificially critical trajectories toward pedestrians and bicyclists, the authors find that this treatment raises the share of true critical points that end up clustered to roughly 76–77%, independent of the baseline filter strength, and lowers the rate of non-clustered critical points by 74.8% relative to a moderate generic filter in their exemplary setup.
Load-bearing premise
The ground truth for 'true critical points' is made by bending the planned path toward pedestrians using the same road-geometry idea that the scoring method uses to decide what is critical; if such artificial paths do not represent what a real safety planner would output, the reported 95% recall and 74.8% improvement may not transfer to real driving.
Editorial extensions
If this is right
- At the strongest tested RCS filter, the share of true critical points that get clustered rises from 0.04% without the treatment to 76.30% with it.
- The criticality metric reaches 95.02% recall at a safety-oriented threshold, so nearly all artificial critical VRU points are flagged and routed around the filter.
- The improvement is stable across filter settings, with 76.6–77.5% of critical points correctly clustered under all four thresholds tested.
- The posteriori method reduces the rate of non-clustered critical points by 74.8% relative to a moderate generic filter.
- The mechanism transfers to a velocity-based criticality metric, though with a larger false-positive burden.
Reading between the lines
- If real safety-critical scenes with planner-issued trajectories were available, the recall measure would be more credible; the current ground truth reuses the same drive-tube geometry that the metric itself consumes, so an independent planner or human-annotated critical set is the natural next test.
- Because precision at the chosen threshold is only about 3.7%, a production deployment would likely pair criticality-aware filtering with confirmation actions—temporal evidence, camera association, or a second sensor—before releasing a braking decision; the paper sketches these actions but does not implement them.
- The posteriori delay of several measurement cycles could let a faster-moving cyclist escape the expanding region; coupling region growth to estimated object velocity or to an upstream tracker would be a direct testable extension.
- Applying the same criticality-weighted threshold relaxation at the digital signal processing level, as the paper suggests, would move the safety gain further upstream and could make error-rate guarantees more tractable.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SAFERad, a framework for safety-aware radar point cloud filtering. It computes a per-point criticality score from the vehicle's planned trajectory and drive-tube geometry, defines criticality regions around critical points, and in those regions lowers the RCS filter threshold so that weak radar reflections (e.g., from pedestrians) are not deleted. The method is evaluated on the RadarScenes dataset by artificially adapting the planned trajectory toward vulnerable road users to create ground-truth critical points. The authors report a 95.02% recall for the criticality evaluation (at threshold 0.1) and a 74.8% reduction in non-clustered critical points with the posteriori method compared to a moderate generic filter.
Significance. The idea of making radar point-cloud filtering safety-aware is timely and relevant for SOTIF-oriented perception. The contribution is a concrete pipeline: per-point criticality scoring, criticality regions, and threshold adaptation. The qualitative motivation is strong, and the paper includes a comparison with a velocity-based criticality baseline. However, the experimental validation is undermined by a circular ground-truth construction: the set of 'true critical' points is generated by the same drive-tube/reachability model that the criticality metric itself uses. In addition, a key metric component (distance criticality) is left undefined, and the reported improvement in non-clustered points conflates filtering and clustering effects. If the framework were evaluated with independent ground truth or reframed as a synthetic in-sample validation, the core idea would be publishable; in its current form, the headline quantitative claims are not supported.
major comments (4)
- [Section IV, Set B construction; Section III-E; Fig. 8] The ground-truth critical points (Set B) are constructed by adapting the planned trajectory toward VRUs using the same reachability analysis (Eqs. (3)-(4)) and drive-tube geometry that the criticality metric consumes. The modified trajectory both creates the true critical labels and determines the tube in which points are scored critical. Therefore, the reported 95.02% recall (Table I) and the 74.8% reduction in non-clustered points are largely self-consistency checks of one trajectory model, not evidence about real safety-relevant perception. The paper itself concedes in Section VI that this ground truth 'lacks accuracy' and that 'only single measurements could be investigated.' Please re-frame the evaluation as an in-sample validation or obtain independent critical-point labels (e.g., from a different planner model, a simulator, or human-annotated near-miss data). The conclusion's statement that the method 'proved' robust identification of true critical points is not supported without such a change.
- [Table III and Table IV, set definitions in Section V-A] The column labeled 'B ∩ ¯C' in Table III appears to denote B∩F∩¯C (points that pass the RCS filter but are not clustered), not all non-clustered critical points. For example, at a threshold of 0.0 dB m2, Table III reports B∩¯F = 185,203 and B∩¯C = 1,085; if B∩¯C meant 'all not clustered', it would have to be at least 185,203. This mislabeling makes the abstract's 74.8% reduction in 'non-clustered critical points' ambiguous, since the reduction could be dominated by points that are no longer filtered out rather than by points that become clustered. Please define F, C, and the disjoint categories B∩C, B∩F∩¯C, and B∩¯F precisely, and report these categories separately for the baseline and the posteriori method for all thresholds.
- [Section III-B, Eq. (2)] The distance criticality critttc is never defined mathematically. The text states that it is 'energy-driven' and that it grows quadratically with the velocity difference, but no formula is given. Since critttc is a multiplicative factor in Eq. (2) that determines membership in Set A, the recall and precision values in Table I are not reproducible without this definition. Please provide an explicit equation for critttc that shows its dependence on distance, reaction time treact, and deceleration a, and state the parameter values used in the experiments.
- [Section V-C, Table IV; Section VI] The posteriori method increases the total number of clustered points by 77.6% to 2330% depending on the filter threshold (Table IV). The paper notes that RadarScenes does not distinguish static objects from noise, so these new clusters could largely be false positives that would trigger spurious planner interventions. Without false-positive labels, the safety benefit of the method cannot be assessed. Please either quantify false positives using a dataset with noise/static-object annotations or prominently state that the false-positive cost is unknown and that the reported improvement is only in recall-oriented metrics.
minor comments (7)
- [Abstract] The sentence 'Evaluation of the criticality metric prove high recall rates' should be 'proves'.
- [Section V-A] The sentence 'The trajectory is modified during 12.6 % of all timestamps' is ambiguous; clarify whether this refers to all 158 sequences or only the 109 sequences that contained reachable VRU points.
- [Figure 4] The color bars are not labeled with the criticality variable, and the two panels use different upper clipping values (0.5 vs. 1.0), making visual comparison difficult.
- [Section III-F] The posteriori region radii r1...r5 are listed as 0.2, 0.4, 0.6, 0.8, 1.0 m, but it is not explicitly stated that r_k corresponds to the k-th cycle after region creation; specify this mapping and the region lifetime tlif e=5 cycles in the text.
- [Section V-D, Eq. (8)] The comparison criticality formula 'crit = (|vdopp,comp|−0.4)/1.1' is not fully specified; please state the clipping behavior below 0.4 and above 1.5 m/s and whether the metric is the same for positive and negative relative velocities.
- [Section III-B] The reaction time treact is mentioned in the description of distance criticality but its value used in the experiments is never given; please provide it.
- [Figure 1] The blocks in the 'red box' section of the framework overview are not labeled; label the criticality evaluation and posteriori processing blocks to match the text.
Circularity Check
Ground-truth set B is generated with the same reachability/trajectory model that defines the criticality metric, making the reported 95% recall and 74.8% reduction in-sample; the posteriori regions then reinforce the same constructed set.
-
self definitional
[Section IV (definition of set B) and Section III-E (Reachability Analysis), with the criticality metric in Section III-B]
"Once a reachable point preach is found which belongs to one of the semantic classes [bicycles, pedestrians, group of pedestrians], this point is added to set B and the search is terminated. However, all points with the same track id as the point preach are also classified as critical points and added to set B, as far as the Euclidean distance to the point preach is less than 2.0 m. ... The critical trajectory is found using the method outlined in section III-E."
Set B is not an independent 'true critical' label: membership requires that a reachable circular-arc trajectory (Eqs. 3-4, Sec. III-E) can be adapted toward the VRU point using the same v0, yaw rate, vcoll, and drive-tube geometry that Section III-B's criticality metric scores. The metric's components critvel, crittube and critttc are positive exactly for points in the tube, at collision speed and close to the vehicle; a reachable target point on the adapted arc satisfies these by construction. Hence recall=95.02% over B largely confirms that A recognizes B's own generation rule; it is not a falsifiable test against independent safety ground truth.
-
other
[Section III-F (Point Treatment) and Table IV (Results of the Posteriori Processing Method)]
"These criticality regions are defined for the following measurements Pt+1, . . . ,Pt+Nposterior around each critical point of the current point cloud Pt. ... we propose to completely remove the filter criterion for the RCS value of each radar point which lies inside of a criticality region."
The 74.8% reduction in non-clustered B points is obtained after placing filter-exemption regions around the metric's critical points (Set A). Because Step 1 makes A coincide with B at 95% recall, the regions cover the constructed B, so the post-filter clustering improvement is an in-sample consequence of the same model rather than independent evidence. Moreover, Table IV shows total clustered points increase by up to +2330% and the paper states RadarScenes 'does not distinguish between static objects and noise points,' so the false-positive cost of suspending the RCS filter is unquantified; the claimed safety gain therefore rests on the same constructed set B.
full rationale
The central circularity is experimental, not bibliographic: no load-bearing self-citation chain was found, and the method has independent engineering content (per-point criticality scoring, DBSCAN clustering, comparison against a velocity-based metric). However, the paper's headline quantitative claims are weakened by construction. The 'true critical' ground-truth set B is created by adapting the planned trajectory toward VRU points using the same reachable circular-arc model, vehicle dynamics, and drive-tube concepts that the criticality metric consumes; points are then labeled critical at the reachable target and its track-id neighbors within 2.0 m. Thus a high recall of set A over set B is partly a self-consistency check of one trajectory model, not evidence that the metric identifies genuinely safety-critical radar points in the real world. The posteriori filtering result is then evaluated on the same constructed B, with exemption regions drawn around the metric's own critical points, so the reduction in non-clustered B points is substantially in-sample. The paper explicitly concedes that the ground truth 'lacks accuracy in some cases' and that RadarScenes cannot quantify static-object versus noise false positives. These limitations do not make the framework useless, but they mean the reported 95.02% recall and 74.8% improvement should not be read as externally validated safety performance. A score of 6 reflects partial circularity: the central evaluation reduces by construction, while the pipeline itself retains independent and potentially useful content.
Assumptions & free parameters
free parameters (9)
- Criticality threshold critthresh =
0.1 (safety setup), 0.35 (F1 optimum)
- vmax,domain =
30 km/h, later 8.5 m/s in experiments
- Tube safety margin =
0.1 m
- Insecurity zone width =
2.0 m
- DBSCAN epsilon =
1.0 m
- Posteriori region radii =
0.2, 0.4, 0.6, 0.8, 1.0 m for ages 1 to 5
- Reachability dynamics limits =
along,max=10 m/s2, alat,max=8 m/s2, rmin=6 m
- Velocity filter bounds =
|vcomp| >= 0.5 m/s, vdopp <= 20 m/s
- Reaction time and deceleration a =
not specified
assumptions (5)
- domain assumption RadarScenes semantic labels correctly identify vulnerable road user points.
- ad hoc to paper A point inside the drive tube with high speed and short stopping distance is a valid proxy for collision criticality.
- ad hoc to paper The artificially modified trajectory is an acceptable proxy for a real safety-critical planning output.
- domain assumption DBSCAN clustering with epsilon 1.0 and minpts 4 is sufficient to decide whether a critical point was correctly treated.
- domain assumption Static radar reflections are not false positives; only moving non-VRU points would be false positives.
Cite this review
Pith. "Pith review of SAFERad: A Framework to Enable Radar Data for Safety-Relevant Perception Tasks." pith.science (2026). https://pith.science/paper/JCVO4DX4
@misc{pith2026250703959,
author = {Pith},
title = {Pith review of: SAFERad: A Framework to Enable Radar Data for Safety-Relevant Perception Tasks},
year = {2026},
howpublished = {\url{https://pith.science/paper/JCVO4DX4}},
note = {Machine review of arXiv:2507.03959}
}
read the original abstract
Radar sensors play a crucial role for perception systems in automated driving but suffer from a high level of noise. In the past, this could be solved by strict filters, which remove most false positives at the expense of undetected objects. Future highly automated functions are much more demanding with respect to error rate. Hence, if the radar sensor serves as a component of perception systems for such functions, a simple filter strategy cannot be applied. In this paper, we present a modified filtering approach which is characterized by the idea to vary the filtering depending on the potential of harmful collision with the object which is potentially represented by the radar point. We propose an algorithm which determines a criticality score for each point based on the planned or presumable trajectory of the automated vehicle. Points identified as very critical can trigger manifold actions to confirm or deny object presence. Our pipeline introduces criticality regions. The filter threshold in these criticality regions is omitted. Commonly known radar data sets do not or barely feature critical scenes. Thus, we present an approach to evaluate our framework by adapting the planned trajectory towards vulnerable road users, which serve as ground truth critical points. Evaluation of the criticality metric prove high recall rates. Besides, our post-processing algorithm lowers the rate of non-clustered critical points by 74.8 % in an exemplary setup compared to a moderate, generic filter.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
SAE International., “J3016 - Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles,” Online, Tech. Rep., April 2021
work page 2021
-
[2]
Sensor and Sensor Fusion Technology in Autonomous Vehicles: A Review,
D. J. Yeong, G. Velasco-Hernandez, J. Barry, and J. Walsh, “Sensor and Sensor Fusion Technology in Autonomous Vehicles: A Review,” Sensors, vol. 21, no. 6, p. 2140, Jan. 2021
work page 2021
-
[3]
Perception and sensing for autonomous vehicles under adverse weather conditions: A survey,
Y . Zhang, A. Carballo, H. Yang, and K. Takeda, “Perception and sensing for autonomous vehicles under adverse weather conditions: A survey,” ISPRS Journal of Photogrammetry and Remote Sensing , vol. 196, pp. 146–177, Feb. 2023. 13
work page 2023
-
[4]
E. F. Knott, J. Shaeffer, and M. T. Tuley, “Radar cross section,” in Aspects of modern radar . Artech House Norwood, 1988, pp. 444– 448
work page 1988
-
[5]
Radars for Autonomous Driving: A Review of Deep Learning Methods and Challenges,
A. Srivastav and S. Mandal, “Radars for Autonomous Driving: A Review of Deep Learning Methods and Challenges,” IEEE Access , vol. 11, pp. 97 147–97 168, 2023
work page 2023
-
[6]
Multi-Object Tracking with mmWave Radar: A Review,
A. Pearce, J. A. Zhang, R. Xu, and K. Wu, “Multi-Object Tracking with mmWave Radar: A Review,” Electronics, vol. 12, no. 2, p. 308, Jan. 2023
work page 2023
-
[7]
A density-based algorithm for discovering clusters in large spatial databases with noise,
M. Ester, H.-P. Kriegel, J. Sander, X. Xu et al. , “A density-based algorithm for discovering clusters in large spatial databases with noise,” in kdd, vol. 96, no. 34, 1996, pp. 226–231
1996
-
[8]
Safety-Aware Metric for People Detection,
M. Wolf, L. R. Douat, and M. Erz, “Safety-Aware Metric for People Detection,” in 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) , Sep. 2021, pp. 2759–2765
work page 2021
Show all 21 references
-
[9]
Refining Obstacle Perception Safety Zones via Maneuver-Based Decomposition,
S. Topan, Y . Chen, E. Schmerling, K. Leung, J. Nilsson, M. Cox, and M. Pavone, “Refining Obstacle Perception Safety Zones via Maneuver-Based Decomposition,” in 2023 IEEE Intelligent V ehicles Symposium (IV) , Jun. 2023, pp. 1–8, iSSN: 2642-7214
2023
-
[10]
Maneuver-based adap- tive safety zone for infrastructure-supported automated valet parking
V . Sch ¨onemann, M. Duschek, and H. Winner, “Maneuver-based adap- tive safety zone for infrastructure-supported automated valet parking.” 2019
2019
-
[11]
Criticality-Based Data Segmentation and Resource Allocation in Machine Inference Pipelines,
S. Liu, L. Sha, and T. Abdelzaher, “Criticality-Based Data Segmentation and Resource Allocation in Machine Inference Pipelines,” M. Srivatsa, T. Abdelzaher, and T. He, Eds. Cham: Springer International Publishing, 2023, pp. 335–352
2023
-
[12]
Towards Safety-Aware Pedestrian Detection in Autonomous Systems,
M. Lyssenko, C. Gladisch, C. Heinzemann, M. Woehrle, and R. Triebel, “Towards Safety-Aware Pedestrian Detection in Autonomous Systems,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , Oct. 2022, pp. 293–300, iSSN: 2153-0866
2022
-
[13]
Focal loss for dense object detection,
T.-Y . Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, “Focal loss for dense object detection,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV) , Oct 2017
2017
-
[14]
Systematization of Relevant Road Users for the Evaluation of Autonomous Vehicle Perception,
R. Philipp, J. Rehbein, F. Gr ¨un, L. Hartjen, Z. Zhu, F. Schuldt, and F. Howar, “Systematization of Relevant Road Users for the Evaluation of Autonomous Vehicle Perception,” in 2022 IEEE International Systems Conference (SysCon) , Apr. 2022, pp. 1–8, iSSN: 2472-9647
2022
-
[15]
Task-aware risk estimation of perception failures for autonomous vehicle,
P. Antonante, S. Veer, K. Leung, X. Weng, L. Carlone, and M. Pavone, “Task-aware risk estimation of perception failures for autonomous vehicle,” in Robotics: Science and Systems (RSS) . Robotics: Science and Systems (RSS), 2023
2023
-
[16]
Object criticality for safer navigation,
A. Ceccarelli and L. Montecchi, “Object criticality for safer navigation,” Tech. Rep., Apr. 2024, arXiv:2406.10232 [cs] type: article
2024 arXiv
-
[17]
PolarPoint-BEV: Bird-eye-view Perception in Polar Points for Explainable End-to-end Autonomous Driving,
Y . Feng and Y . Sun, “PolarPoint-BEV: Bird-eye-view Perception in Polar Points for Explainable End-to-end Autonomous Driving,” IEEE Transactions on Intelligent V ehicles, pp. 1–11, 2024
2024
-
[18]
Vehicle detection with automotive radar using deep learning on range-azimuth- doppler tensors,
B. Major, D. Fontijne, A. Ansari, R. Teja Sukhavasi, R. Gowaikar, M. Hamilton, S. Lee, S. Grzechnik, and S. Subramanian, “Vehicle detection with automotive radar using deep learning on range-azimuth- doppler tensors,” in Proceedings of the IEEE/CVF International Con- ference o...
2019
-
[19]
RCS measurements for vehicles and pedestrian at 26 and 79GHz,
I. Matsunami, R. Nakamura, and A. Kajiwara, “RCS measurements for vehicles and pedestrian at 26 and 79GHz,” in 2012 6th International Conference on Signal Processing and Communication Systems, Dec. 2012, pp. 1–4
2012
-
[20]
RCS Patterns of Pedestrians at 76-77 GHz,
M. Chen and C.-C. Chen, “RCS Patterns of Pedestrians at 76-77 GHz,” IEEE Antennas and Propagation Magazine , vol. 56, no. 4, pp. 252–263, Aug. 2014
2014
-
[21]
RadarScenes: A Real-World Radar Point Cloud Data Set for Automotive Applications,
O. Schumann, M. Hahn, N. Scheiner, F. Weishaupt, J. F. Tilly, J. Dickmann, and C. W ¨ohler, “RadarScenes: A Real-World Radar Point Cloud Data Set for Automotive Applications,” in 2021 IEEE 24th International Conference on Information Fusion (FUSION) , Nov. 2021, pp. 1–8. Tim B...
2021
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.