REVIEW 3 major objections 4 minor 48 references
A multi-modal dataset and tracker pipeline can put temporary roadwork objects on the map to within one meter of ground truth GPS.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-11 20:00 UTC pith:W35FVWEE
load-bearing objection Useful public multi-modal roadwork dataset with object-level GPS; the geo-localization numbers rest on only 43 real objects and the pipeline is a thin AB3DMOT extension. the 3 major comments →
Framework and Multi-modal Dataset for Roadwork Zone Detection and Geo-localization
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The authors show that an extension of a standard 3-D multi-object tracker, fed by camera- or LiDAR-based detectors and ego GPS/IMU, can convert repeated local detections of roadwork objects into unique global coordinates that stay inside a one-meter Haversine ball of RTK ground truth on both real and simulated data.
What carries the argument
The RZDG pipeline: 3-D detections are tracked into tracklets, each tracklet is collapsed to a single local center (last-frame or weighted-average), and that center is transformed into WGS84 latitude-longitude by the ego vehicle’s GPS/IMU and heading.
Load-bearing premise
That the 43 real objects whose GPS was measured by parking an RTK receiver over their centers, drawn from only three consecutive scenes, are enough to prove the pipeline works in general.
What would settle it
Collect RTK ground truth for a larger, independent set of real roadwork objects (different cities, weather, denser clutter) and recompute the one-meter Haversine precision/recall; a clear drop below the reported F1 would falsify the claim of practical accuracy.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces the RZDG multi-modal dataset (real + CARLA-simulated) with camera/LiDAR/GPS-IMU data and annotations for semantic segmentation, 3D bounding boxes, and object global positions of roadwork elements (barriers, road beacons). It also presents a tracker-based geo-localization pipeline that extends AB3DMOT: 3D detections are tracked into tracklets, fused (last-frame or weighted-average), and transformed from the camera frame to WGS84 via ego GPS/IMU using the spherical trigonometry of Eqs. (1)–(7). Under a 1 m Haversine true-positive rule the pipeline reports Precision/Recall/F1 of 0.565/0.898/0.597 (real) and 0.615/0.809/0.665 (sim) for the best detector–fusion combination; standard AP and mIoU numbers are also given for three detectors and three segmentors.
Significance. If the numbers hold under broader evaluation, the work supplies a missing public multi-modal benchmark for temporary roadwork zones—an acknowledged gap in HD-map maintenance for autonomous driving—and a simple, detector-agnostic pipeline that converts local detections into global coordinates. Explicit strengths include the promised open release of both real and simulated data plus code, multi-task annotations in KITTI format, and transparent reporting of train/val splits and three detector families. The contribution is therefore primarily infrastructural rather than algorithmic; the pipeline itself is a straightforward extension of AB3DMOT plus textbook coordinate conversion.
major comments (3)
- Section III-B/C and VI-B: real-world geo-localization metrics in Table VI rest exclusively on 43 RTK-measured objects collected across only three consecutive scenes. No confidence intervals, bootstrap estimates, leave-one-scene-out results, or uncertainty on the RTK ground truth itself are reported. With N=43 the headline F1 of 0.597 (and the abstract’s claim of “high accuracy”) is statistically under-powered and cannot be taken as a reliable estimate of pipeline performance.
- Abstract and Section VIII: the language “high accuracy” is inconsistent with the reported real-data Precision of 0.565 (Table VI, LF-MVXNet) and with the modest 3D detection APs in Table V (MVXNet ~33–39 % AP3D on real data). The discrepancy between low detection quality and high geo-localization recall needs explicit discussion or a more conservative claim.
- Section V-B and Table VI: the two tracklet-fusion strategies (LF vs WA) produce dramatically different rankings on real versus simulated data, yet no analysis of the failure modes (e.g., track fragmentation, ego-pose noise, or calibration residual) is supplied. Because the coordinate transform (Eqs. 1–7) is deterministic once a tracklet is chosen, the fusion choice is load-bearing; its sensitivity should be quantified.
minor comments (4)
- Throughout: inconsistent spacing and hyphenation (“A Vs”, “V oxelFusion”, “W A”, “H. thres.a”) should be cleaned for readability.
- Table I: the column “# Object’s global position 43/673” is useful but the caption should clarify that 43 is the only real-world geo-localization ground truth used for evaluation.
- Section III-A: the sensor-car figure caption mentions seven cameras and three radars while the text and dataset use only one camera + one LiDAR + GPS/IMU; a clarifying sentence would avoid confusion.
- Equations (1)–(7): the Earth radius is given as 6 372 800 m without citation or discussion of the approximation error relative to the 1 m TP threshold.
Circularity Check
No circularity: empirical multi-sensor pipeline and dataset benchmarked against independent RTK/simulator ground truth with fixed 1 m Haversine rule; no fitted parameters or self-definitional reductions.
full rationale
The paper presents a new multi-modal dataset (RZDG-real/sim) with independent annotations (semantic masks, 3D boxes, and 43 real RTK-measured object GPS positions obtained by placing a receiver over object centers) plus a tracker-based pipeline that extends AB3DMOT with standard 3D Kalman/Hungarian tracking, two simple tracklet-fusion heuristics (last-frame or weighted average), and textbook coordinate transforms (Eqs. 1-7 from an external navigation reference, plus Haversine distance). Performance numbers (Precision/Recall/F1 under a fixed 1 m TP threshold) are pure empirical evaluations of trained detectors + tracker on held-out scenes; no free parameter is fitted to the geo-localization F1 itself, no quantity is defined in terms of the reported metrics, and no uniqueness theorem or ansatz is imported via overlapping-author citation. The derivation chain is therefore self-contained engineering evaluation against external ground truth, not a circular reduction.
Axiom & Free-Parameter Ledger
free parameters (2)
- Haversine TP threshold =
1 meter
- Tracklet fusion strategy (LF vs WA) =
LF preferred on real data
axioms (4)
- domain assumption Constant-velocity 3D Kalman filter + Hungarian association (AB3DMOT) correctly links detections of the same static roadwork object across frames.
- domain assumption RTK GPS placed over object centers yields ground-truth latitude/longitude accurate to a few centimeters, sufficient for a 1 m evaluation threshold.
- standard math Earth radius R = 6 372 800 m and the spherical Haversine formulas (Eqs. 8–10) are adequate for meter-scale distances.
- domain assumption CARLA sensor noise models and imported CAD assets are sufficiently realistic that sim results transfer to real performance claims.
read the original abstract
Autonomous vehicles often rely on high-definition (HD) maps for navigation; however, these maps are not frequently updated and often lack semi-static information, such as temporary roadwork zones, which can significantly alter the road network. This limitation underscores the urgent need for an accurate global position of roadwork zones. However, the absence of publicly available datasets for evaluating roadwork zone detection and geo-localization models has hindered the development of reliable autonomous driving systems. To address this challenge, we propose the Roadwork Zone Detection and Geo-localization (RZDG) dataset, which includes both simulated and real-world data, providing multimodal sensor inputs along with comprehensive annotations. The dataset supports multiple perception tasks, including image semantic segmentation, 3D object detection, and object geo-localization. In addition, we introduce a tracker-based roadwork zone detection and geo-localization (RZDG) pipeline, an extension of AB3DMOT, for accurate object geo-localization in roadwork zones. We benchmark our approach on the RZDG dataset, demonstrating its effectiveness in detecting roadwork zones and transforming object positions from the local coordinate system to the global coordinate system. A prediction is considered a true positive (TP) if its estimated position falls within one meter of the ground truth. Our experimental results show that our approach achieves high accuracy on both real and simulated data. Specifically, we report: Precision: 0.565 (real) / 0.615 (simulated) Recall: 0.898 (real) / 0.809 (simulated) F1-score: 0.597 (real) / 0.665 (simulated).
Figures
Reference graph
Works this paper leans on
-
[1]
ildm: An interoperable graph-based local dynamic map,
M. Garc ´ıa, I. Urbieta, M. Nieto, J. Gonz ´alez de Mendibil, and O. Otaegui, “ildm: An interoperable graph-based local dynamic map,” Vehicles, vol. 4, no. 1, pp. 42–59, 2022
2022
-
[2]
Intelligent transport systems (its); vehicular communica- tions; basic set of applications; definitions,
T. ETSI, “Intelligent transport systems (its); vehicular communica- tions; basic set of applications; definitions,”Tech. Rep. ETSI TR 102 6382009, 2009
2009
-
[3]
Characterization, detection, and segmentation of work-zone scenes from naturalistic driving data,
V . Sundharam, A. Sarkar, A. Svetovidov, J. S. Hickman, and A. L. Abbott, “Characterization, detection, and segmentation of work-zone scenes from naturalistic driving data,”Transportation research record, vol. 2677, no. 3, pp. 490–504, 2023
2023
-
[4]
Low-cost object detection models for traffic control devices through domain adaption of geographical regions,
D. Oh, K. Kang, S. Seo, J. Xiao, K. Jang, K. Kim, H. Park, and J. Won, “Low-cost object detection models for traffic control devices through domain adaption of geographical regions,”Remote Sensing, vol. 15, no. 10, p. 2584, 2023
2023
-
[5]
Tracon: A novel dataset for real-time traffic cones detection using deep learning,
I. Katsamenis, E. E. Karolou, A. Davradou, E. Protopapadakis, A. Doulamis, N. Doulamis, and D. Kalogeras, “Tracon: A novel dataset for real-time traffic cones detection using deep learning,” in Novel & Intelligent Digital Systems Conferences. Springer, 2022, pp. 382–391
2022
-
[6]
Work zone detection for autonomous ve- hicles,
W. Shi and R. R. Rajkumar, “Work zone detection for autonomous ve- hicles,” in2021 IEEE International Intelligent Transportation Systems Conference (ITSC). IEEE, 2021, pp. 1585–1591
2021
-
[7]
Ab3dmot: A baseline for 3d multi-object tracking and new evaluation metrics,
X. Weng, J. Wang, D. Held, and K. Kitani, “Ab3dmot: A baseline for 3d multi-object tracking and new evaluation metrics,”arXiv preprint arXiv:2008.08063, 2020
Pith/arXiv arXiv 2008
-
[8]
Effects of highway work zone temporary countermeasures,
F. Shahin, W. Elias, and T. Toledo, “Effects of highway work zone temporary countermeasures,”European Transport Research Review, vol. 15, no. 1, p. 20, 2023
2023
-
[9]
Encoder-decoder with atrous separable convolution for semantic image segmentation,
L.-C. Chen, Y . Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” inProceedings of the European conference on computer vision (ECCV), 2018, pp. 801–818
2018
-
[10]
Swin transformer: Hierarchical vision transformer using shifted windows,
Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10 012–10 022
2021
-
[11]
Segformer: Simple and efficient design for semantic segmentation with transformers,
E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “Segformer: Simple and efficient design for semantic segmentation with transformers,”Advances in neural information processing sys- tems, vol. 34, pp. 12 077–12 090, 2021
2021
-
[12]
Gs3d: An efficient 3d object detection framework for autonomous driving,
B. Li, W. Ouyang, L. Sheng, X. Zeng, and X. Wang, “Gs3d: An efficient 3d object detection framework for autonomous driving,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 1019–1028
2019
-
[13]
Monocular 3d object detection with decoupled structured polygon estimation and height-guided depth estimation,
Y . Cai, B. Li, Z. Jiao, H. Li, X. Zeng, and X. Wang, “Monocular 3d object detection with decoupled structured polygon estimation and height-guided depth estimation,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, 2020, pp. 10 478– 10 485
2020
-
[14]
Monocular 3d object detection for autonomous driving,
X. Chen, K. Kundu, Z. Zhang, H. Ma, S. Fidler, and R. Urtasun, “Monocular 3d object detection for autonomous driving,” inPro- ceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2147–2156
2016
-
[15]
Ground-aware monocular 3d object detection for autonomous driving,
Y . Liu, Y . Yixuan, and M. Liu, “Ground-aware monocular 3d object detection for autonomous driving,”IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 919–926, 2021
2021
-
[16]
Smoke: Single-stage monocular 3d object detection via keypoint estimation,
Z. Liu, Z. Wu, and R. T ´oth, “Smoke: Single-stage monocular 3d object detection via keypoint estimation,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 2020, pp. 996–997
2020
-
[17]
Objects are different: Flexible monocu- lar 3d object detection,
Y . Zhang, J. Lu, and J. Zhou, “Objects are different: Flexible monocu- lar 3d object detection,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 3289–3298
2021
-
[18]
Pointnet: Deep learning on point sets for 3d classification and segmentation,
R. Q. Charles, H. Su, M. Kaichun, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 77–85
2017
-
[19]
Dynamic graph cnn for learning on point clouds,
Y . Wang, Y . Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, “Dynamic graph cnn for learning on point clouds,” ACM Trans. Graph., vol. 38, no. 5, Oct. 2019. [Online]. Available: https://doi.org/10.1145/3326362
doi:10.1145/3326362 2019
-
[20]
Kpconv: Flexible and deformable convolution for point clouds,
H. Thomas, C. R. Qi, J.-E. Deschaud, B. Marcotegui, F. Goulette, and L. J. Guibas, “Kpconv: Flexible and deformable convolution for point clouds,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019
2019
-
[21]
V oxelnet: End-to-end learning for point cloud based 3d object detection,
Y . Zhou and O. Tuzel, “V oxelnet: End-to-end learning for point cloud based 3d object detection,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018
2018
-
[22]
Transformer-based global pointpillars 3d object detection method,
L. Zhang, H. Meng, Y . Yan, and X. Xu, “Transformer-based global pointpillars 3d object detection method,”Electronics, vol. 12, no. 14, p. 3092, 2023. [Online]. Available: https://doi.org/10.3390/ electronics12143092
2023
-
[23]
Pv- rcnn: Point-voxel feature set abstraction for 3d object detection,
S. Shi, C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li, “Pv- rcnn: Point-voxel feature set abstraction for 3d object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020
2020
-
[24]
Pointpillars: Fast encoders for object detection from point clouds,
A. H. Lang, S. V ora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for object detection from point clouds,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019
2019
-
[25]
Fusion of 3d lidar and camera data for object detection in autonomous vehicle applications,
X. Zhao, P. Sun, Z. Xu, H. Min, and H. Yu, “Fusion of 3d lidar and camera data for object detection in autonomous vehicle applications,” IEEE Sensors Journal, vol. 20, no. 9, pp. 4901–4913, 2020
2020
-
[26]
Mvx-net: Multimodal voxelnet for 3d object detection,
V . A. Sindagi, Y . Zhou, and O. Tuzel, “Mvx-net: Multimodal voxelnet for 3d object detection,” in2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019, pp. 7276–7282
2019
-
[27]
3d dual-fusion: Dual-domain dual-query camera-lidar fusion for 3d object detection,
Y . Kim, K. Park, M. Kim, D. Kum, and J. W. Choi, “3d dual-fusion: Dual-domain dual-query camera-lidar fusion for 3d object detection,” arXiv preprint arXiv:2211.13529, 2022
Pith/arXiv arXiv 2022
-
[28]
Image and object geo-localization,
D. Wilson, X. Zhang, W. Sultani, and S. Wshah, “Image and object geo-localization,”International Journal of Computer Vision, vol. 132, no. 4, pp. 1350–1392, 2024
2024
-
[29]
Road deformation detection,
K. A. Govada, H. P. Jonnalagadda, P. Kapavarapu, S. Alavala, and K. S. Vani, “Road deformation detection,” in2020 7th International Conference on Smart Structures and Systems (ICSSS). IEEE, 2020, pp. 1–5
2020
-
[30]
Object tracking and geo-localization from street images,
D. Wilson, T. Alshaabi, C. Van Oort, X. Zhang, J. Nelson, and S. Wshah, “Object tracking and geo-localization from street images,” Remote Sensing, vol. 14, no. 11, p. 2575, 2022
2022
-
[31]
End-to-end learning improves static object geo-localization from video,
M. Chaabane, L. Gueguen, A. Trabelsi, R. Beveridge, and S. O’Hara, “End-to-end learning improves static object geo-localization from video,” inProceedings of the IEEE/CVF Winter Conference on Ap- plications of Computer Vision, 2021, pp. 2063–2072
2021
-
[32]
Automatic discovery and geotagging of objects from street view imagery,
V . A. Krylov, E. Kenny, and R. Dahyot, “Automatic discovery and geotagging of objects from street view imagery,”Remote Sensing, vol. 10, no. 5, p. 661, 2018
2018
-
[33]
Geolocation estimation of target vehicles using image processing and geometric computation,
E. Namazi, R. Mester, C. Lu, and J. Li, “Geolocation estimation of target vehicles using image processing and geometric computation,” Neurocomputing, vol. 499, pp. 35–46, 2022
2022
-
[34]
Vision meets robotics: The kitti dataset,
A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, “Vision meets robotics: The kitti dataset,”The International Journal of Robotics Research, vol. 32, no. 11, pp. 1231–1237, 2013
2013
-
[35]
Robot operating system 2: Design, architecture, and uses in the wild,
S. Macenski, T. Foote, B. Gerkey, C. Lalancette, and W. Woodall, “Robot operating system 2: Design, architecture, and uses in the wild,” Science robotics, vol. 7, no. 66, p. eabm6074, 2022
2022
-
[36]
Audi a6 avant
Bibliocad, “Audi a6 avant.” [Online]. Available: https://www. bibliocad.com/de/library/audi-a6-avant- 12984/
-
[37]
Planet dump retrieved from https://planet.osm.org ,
OpenStreetMap contributors, “Planet dump retrieved from https://planet.osm.org ,” https://www.openstreetmap.org, 2017
2017
-
[38]
A flexible new technique for camera calibration,
Z. Zhang, “A flexible new technique for camera calibration,”IEEE Transactions on pattern analysis and machine intelligence, vol. 22, no. 11, pp. 1330–1334, 2000
2000
-
[39]
Calib-anything: Zero-training lidar-camera extrinsic calibration method using segment anything,
Z. Luo, G. Yan, and Y . Li, “Calib-anything: Zero-training lidar-camera extrinsic calibration method using segment anything,”arXiv preprint arXiv:2306.02656, 2023
Pith/arXiv arXiv 2023
-
[40]
Yolov8: A novel object detection algorithm with enhanced performance and robustness,
R. Varghese and S. M., “Yolov8: A novel object detection algorithm with enhanced performance and robustness,” in2024 International Conference on Advances in Data Engineering and Intelligent Com- puting Systems (ADICS), 2024, pp. 1–6
2024
-
[41]
Sustech points: A portable 3d point cloud interactive annotation platform system,
E. Li, S. Wang, C. Li, D. Li, X. Wu, and Q. Hao, “Sustech points: A portable 3d point cloud interactive annotation platform system,” in 2020 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2020, pp. 1108–1115
2020
-
[42]
Labelme: Online image annotation and applications,
A. Torralba, B. C. Russell, and J. Yuen, “Labelme: Online image annotation and applications,”Proceedings of the IEEE, vol. 98, no. 8, pp. 1467–1484, 2010
2010
-
[43]
Carla: An open urban driving simulator,
A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun, “Carla: An open urban driving simulator,” inConference on robot learning. PMLR, 2017, pp. 1–16
2017
-
[44]
H. D. Sch ¨onborn and W. Schulte,RSA Handbuch: Richtlinien f ¨ur die verkehrsrechtliche Sicherung von Arbeitsstellen an Straßen RSA 21, Ausgabe 2021: Handbuch und Kommentar, 2022
2021
-
[45]
Distance, latitude, longitude, and navigation,
C. Heidenreich, “Distance, latitude, longitude, and navigation,”Carto- graphica: The International Journal for Geographic Information and Geovisualization, vol. 13, no. 2, pp. 42–76, 1976
1976
-
[46]
MMDetection3D: OpenMMLab next-generation platform for general 3D object detection,
M. Contributors, “MMDetection3D: OpenMMLab next-generation platform for general 3D object detection,” https://github.com/ open-mmlab/mmdetection3d, 2020
2020
-
[47]
Simultaneous multi- view instance detection with learned geometric soft-constraints,
A. S. Nassar, S. Lef `evre, and J. D. Wegner, “Simultaneous multi- view instance detection with learned geometric soft-constraints,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 6559–6568
2019
-
[48]
Mmdetection: Open mmlab detection toolbox and benchmark,
K. Chen, J. Wang, J. Pang, Y . Cao, Y . Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Xuet al., “Mmdetection: Open mmlab detection toolbox and benchmark,”arXiv preprint arXiv:1906.07155, 2019
Pith/arXiv arXiv 1906
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.