REVIEW 4 major objections 4 minor 43 references
A workflow for generating synthetic LiDAR datasets in simulation environments
T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A simulator workflow can generate synchronized multimodal LiDAR datasets with ground-truth pose, the authors claim, making perception and security research possible without real-world data collection.
desk verdict A competent but modest CoppeliaSim workflow paper whose 'high-fidelity' claim is contradicted by its own limitations section; not a breakthrough, but a reasonable engineering note worth a conditional review. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the coordinate-frame bookkeeping inside the simulator. Each object, sensor, and vehicle is placed in a shared world frame; per frame, the pipeline records the vehicle's six-degree-of-freedom pose (position plus roll, pitch, yaw) in a CSV and reads the point cloud from the LiDAR together with synchronized RGB and depth images. An aggregation script applies the recorded rigid-body transform to each per-frame cloud, so thousands of frames can be merged into a single world-coordinate map, and shared timestamps link all modalities. The workflow's modularity comes from mounting different sensors, including a 16-beam rotating scanner in addition to the default unit, and from reusing imported mesh-based scenes.
What would settle it
Train a standard 3D object detector on the synthetic point clouds and evaluate it on real LiDAR scans of a similar urban scene; a large accuracy drop would show the clean simulated data is not a reliable stand-in for real sensor input. A more direct measurement is to compare per-point range residuals: real sensors exhibit multi-path, mixed-pixel, and missing-return noise that the simulated clouds do not contain.
Extended reading notes
Core claim
On its own terms, the paper's central claim is that a modular simulator workflow can generate synthetic LiDAR datasets that are multimodal, synchronized, and pose-annotated, and that this makes them usable for perception research and as a controlled setting for LiDAR security studies. The workflow combines a time-of-flight LiDAR, a two-dimensional scanner, and RGB and depth cameras on a virtual vehicle in an urban scene; each frame records the vehicle's position and orientation, and point clouds are transformed into a world coordinate frame to build aggregated maps. The paper validates the pipeline by producing large point clouds and corresponding imagery, demonstrates feature matching on the captured frames, and argues the same data can support tests of adversarial point injection and spoofing defenses. The authors conclude that the workflow gives a versatile, reproducible route to high-fidelity synthetic LiDAR data for perception and sensor security.
Load-bearing premise
That the simulator's LiDAR sensor and surface materials produce returns that are realistic enough to stand in for real sensor data, despite the paper's own note that the pipeline adds no range noise, dropouts, multipath effects, weather, or reflectivity variation.
Editorial extensions
If this is right
- Researchers can generate large synchronized multimodal datasets on demand, with exact ground-truth pose for every frame, without field campaigns.
- SLAM, odometry, and sensor-fusion methods can be benchmarked under controlled conditions where the correct answer is known.
- Security studies gain a repeatable setting for injecting adversarial points or spoofed returns and checking whether defenses detect them.
- Because sensors and scene meshes are modular, the same pipeline can produce varied datasets, including reconstructions of real-world locations.
Reading between the lines
- The paper's 'high-fidelity' claim is strongest for geometric structure and temporal synchronization, not raw-signal realism: adding range noise, dropouts, multipath, weather, and material reflectivity would be needed to close the gap the authors themselves identify.
- A natural extension is to use the per-frame ground-truth poses to inject adversarial points at known world locations, turning the pipeline into a generator of labeled attack-defense benchmark sets.
- The workflow's transferability to real perception is not demonstrated by the paper; a paired synthetic-versus-real evaluation would settle how much the missing noise models matter.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper describes a workflow for generating synthetic LiDAR datasets inside the CoppeliaSim simulation environment, integrating a time-of-flight 3D LiDAR, a 2D scanner, RGB and depth cameras, and ground-truth pose logging on a simulated vehicle in an urban scene. The pipeline outputs point clouds in PCD/PLY and metadata in CSV, with scripts for aggregation, visualization, feature matching, and a VLP-16 sensor variant. The authors claim the workflow is a versatile, reproducible framework for generating high-fidelity synthetic LiDAR datasets for perception research and sensor security evaluation, and they discuss limitations and future work including weather effects, noise modeling, and real-world terrain reconstructions.
Significance. If the fidelity and reproducibility claims were substantiated, the workflow would be a useful resource for autonomous-vehicle perception research, SLAM benchmarking, and sensor-security studies. The paper's strengths include the use of a widely available simulator, multimodal synchronized outputs, multiple export formats, a concrete aggregation pipeline, and the inclusion of an additional VLP-16 scanner configuration. However, the central advertised property—fidelity—is asserted rather than demonstrated: the results section contains qualitative screenshots and format descriptions but no quantitative comparison to real LiDAR data, no error analysis, and no downstream task evaluation. The paper's own limitations section explicitly concedes the absence of noise, weather, and realistic reflectivity, which directly undercuts the high-fidelity claim as currently stated.
major comments (4)
- [Section IV and Section V] The central claim that the workflow produces 'high-fidelity synthetic LiDAR datasets' (abstract and Section VI) is not supported by the evidence presented. Section IV reports only qualitative screenshots, file-format descriptions, and visual aggregations; there is no quantitative comparison against real LiDAR scans, no range-error statistics, no point-density validation, and no downstream perception benchmark. Section V explicitly concedes that weather effects are not integrated, that real-world range noise, multi-path detections, dropouts, and mixed-pixel returns are absent, and that intensity is only approximated by uniform material properties. These concessions directly contradict the fidelity claim. The authors should either add quantitative validation (e.g., distributional comparison with a real dataset, noise-injection studies, or a downstream task evaluation) or replace 'high-fidelity' with a more modest characterization such as 'structured synthetic data'.
- [Section V.B] The abstract promises that the study 'demonstrates how synthetic datasets can facilitate the evaluation of defense strategies' and mentions adversarial point injection and spoofing attacks, but Section V.B explicitly defers this work to future research ('Future work...'). The current manuscript does not implement or evaluate any attack or defense on the generated data. The security-related contribution should be repositioned as a planned use case, or the authors should add an actual attack/defense experiment to the pipeline validation.
- [Section IV, aggregation paragraph] The aggregation procedure that underpins the main qualitative results assumes perfect pose accuracy and performs no outlier removal or noise modeling, as the text itself states. Because the merged point-cloud map is used to demonstrate the pipeline's output, the absence of pose-error analysis and outlier treatment is load-bearing for the map-quality and fidelity claims. At minimum, the authors should quantify the sensitivity of the aggregated cloud to pose error, or present the aggregation explicitly as a convenience visualization rather than as a validated product.
- [Section IV.A] The VLP-16 experiment reports the maximum range setting but provides no comparison with the physical Velodyne Puck's beam pattern, angular resolution, intensity response, or noise characteristics. A reader cannot assess whether the simulated scanner faithfully represents the real sensor. The fidelity claim would require at least a statistical comparison of the simulated point cloud with real VLP-16 data, or a documented calibration procedure for the simulator's sensor parameters.
minor comments (4)
- [Figure 1 caption and Section IV] There are typographical errors such as 'genralized' (Figure 1), 'senor' and 'Dept aware perception' (Table III), and 'V oxel' (Section II.C); these should be corrected.
- [Section IV.A] The sensor is referred to as 'Velodyne VPL 16' in the text and Figure 9; the correct product name is Velodyne VLP-16 (Puck).
- [Abstract and References] The abstract contains the placeholder 'this Link' with no URL, and Reference [34] is incomplete ('SVL simulator: brief overview' lacks venue, year, and bibliographic details); these should be completed.
- [Section V.A and Figure 12] The phrase 'a facet of dimensionality' is awkward, and the Figure 12 caption contains a spacing issue ('Norfolk, V A'); also, the acronym 'UAV' is inconsistently rendered as 'UA V' in several places.
Circularity Check
No circular derivation found; the workflow is self-contained and self-citations are contextual, not load-bearing.
full rationale
The paper presents a constructive workflow: it uses CoppeliaSim, its Python API, and vehicle-mounted sensors to capture and store synthetic LiDAR, RGB, depth, and pose data. There is no equation whose output is fitted to the target claim, and no predicted quantity is obtained from a parameter estimated on the same data. The central claim is that the workflow generates 'high-fidelity synthetic LiDAR datasets,' but that claim is asserted rather than derived from any fitted model or imported uniqueness theorem. The self-citations at refs. [37] and [39] are contextual: [37] is cited to explain that simulation environment instances were reused from prior UAV experiments, and [39] is cited to note that weather effects are not integrated even though the simulator supports them. Neither citation is load-bearing for the workflow's operation, which is described concretely through file formats, viewports, aggregation scripts, and feature-matching demos. The paper's Section V explicitly states that weather effects are not integrated, that real sensors suffer range noise, multi-path detections, missed dropouts, and mixed pixel returns, and that algorithms trained on clean simulated data can overfit. This undercuts the 'high-fidelity' descriptor, but that is a correctness or evidence concern, not a circularity: the workflow does not define its outputs as its inputs, nor does it rename a fitted result as a prediction. Accordingly, no circularity steps are identified.
Assumptions & free parameters
assumptions (3)
- domain assumption CoppeliaSim's simulated LiDAR sensors produce geometrically accurate range measurements.
- domain assumption Ground-truth poses from the simulator are exact.
- domain assumption Object material properties in the scene represent realistic reflectivity.
Cite this review
Pith. "Pith review of A workflow for generating synthetic LiDAR datasets in simulation environments." pith.science (2026). https://pith.science/paper/UGL6XOCO
@misc{pith2026250617378,
author = {Pith},
title = {Pith review of: A workflow for generating synthetic LiDAR datasets in simulation environments},
year = {2026},
howpublished = {\url{https://pith.science/paper/UGL6XOCO}},
note = {Machine review of arXiv:2506.17378}
}
read the original abstract
This paper presents a simulation workflow for generating synthetic LiDAR datasets to support autonomous vehicle perception, robotics research, and sensor security analysis. Leveraging the CoppeliaSim simulation environment and its Python API, we integrate time-of-flight LiDAR, image sensors, and two dimensional scanners onto a simulated vehicle platform operating within an urban scenario. The workflow automates data capture, storage, and annotation across multiple formats (PCD, PLY, CSV), producing synchronized multimodal datasets with ground truth pose information. We validate the pipeline by generating large-scale point clouds and corresponding RGB and depth imagery. The study examines potential security vulnerabilities in LiDAR data, such as adversarial point injection and spoofing attacks, and demonstrates how synthetic datasets can facilitate the evaluation of defense strategies. Finally, limitations related to environmental realism, sensor noise modeling, and computational scalability are discussed, and future research directions, such as incorporating weather effects, real-world terrain models, and advanced scanner configurations, are proposed. The workflow provides a versatile, reproducible framework for generating high-fidelity synthetic LiDAR datasets to advance perception research and strengthen sensor security in autonomous systems. Documentation and examples accompany this framework; samples of animated cloud returns and image sensor data can be found at this Link.
Figures
Figures from the paper (9 more)
Reference graph
Works this paper leans on
-
[1]
D. Liang et al., ”Evolution of laser technology for automotive LiDAR, an industrial viewpoint,” Nat Commun, vol. 15, no. 1, p. 7660, Sep 3 2024, doi:https://10.1038/s41467-024-51975-6
-
[2]
M. Hu, Y . Pang, and L. Gao, ”Advances in Silicon-Based In- tegrated Lidar,” Sensors (Basel), vol. 23, no. 13, Jun 26 2023, doi:https://10.3390/s23135920
-
[3]
M. A. Wulder et al., ”Lidar sampling for large-area forest characteriza- tion: A review,” Remote Sensing of Environment, vol. 121, pp. 196-209, 2012/06/01/ 2012, doi:https://https://doi.org/10.1016/j.rse.2012.02.001
-
[4]
M. Dassot, T. Constant, and M. Fournier, ”The use of terrestrial LiDAR technology in forest science: application fields, benefits and challenges,” Annals of Forest Science, vol. 68, no. 5, pp. 959-974, 2011/08/01 2011, doi:https://10.1007/s13595-011-0102-2
-
[5]
S. Chen, S. Song, Y . Wang, H. Pan, F. Li, and Y . Chen, ”Dual- Wavelength LiDAR with a Single-Pixel Detector Based on the Time- Stretched Method,” Sensors (Basel), vol. 24, no. 17, Sep 4 2024, doi:https://10.3390/s24175741
-
[6]
P. Rodr ´ıguez-Gonz´alvez, B. Jim ´enez Fern ´andez-Palacios, ´A. L. Mu ˜noz- Nieto, P. Arias-Sanchez, and D. Gonzalez-Aguilera, ”Mobile LiDAR System: New Possibilities for the Documentation and Dissemination of Large Cultural Heritage Sites,” Remote Sensing, vol. 9, no. 3, p. 189, 2017, doi:https://doi.org/10.3390/rs9030189
-
[7]
Y . Li, L. Zhao, Y . Chen, N. Zhang, H. Fan, and Z. Zhang, ”3D LiDAR and multi-technology collaboration for preservation of built heritage in China: A review,” International Journal of Applied Earth Observation and Geoinformation, vol. 116, p. 103156, 2023/02/01/ 2023, doi:https://doi.org/10.1016/j.jag.2022.103156
arXiv 2023
-
[8]
H. Prumers, C. J. Betancourt, J. Iriarte, M. Robinson, and M. Schaich, ”Lidar reveals pre-Hispanic low-density urbanism in the Bo- livian Amazon,” Nature, vol. 606, no. 7913, pp. 325-328, Jun 2022, doi:https://10.1038/s41586-022-04780-4
Show all 43 references
-
[9]
Ortega, J
S. Ortega, J. M. Santana, J. Wendel, A. Trujillo, and S. M. Murshed, ”Generating 3D City Models from Open LiDAR Point Clouds: Ad- vancing Towards Smart City Applications,” in Open Source Geospatial Science for Urban Studies: The Value of Open Geospatial Data, A. Mobasheri Ed. ...
2021
-
[10]
Garnett and M
R. Garnett and M. D. Adams, ”LiDAR—A Technology to Assist with Smart Cities and Climate Change Resilience: A Case Study in an Urban Metropolis,” ISPRS International Journal of Geo-Information, vol. 7, no. 5, p. 161, 2018, doi:https://doi.org/10.3390/ijgi7050161
2018 doi
-
[11]
Zhou, ”A Review of LiDAR sensor Technologies for Perception in Automated Driving,” 2022
J. Zhou, ”A Review of LiDAR sensor Technologies for Perception in Automated Driving,” 2022
2022
-
[12]
Kaartinen, K
E. Kaartinen, K. Dunphy, and A. Sadhu, ”LiDAR-Based Structural Health Monitoring: Applications in Civil Infrastructure Systems,” Sensors (Basel), vol. 22, no. 12, Jun 18 2022, doi:https://10.3390/s22124610
2022 doi
-
[13]
Cao et al., ”Adversarial Objects Against LiDAR-Based Autonomous Driving Systems,” 2019, http://doi.org/https://doi.org/10.48550/arXiv.1907.05418
Y . Cao et al., ”Adversarial Objects Against LiDAR-Based Autonomous Driving Systems,” 2019, http://doi.org/https://doi.org/10.48550/arXiv.1907.05418
-
[14]
J. Wang, F. Li, X. Zhang, and H. Sun, ”Adversarial Ob- stacle Generation Against LiDAR-Based 3D Object Detection,” IEEE Transactions on Multimedia, vol. 26, pp. 2686-2699, 2024, doi:https://10.1109/tmm.2023.3302018
2024
-
[15]
T. Sato et al., ”On the Realism of LiDAR Spoofing Attacks against Autonomous Driving Vehicle at High Speed and Long Distance,” presented at the Proceedings 2025 Network and Distributed System Security Symposium, 2025. http://doi.org/10.14722/ndss.2025.230628
2025
-
[16]
Marazzi, S
M. Marazzi, S. Longari, M. Carminati, and S. Zanero, ”Securing Lidar Communication through Watermark-based Tampering Detection,” presented at the Proceedings Symposium on Vehicle Security & Privacy,
-
[17]
Hamdi, S
A. Hamdi, S. Rojas, A. Thabet, and B. Ghanem, ”AdvPC: Trans- ferable Adversarial Perturbations on 3D Point Clouds,” 2020. http://doi.org/doi.org/10.1007/978-3-030-58610-2 15
2020 doi
-
[18]
J. Sun, Y . Cao, Q. A. Chen, and Z. M. Mao, ”Towards Robust LiDAR- based Perception in Autonomous Driving: General Black-box Adver- sarial Sensor Attack and Countermeasures,” presented at the USENIX Security Symposium, August 12–14, 2020, 2020. [Online]. Available: https://www...
2020
- [19]
-
[20]
Changalvala and H
R. Changalvala and H. Malik, ”LiDAR Data Integrity Verification for Autonomous Vehicle,” IEEE Access, vol. 7, pp. 138018-138031, 2019, doi:https://10.1109/ACCESS.2019.2943207
2019
-
[21]
Y . Cao et al., ”Invisible for both Camera and LiDAR: Secu- rity of Multi-Sensor Fusion based Perception in Autonomous Driv- ing Under Physical-World Attacks,” in 2021 IEEE Symposium on Security and Privacy (SP), 24-27 May 2021 2021, pp. 176-194, http://doi.org/10.1109/SP40001...
2021
-
[22]
Zhang et al., ”Toward Robust Robot 3-D Perception in Urban Environments: The UT Campus Object Dataset,” IEEE Transactions on Robotics, vol
A. Zhang et al., ”Toward Robust Robot 3-D Perception in Urban Environments: The UT Campus Object Dataset,” IEEE Transactions on Robotics, vol. 40, pp. 3322-3340, 2024, doi:https://10.1109/tro.2024.3400831
2024
-
[23]
Attila and N
J. Attila and N. Hajnalka, ”Detecting Military Historical Objects by LiDAR Data,” 2015, doi:https://10.32565/aarms.2015.2.8
2015 doi
-
[24]
Paniego, E
S. Paniego, E. Shinohara, and J. Ca ˜nas, ”Autonomous driving in traffic with end-to-end vision-based deep learning,” Neurocomputing, vol. 594, p. 127874, 2024/08/14/ 2024, doi:https://https://doi.org/10.1016/j.neucom.2024.127874
2024
-
[25]
Gupta and A
M. Gupta and A. Mishra, ”A systematic review of deep learning based image segmentation to detect polyp,” Artificial Intelligence Review, vol. 57, no. 1, p. 7, 2024/01/05 2024, doi:https://10.1007/s10462-023-10621- 1
2024 doi
-
[26]
J. Beltr ´an et al., ”A Method for Synthetic LiDAR Generation to Create Annotated Datasets for Autonomous Vehicles Perception,” in 2019 IEEE Intelligent Transportation Systems Conference (ITSC), 27-30 Oct. 2019 2019, pp. 1091-1096, http://doi.org/10.1109/ITSC.2019.8917176
2019
-
[27]
F. Wang, Y . Zhuang, H. Gu, and H. Hu, ”Automatic Generation of Syn- thetic LiDAR Point Clouds for 3-D Data Analysis,” IEEE Transactions on Instrumentation and Measurement, vol. 68, no. 7, pp. 2671-2673, 2019, doi:https://10.1109/tim.2019.2906416
2019
-
[28]
Dosovitskiy, G
A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun, ”CARLA: An open urban driving simulator,” in Conference on robot learning, 2017: PMLR, pp. 1-16
2017
-
[29]
Chen et al., ”STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point Cloud Dataset,” 2022, http://doi.org/10.48550/arXiv.2203.09065
M. Chen et al., ”STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point Cloud Dataset,” 2022, http://doi.org/10.48550/arXiv.2203.09065
2022 doi
-
[30]
X. Yue, B. Wu, S. A. Seshia, K. Keutzer, and A. L. Sangiovanni- Vincentelli, ”A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Driving,” presented at the Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, Yokohama, Japan,
2018
-
[31]
Montalvo, P
J. Montalvo, P. Carballeira, and G.-M. ´A, ”Synthmanticlidar: A Synthetic Dataset For Semantic Segmentation On Lidar Imaging,” in 2024 IEEE International Conference on Image Processing (ICIP), 27-30 Oct. 2024 2024, pp. 137-143, http://doi.org/10.1109/ICIP51287.2024.10648055
2024
-
[32]
B. Hurl, K. Czarnecki, and S. Waslander, ”Precise Synthetic Image and LiDAR (PreSIL) Dataset for Autonomous Vehicle Perception,” in 2019 IEEE Intelligent Vehicles Symposium (IV), 9-12 June 2019 2019, pp. 2522-2529, http://doi.org/10.1109/IVS.2019.8813809
2019
-
[33]
Wisultschew, R
C. Wisultschew, R. Hern ´andez, C. Pastor, and J. Portilla, ”Syn- thetic LiDAR-Labeled Data Set Generation to Train Deep Neural Networks for Object Classification in IoT at the Edge,” IEEE In- ternet of Things Journal, vol. 9, no. 24, pp. 24812-24821, 2022, doi:https://10.1109...
2022
-
[34]
Wang, ”SVL simulator: brief overview
G. Wang, ”SVL simulator: brief overview
-
[35]
Rohmer, S
E. Rohmer, S. P. N. Singh, and M. Freese, ”CoppeliaSim (formerly V- REP): a Versatile and Scalable Robot Simulation Framework,” presented at the IEEE/RSJ International Conference on Intelligent Robots and Sys- tems., Tokyo, Japan., 2013. http://doi.org/10.1109/IROS.2013.6696520
2013
-
[36]
”Bullet Real-Time Physics Simulation.” https://pybullet.org/wordpress/ (accessed May, 2025)
2025
-
[37]
Phadke, F
A. Phadke, F. A. Medrano, T. Chu, C. N. Sekharan, and M. J. Starek, ”Modeling Wind and Obstacle Disturbances for Effective Performance Observations and Analysis of Resilience in UA V Swarms,” Aerospace, vol. 11, no. 3, 2024, doi:https://10.3390/aerospace11030237
2024 doi
-
[38]
Available: https://www.mapix.com/lidar-scanner-sensors/velodyne/velodyne-puck- vlp16/
”Mapix technologies-Velodyne Puck (VLP-16),” ed. Available: https://www.mapix.com/lidar-scanner-sensors/velodyne/velodyne-puck- vlp16/
-
[39]
Phadke, F
A. Phadke, F. A. Medrano, C. N. Sekharan, and T. Chu, ”Design- ing UA V Swarm Experiments: A Simulator Selection and Exper- iment Design Process,” Sensors, vol. 23, no. 17, p. 7359, 2023, doi:https://10.3390/s23177359
2023 doi
-
[40]
X. Fen, L. ZhengXi, and Y . Kui, ”The design and implementation of an autonomous campus patrol robot,” in 2007 IEEE International Conference on Robotics and Biomimetics (ROBIO), 15-18 Dec. 2007 2007, pp. 250-255, http://doi.org/10.1109/ROBIO.2007.4522169
2007
-
[41]
D. Bose, M. Karthi, C. S. Meera, Y . Monika, and D. K. and Saini, ”Review of Autonomous Campus and Tour Guiding Robots with Navigation Techniques,” Australian Journal of Mechan- ical Engineering, vol. 21, no. 5, pp. 1580-1590, 2023/10/20 2023, doi:https://10.1080/14484846.2021.2023266
2023
-
[2018]
Available: https://doi.org/10.1145/3206025.3206080, http://doi.org/10.1145/3206025.3206080
[Online]. Available: https://doi.org/10.1145/3206025.3206080, http://doi.org/10.1145/3206025.3206080
-
[2024]
http://doi.org/10.14722/vehiclesec.2024.23055
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.