REVIEW 3 major objections 6 minor 30 references
FogDrive is a synthetic multi-modal driving dataset with paired clean and foggy scenes; its benchmarks claim mixing fog densities during 3D training improves detection geometry, and fog-trained 2D detectors outperform dehaze-then-detect.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 20:16 UTC pith:3C65SR4D
load-bearing objection Solid dataset contribution; the main empirical claim about mixed-fog training rests on test-set checkpoint selection and an unfinished row, so it's not yet established. the 3 major comments →
FogDrive: A Multi-Modal Synthetic Driving Dataset for Perception under Graded Fog
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central assertion is that FogDrive fills a gap no existing dataset covers: simultaneously providing paired clean and foggy observations, three calibrated fog densities, a multi-modal sensor suite, and consistent 2D/3D annotations across identical scenes. Built in CARLA, each scene is released clean plus light/moderate/dense fog, with camera fog from Koschmieder's model and LiDAR fog from Beer-Lambert attenuation; radar is left clean. On this benchmark, TransFusion experiments show mixed-density training beats single-density training for detection rate at equal data budget, and scaling the mix improves Mean TP and translation error (about 4 cm under moderate fog); BEVFusion result
What carries the argument
The key machinery is the four-variant paired-scene design: every scene is generated once and then degraded independently per modality using physics-based fog models—Koschmieder's law for camera transmission t(x)=exp(-βd(x)) and Beer-Lambert for LiDAR returns—at three extinction coefficients corresponding to 160, 100, and 50 m visibility. Because geometry and annotations are identical across the four variants, any performance difference is attributable to fog density alone. The semantic-LiDAR visibility filter that produces 2D boxes from 3D boxes is the main authored component; the audit quantifies its precision (95.1%) and recall (over 99% within 40 m).
Load-bearing premise
All benchmark conclusions rest on the assumption that the offline fog simulations—Koschmieder camera fog, Beer-Lambert LiDAR fog, and the decision to leave radar clean—faithfully reproduce real fog, a premise the paper does not test against real foggy multi-modal data.
What would settle it
A concrete check: take a LiDAR/camera detector trained on FogDrive's mixed-density fog and evaluate it on real foggy multi-modal data; if the mixed-density training advantage over single-density training (the ~4 cm translation-error gain under moderate fog) fails to appear, the central 3D claim is falsified for real-world transfer.
If this is right
- If the benchmark is accepted, researchers can directly compare defog-then-detect vs train-on-fog under matched geometry, which was previously impossible.
- The 3D result implies practitioners should mix fog densities in training even with a fixed data budget; detection rate improves at no extra data cost.
- Scaling the mix specifically improves localization quality, so trajectory prediction and motion planning stand to benefit more than raw detection from larger multi-density training sets.
- The 2D result implies dehazer selection should be based on downstream detection performance, not PSNR/SSIM.
- The radar-invariance argument implies FogDrive's radar channel can serve as a clean reference modality in fog, isolating camera/LiDAR degradation.
Where Pith is reading between the lines
- Because FogDrive does not validate its fog simulations against real foggy multi-modal recordings, the transferability of its training conclusions to physical fog remains an open question; a direct test would be to fine-tune on real foggy LiDAR or image data and compare mixed-density gains.
- The PSNR/SSIM finding suggests a need for detection-aware image-quality metrics; one could test whether simple contrast-based or task-based metrics predict detection better on the same paired data.
- The mixed-density training idea may extend beyond fog to other continuous degradations (rain, snow, dusk), since the mechanism is diversity of the degradation distribution rather than fog-specific.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. FogDrive is a synthetic CARLA-based multi-modal driving dataset with 666 scenes (~133k frames), four RGB/depth/segmentation cameras, LiDAR and semantic LiDAR, and front radar. Each scene is released in a clean variant plus three fog levels (≈160 m, 100 m, 50 m visibility) generated with Koschmieder-based camera fog and Beer–Lambert LiDAR attenuation. The paper claims this is the first dataset to combine paired clean/foggy observations, multiple calibrated fog densities, a complete multi-modal sensor suite, and consistent 2D/3D annotations, and it reports a semantic-segmentation-based annotation audit (95.1% precision, >99% recall within 40 m). Two benchmarking experiments are presented: Experiment 1 compares single-density versus mixed-density training for TransFusion and BEVFusion 3D detectors; Experiment 2 compares dehaze-then-detect versus train-on-fog for YOLOv8-m, finding that fog-trained detectors approach the clean upper bound and that PSNR/SSIM do not predict detection performance. The paper concludes that mixing fog densities at training time improves 3D detection geometry, and that direct fog training beats defog-then-detect.
Significance. If the empirical claims hold, FogDrive would be a valuable controlled instrument for robustness research: it provides matched multi-modal observations across graded fog, which existing real-world and synthetic benchmarks do not simultaneously offer. The explicit use of physics-based fog models, paired clean/foggy structure, three calibrated densities, consistent annotations, and a pre-registered annotation audit are genuine strengths. The open release of the generation framework and audit scripts, if carried out, would further increase reproducibility. The data-centric insights — especially that mixed-density training can improve detection geometry and that image-quality metrics mislead on dehazer selection — are interesting and actionable. However, the current support for the 3D mixing claim is incomplete: the headline conclusion rests on a single detector, a TODO row for the fusion model, and a checkpoint-selection protocol that optimizes on the test set.
major comments (3)
- [§4.1, Tables 6–8] The evaluation protocol selects the best-FDS checkpoint per (training, test) pair on the test subset itself: “we report the checkpoint that maximises the composite FogDrive Detection Score (FDS) on that test subset across the training schedule.” This is test-set fitting and can systematically inflate apparent differences, especially for training regimes with noisier training curves. The central claim that mixed-density training improves detection/geometry is therefore not established by the reported numbers. Re-evaluation with checkpoints selected on the validation split, with epoch numbers and multiple seeds, is required before the headline conclusion can be accepted.
- [§4.1, Tables 6–8 and Abstract] The BEVFusion Mixed-L row is explicitly reported as TODO in all three tables, and §4.1 states “the BEVFusion picture is pending” and “Does BEVFusion show the same split? (To be filled when BEVFusion Mixed-L training completes).” Yet the abstract and conclusion assert as established that “mixing multi-density fog during training tightens 3D bounding-box geometry.” The only complete evidence comes from TransFusion, and the already-collected BEVFusion rows actually show the opposite ATE pattern (Light sweeping ATE). The manuscript must either complete the BEVFusion run or explicitly scope the mixing claim to TransFusion as a preliminary, single-detector observation.
- [§4.1, Tables 6–8] No error bars or multi-seed statistics are reported. Several of the headline margins are very small — Mixed-L’s MeanTP margins over the next best are +0.0077, +0.0079, and +0.0003, and the ATE margins are +0.0016 to +0.0192. With a single run per condition and test-set checkpoint selection, these differences are not distinguishable from noise. Additionally, the table note “TransFusion Mixed-L row at epoch 23” appears inconsistent with the stated best-FDS selection rule; the paper should clarify whether this row uses a fixed epoch or a selected checkpoint, since this affects the interpretation of every Mixed-L cell.
minor comments (6)
- [Abstract / §3.6] The abstract says “660 scenes” while §3.6 and Table 1 report 666 scenes and 132,996 frames. The number should be made consistent.
- [§3.2 / §5] The paper does not validate the synthetic fog against any real foggy multi-modal data (e.g., DENSE/STF). Since the benchmark conclusions may be sensitive to the fidelity of the Koschmieder/Beer–Lambert simulations, adding a qualitative or quantitative comparison with real fog, or at least an explicit discussion of the expected sim-to-real gap, would strengthen the paper.
- [§4.2] Experiment 2 also reports single-run results without variance estimates. Given that the claim “PSNR/SSIM do not predict detection” is of practical importance, a small multi-seed analysis would make it more robust. The current evidence is strong but not statistically quantified.
- [§4.1] TransFusion is trained for 50 epochs while BEVFusion is trained for 15. The comparison between detectors is fine as a benchmark, but the difference in schedules and the absence of a learning-rate/schedule sweep should be acknowledged as a potential confound when making architecture-level statements.
- [§3.7 / §5] The annotation caveats are honestly reported, but the sentence “the practical limitation is that the cameras must remain close to the LiDAR … will be mis-annotated” is somewhat hidden. Suggest moving the quantified per-camera numbers (§3.7) earlier or into the main limitations list, since they matter for downstream users selecting cameras.
- [Conclusion] The conclusion says the dataset will be released “upon acceptance,” while the abstract and §3.4 imply it is “fully open-sourced” now. Please clarify the release status.
Circularity Check
No definitional circularity; Experiment 1's test-set checkpoint selection is a self-referential validity threat but not a circular derivation.
full rationale
FogDrive is a dataset and benchmark paper rather than a derivation chain. The fog densities are generation inputs: alpha values are mapped to visibility via V≈ln(20)/alpha, which is the standard 5%-contrast definition, and the extinction/backscatter settings are not fitted to any experimental conclusion. The annotation audit uses a separate semantic-segmentation stream whose failure modes are independent of the LiDAR visibility filter, making it a consistency check rather than a circular validation. The 2D boxes are projections of CARLA's 3D boxes, and the paper explicitly says the 3D extent and projection are 'correct by construction'—a transparency statement, not a prediction. Experiments 1 and 2 are empirical comparisons; no parameter of the headline conclusions is defined in terms of those conclusions. There is no load-bearing self-citation chain and no imported uniqueness or ansatz from the authors' prior work. The only circularity-adjacent element is the evaluation protocol in §4.1: checkpoints are selected by maximizing FDS on the test subset and then scored on that same subset, which makes the reported numbers self-referential and can inflate the apparent mixed-density benefit. This is a statistical validity threat rather than a reduction of the result to its inputs, so under the hard rules it does not count as a circular step. The incomplete BEVFusion Mixed-L row and lack of error bars are additional correctness risks, not circularity. Score 2 reflects the minor self-referential protocol concern; the dataset construction and qualitative benchmark findings remain independent of their inputs.
Axiom & Free-Parameter Ledger
free parameters (7)
- fog extinction coefficient α (dense) =
0.060 m^-1
- fog extinction coefficient α (moderate) =
0.030 m^-1
- fog extinction coefficient α (light) =
0.018 m^-1
- LiDAR backscatter retention γ =
1e-6
- LiDAR fog noise scale =
10
- min_detect points for annotation visibility filter =
5
- max_dist for annotation =
100 m
axioms (6)
- domain assumption Koschmieder's model with transmission t=exp(-βd) faithfully models real camera fog.
- domain assumption Beer-Lambert attenuation with round-trip exp(-2βr) faithfully models real LiDAR fog.
- domain assumption 77 GHz automotive radar is essentially unaffected by fog, so releasing clean radar is correct.
- domain assumption CARLA's 3D bounding boxes and semantic-segmentation camera are ground-truth correct by construction.
- domain assumption Semantic-segmentation camera errors are orthogonal to the LiDAR visibility filter's errors.
- domain assumption CARLA Traffic Manager driving behavior is representative enough to draw data-centric training conclusions.
read the original abstract
Perception under adverse weather remains a critical bottleneck for reliable autonomous driving, yet existing benchmarks lack the systematic multi-modal alignments needed to evaluate robust sensor fusion. Real-world weather datasets suffer from uncontrolled collection and single-level, uncalibrated conditions, while synthetic alternatives either target camera-only restoration or lack the paired clean-and-foggy structure needed to benchmark "defog-then-detect" pipelines. We present FogDrive, a rigorously calibrated, multi-modal autonomous-driving dataset bridging data-centric engineering and robust machine learning. Built with the CARLA simulator, FogDrive contains 660 scenes (~133k fully annotated frames, 50:50 day/night) across four synchronized cameras (RGB, depth, semantic segmentation), a LiDAR and semantic-LiDAR pair, and front radar. Physically consistent fog is modeled independently on camera channels (Koschmieder model) and LiDAR channels (Beer-Lambert law) at three calibrated visibility densities (160m, 100m, 50m). Every scene ships in four matched variants (clean plus three graded fog levels) with cross-calibrated 2D and 3D bounding boxes. A semantic-segmentation-based quality audit over 8k images validates annotations at 95.1% precision and over 99% recall for vehicles within 40m. We establish baseline benchmarks with state-of-the-art architectures (TransFusion, BEVFusion, YOLOv8-m) across two paradigms: 3D multi-modal fusion and 2D image restoration. These yield critical data-centric insights: mixing multi-density fog during training tightens 3D bounding-box geometry without added data-scaling cost, while in 2D pipelines image-quality metrics (PSNR, SSIM) prove poor predictors of downstream detection performance. FogDrive will be fully open-sourced alongside our data-generation framework to accelerate robust, multi-modal research.
Figures
Reference graph
Works this paper leans on
-
[1]
CARLA-2DBBox: Vehicle 2D bounding box annotation module for the CARLA simulator.https://mukhlasadib.github.io/CARLA-2DBBox/,
Mukhlas Adib. CARLA-2DBBox: Vehicle 2D bounding box annotation module for the CARLA simulator.https://mukhlasadib.github.io/CARLA-2DBBox/,
-
[2]
TransFusion: Robust LiDAR-camera fusion for 3D object detection with transformers
Xuyang Bai, Zeyu Hu, Xinge Zhu, Qingqiu Huang, Yilun Chen, Hongbo Fu, and Chiew-Lan Tai. TransFusion: Robust LiDAR-camera fusion for 3D object detection with transformers. InProc. IEEE/CVF Conf. Computer Vision and Pattern Recogni- tion (CVPR), 2022
2022
-
[3]
Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather
Mario Bijelic, Tobias Gruber, Fahim Mannan, Florian Kraus, Werner Ritter, Klaus Dietmayer, and Felix Heide. Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather. InProc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), 2020
2020
-
[4]
Yoon, Yuchen Wu, Andrew Z
Keenan Burnett, David J. Yoon, Yuchen Wu, Andrew Z. Li, Haowei Zhang, Shichen Lu, Jingxing Qian, Wei-Kang Tseng, Andrew Lambert, Keith Y . K. Leung, Angela P. Schoellig, and Timothy D. Barfoot. Boreas: A multi-season autonomous driving dataset.International Journal of Robotics Research (IJRR), 42(1–2):33–42, 2023
2023
-
[5]
Virtual KITTI 2.arXiv preprint arXiv:2001.10773, 2020
Yohann Cabon, Naila Murray, and Martin Humenberger. Virtual KITTI 2.arXiv preprint arXiv:2001.10773, 2020. PANW AR: FOGDRIVE21
Pith/arXiv arXiv 2001
-
[6]
Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom
Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuScenes: A multimodal dataset for autonomous driving. InProc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), 2020
2020
-
[7]
KITTI-CARLA: A KITTI-like dataset generated by CARLA simulator
Jean-Emmanuel Deschaud. KITTI-CARLA: A KITTI-like dataset generated by CARLA simulator. InProc. Int. Conf. 3D Vision (3DV), 2021. arXiv:2109.00892
Pith/arXiv arXiv 2021
-
[8]
CARLA: An open urban driving simulator
Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio López, and Vladlen Koltun. CARLA: An open urban driving simulator. InProc. Conf. Robot Learning (CoRL), 2017
2017
-
[9]
Jörg Gamerdinger, Sven Teufel, Patrick Schulz, Stephan Amann, Jan-Patrick Kirchner, and Oliver Bringmann. SCOPE: A synthetic multi-modal dataset for collective percep- tion including physical-correct weather conditions.arXiv preprint arXiv:2408.03065, 2024
Pith/arXiv arXiv 2024
-
[10]
Are we ready for autonomous driv- ing? the KITTI vision benchmark suite
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driv- ing? the KITTI vision benchmark suite. InProc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2012
2012
-
[11]
Fog simula- tion on real LiDAR point clouds for 3D object detection in adverse weather
Martin Hahner, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Fog simula- tion on real LiDAR point clouds for 3D object detection in adverse weather. InProc. IEEE/CVF Int. Conf. Computer Vision (ICCV), 2021. LiDAR fog simulation (Beer– Lambert) used by FogDrive
2021
-
[12]
Single image haze removal using dark chan- nel prior.IEEE Trans
Kaiming He, Jian Sun, and Xiaoou Tang. Single image haze removal using dark chan- nel prior.IEEE Trans. Pattern Analysis and Machine Intelligence (TPAMI), 33(12): 2341–2353, 2011. Originally CVPR 2009
2011
-
[13]
Junjie Huang, Guan Huang, Zheng Zhu, Yun Ye, and Dalong Du. BEVDet: High- performance multi-camera 3D object detection in bird-eye-view.arXiv preprint arXiv:2112.11790, 2021
Pith/arXiv arXiv 2021
-
[14]
YOLO by Ultralytics (YOLOv8)
Glenn Jocher, Ayush Chaurasia, and Jing Qiu. YOLO by Ultralytics (YOLOv8). https://github.com/ultralytics/ultralytics, 2023. Software, AGPL-3.0
2023
-
[15]
Mateus Karvat and Sidney Givigi. Adver-City: Open-source multi-modal dataset for collaborative perception under adverse weather conditions.arXiv preprint arXiv:2410.06380, 2024
Pith/arXiv arXiv 2024
-
[16]
AOD-Net: All- in-one dehazing network
Boyi Li, Xiulian Peng, Zhangyang Wang, Jizheng Xu, and Dan Feng. AOD-Net: All- in-one dehazing network. InProc. IEEE/CVF Int. Conf. Computer Vision (ICCV), 2017
2017
-
[17]
Swin Transformer: Hierarchical vision transformer using shifted win- dows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin Transformer: Hierarchical vision transformer using shifted win- dows. InProc. IEEE/CVF Int. Conf. Computer Vision (ICCV), 2021
2021
-
[18]
Rus, and Song Han
Zhijian Liu, Haotian Tang, Alexander Amini, Xinyu Yang, Huizi Mao, Daniela L. Rus, and Song Han. BEVFusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation. InProc. IEEE Int. Conf. Robotics and Automation (ICRA), 2023. 22PANW AR: FOGDRIVE
2023
-
[19]
Lift, Splat, Shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3D
Jonah Philion and Sanja Fidler. Lift, Splat, Shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3D. InProc. European Conf. Computer Vision (ECCV), 2020
2020
-
[20]
Canadian adverse driving conditions dataset.International Journal of Robotics Research (IJRR), 40(4–5):681–690, 2021
Matthew Pitropov, Danson Evan Garcia, Jason Rebello, Michael Smart, Carlos Wang, Krzysztof Czarnecki, and Steven Waslander. Canadian adverse driving conditions dataset.International Journal of Robotics Research (IJRR), 40(4–5):681–690, 2021
2021
-
[21]
Model adaptation with synthetic and real data for semantic dense foggy scene understanding.Interna- tional Journal of Computer Vision (IJCV), 2018
Christos Sakaridis, Dengxin Dai, Simon Hecker, and Luc Van Gool. Model adaptation with synthetic and real data for semantic dense foggy scene understanding.Interna- tional Journal of Computer Vision (IJCV), 2018. Camera fog simulation pipeline used by FogDrive (Koschmieder’s law)
2018
-
[22]
Semantic foggy scene under- standing with synthetic data.International Journal of Computer Vision (IJCV), 126 (9):973–992, 2018
Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Semantic foggy scene under- standing with synthetic data.International Journal of Computer Vision (IJCV), 126 (9):973–992, 2018. Introduces Foggy Cityscapes (synthetic) and Foggy Driving (real evaluation set)
2018
-
[23]
ACDC: The adverse condi- tions dataset with correspondences for semantic driving scene understanding
Christos Sakaridis, Dengxin Dai, and Luc Van Gool. ACDC: The adverse condi- tions dataset with correspondences for semantic driving scene understanding. InProc. IEEE/CVF Int. Conf. Computer Vision (ICCV), 2021
2021
-
[24]
Vision transformers for single image dehazing.IEEE Trans
Yuda Song, Zhuqing He, Hui Qian, and Xin Du. Vision transformers for single image dehazing.IEEE Trans. Image Processing (TIP), 32:1927–1941, 2023. In- troduces DehazeFormer; official codebase athttps://github.com/IDKiro/ DehazeFormer
1927
-
[25]
Scalability in perception for autonomous driving: Waymo open dataset
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. InProc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), 2020
2020
-
[26]
SHIFT: A synthetic driving dataset for continuous multi-task domain adaptation
Tao Sun, Mattia Segu, Janis Postels, Yuxuan Wang, Luc Van Gool, Bernt Schiele, Federico Tombari, and Fisher Yu. SHIFT: A synthetic driving dataset for continuous multi-task domain adaptation. InProc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), 2022
2022
-
[27]
AOD-net by PyTorch.https://github.com/weberwcwei/ AODnet-by-pytorch, 2018
Chenwei Wei. AOD-net by PyTorch.https://github.com/weberwcwei/ AODnet-by-pytorch, 2018. Community PyTorch reproduction of AOD-Net [16] with pretrained weights
2018
-
[28]
Yiming Xie, Henglu Wei, Zhenyi Liu, Xiaoyu Wang, and Xiangyang Ji. SynFog: A photo-realistic synthetic fog dataset based on end-to-end imaging simulation for ad- vancing real-world defogging in autonomous driving. InProc. IEEE/CVF Conf. Com- puter Vision and Pattern Recognition (CVPR), 2024. arXiv:2403.17094
Pith/arXiv arXiv 2024
-
[29]
supplementary § Dn
Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. BDD100K: A diverse driving dataset for het- erogeneous multitask learning. InProc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), 2020. PANW AR: FOGDRIVE23 FogDrive: Supplementary Material This supplementary material accompani...
2020
-
[2020]
FogDrive uses a locally modified version that lets four cameras of different orientation share a single semantic LiDAR for the visibility test
Open-source community tool, MIT licence. FogDrive uses a locally modified version that lets four cameras of different orientation share a single semantic LiDAR for the visibility test
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.