REVIEW 5 major objections 5 minor 23 references
Multi-Sensor Alignment for Weather Simulations
T0 review · 5 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read Weather simulations that ignore sensor alignment flatter 3D detection models.
desk verdict Focused, honest empirical paper: the rain/snow particle alignment is a real and checkable contribution, but the fog validation is circular and the headline claim about 'overly optimistic' unaligned simulations is only directly tested for rain/snow. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a shared 3D world-coordinate system for weather particles. For fog, ReDAM tunes the LiDAR fog model's parameters by minimizing KS-statistics against a real aligned fog dataset, while camera fog severity is matched by sampling style references from that same dataset at the same fog level. For rain and snow, particles are sampled in a common world volume, transformed via world-to-LiDAR and world-to-camera matrices, and then rendered independently in each sensor. This shared coordinate representation is what converts 'same weather' from a visual impression into a geometric constraint, verified by reprojection errors below one pixel. The mechanism carries the entire argumen
What would settle it
Take a nuScenes scene, simulate fog with ReDAM, and directly compare the two modalities point-by-point: for each LiDAR beam whose intensity is attenuated by fog, check that the camera image's fog density at the same projected location matches. If the LiDAR and camera severity disagree on the target scene even though both individually match the reference dataset, the alignment claim fails. A stronger test would physically re-create the same fog density in a controlled environment and measure whether aligned simulations predict the observed LiDAR–camera pair more accurately than unaligned ones.
Extended reading notes
Core claim
The paper's central claim is that weather simulation realism depends not just on each sensor looking weather-affected, but on the weather being the same across sensors. Fog severity is aligned by tuning LiDAR attenuation and reflectivity parameters to minimize the Kolmogorov–Smirnov distance against real fog point clouds, and by sampling camera style references from the same real fog dataset at matching fog levels. Snow and rain are aligned by simulating particles in a common 3D world coordinate frame and transforming those positions into each sensor's coordinates, so a snowflake or raindrop blocks the same physical location in both modalities. The authors show that aligned simulations score
Load-bearing premise
The entire alignment claim rests on treating the real fog dataset's LiDAR–camera alignment as a valid model for the target driving dataset; if the fog statistics or sensor geometry of the target scene differ, simulations aligned to the reference may still be mutually misaligned on those target scenes.
Editorial extensions
If this is right
- Detection scores on unaligned simulations should not be interpreted as evidence of robustness; they are optimistic by 1–3% mAP in this study.
- Finetuning fusion models on aligned simulated weather improves weather performance by 0.03–0.13 mAP while largely preserving clean-data performance, suggesting aligned simulation is a usable robustness training signal.
- ReDAM can generate multiple weather severities by referencing different real-weather subsets, enabling severity-conditioned training and evaluation.
- The alignment procedure is stated to extend to other datasets, additional sensor modalities like radar, and mixed-weather scenarios.
Reading between the lines
- If aligned simulation is the more honest test, then prior robustness benchmarks built on unaligned weather corruption may systematically overstate model resilience; re-evaluating those models on aligned versions could revise previous conclusions.
- The sub-pixel reprojection error points to a practical audit: any multi-sensor weather simulator could be checked by projecting simulated particles across modalities and measuring consistency, which could become a standard verification step.
- The 'overly optimistic' claim presumes that realism improves monotonically with alignment; a natural extension would be to vary particle-position noise continuously and measure whether detection mAP tracks the actual visibility reduction, a test the paper does not run.
- ReDAM's reliance on a reference dataset suggests a transfer-risk: if the target dataset's sensor geometry or fog statistics differ from the reference, the alignment may degrade—an experiment comparing cross-dataset alignment quality would clarify when the method holds.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes two sensor-alignment methods for adverse-weather simulation in autonomous-driving perception: ReDAM, which tunes LiDAR fog parameters (α,β) and samples STF camera style images to match the Seeing Through Fog reference dataset, and Unified-weather-edit, which places snow/rain particles in a shared 3D world frame and transforms them into LiDAR and camera coordinates for position and temporal alignment. Validation includes KS statistics and FID/KID for fog, re-projection error for particle positions, MMD on fused LiDAR-camera features, and 3D detection evaluation with BEVFusion, DeepInteraction, and CMT. The authors claim that non-aligned simulations yield overly optimistic detection results and that aligned simulations improve robustness via fine-tuning.
Significance. If the claims hold, the paper would make a useful contribution to simulation-based evaluation of AV perception: a concrete method for cross-sensor particle-position alignment (with a genuine geometric check: re-projection error <1.4 px max) and a cautionary demonstration that unaligned weather simulations can be over-optimistic. The potential importance is real, but the fog half of the contribution is not yet substantiated: ReDAM's validation is circular, no cross-modal fog alignment metric is reported, and the headline 'overly optimistic' claim lacks a fog baseline. The particle-alignment part is stronger, but its re-projection error formula needs correction. The paper should be revised significantly before archival publication.
major comments (5)
- [§II-B1, §III ('Weather Severity Alignment Verification')] The ReDAM validation is circular. Section II-B1 chooses (α*,β*) by minimizing the KS statistic D_KS between simulated and STF LiDAR ECDFs, and Section III then reports that same D_KS (0.092/0.131) as evidence of alignment. No held-out split or independent sample is used. Similarly, camera FID/KID are computed on STF images from which the style images were randomly sampled. This fit-to-validation loop does not measure how well the simulations generalize. Please provide a train/test split of STF, report D_KS on a held-out subset, and compute FID/KID against STF images not used in style sampling.
- [§II-B1, Fig. 2, §III] No cross-modal fog alignment is actually demonstrated. ReDAM aligns LiDAR intensity to the STF ECDF with α,β fixed per severity split, while the camera style image is sampled per frame. A frame can therefore receive a dense-fog style image while the LiDAR is globally tuned to average light-fog statistics. The KS statistic is LiDAR-only, and FID/KID are camera-only; neither verifies that the two sensors represent the same fog state on the same sample. Report a per-sample cross-modal metric—e.g., estimated visibility from both sensors per frame, correlation between LiDAR attenuation and style-image fog density, or paired fused-feature distance—before claiming implicit alignment.
- [Abstract, §III, Table III] The central claim that non-aligned simulations are 'overly optimistic' is not tested for fog. Table II supports this only for snow and rain; Table III has no unaligned-fog baseline, so there is no direct evidence that aligned fog differs from unaligned fog for 3D detection. Since ReDAM is the paper's primary contribution for homogeneous weather, add an unaligned nuScenes-fog condition (e.g., independent random α,β and style images) to the detection evaluation, or restrict the claim to rain and snow.
- [§III, Table I] The fused-feature MMD check is not a reliable discriminator. For rain, Table I reports identical MMD values (0.1870) for aligned and unaligned despite a 2–3% mAP gap in Table II, suggesting the metric cannot detect alignment differences in that condition. No error bars, confidence intervals, or significance tests are provided. Give bootstrap distributions over scene-level samples, or at least show that the MMD check has power to distinguish the two conditions.
- [§III ('Particle Position Alignment Verification')] The re-projection error formula is not normalized as written: E = Σ_{i=1}^{|N_comm|} Σ_{j=1}^{NV} ||u_ij − û(P_j,X_i)||_2^2 is a sum of squared errors, so the reported 'Mean error 0.7632' and 'Max error 1.3843' in pixels do not follow from the formula. The definition must be normalized (e.g., mean over point-view pairs, with square root) for the reported values to be interpretable. Please provide the exact formula and report per-point mean and max errors; otherwise the geometric validation is not verifiable.
minor comments (5)
- [Table I] The re-projection error rows list values for Snow and Rain only; use '—' or 'N/A' for Fog. Also specify the MMD kernel and bandwidth, since MMD values depend on the kernel choice.
- [Tables II and III] Use consistent capitalization: Table III uses 'MAP' while the text and Table II use 'mAP'. Also in the text, 'we gain 0.13,0.03,0.07 mAP' should be stated as approximate gains from Table III baselines (e.g., 0.12, 0.03, 0.07) to avoid rounding ambiguity.
- [References] References [6] and [9] are the same paper by Hahner et al. on LiDAR fog simulation; consolidate them.
- [§III, Figure 3] The ECDF figure lacks axis labels and units; the green/orange bands are not explained in the caption. Adding these would help readers judge the claimed agreement.
- [§II-B1] The phrase 'at the same fog level' is undefined. Specify how fog level is determined when sampling STF style images (e.g., by dataset split, by human rating, or by a visibility estimate).
Circularity Check
Fog-alignment validation is partially circular: the fitted KS residual is reported as the validation metric, and FID/KID are scored against the same STF images used as style references; rain/snow mAP evidence is independent but the 'overly optimistic' claim is not directly tested for fog.
-
fitted input called prediction
[Section II-B1 (ReDAM for Fog, severity alignment) and Section III, 'Weather Severity Alignment Verification']
"Thus, we propose to tune these parameters to minimize the KS-Statistics between the intensity distributions of simulated and STF point cloud. ... (α∗, β∗) = argmin (α,β) [D_KS = sup_x |F_sim,(α,β)(x) − F_real(x)|] ... The parameters are evaluated over all samples in the corresponding split mentioned in STF dataset. The corresponding KS-Statistics values were obtained as D_KS =0.092 and D_KS =0.131 for light and dense fog, respectively."
The same D_KS that is minimized by grid search over all STF samples is then reported as the validation statistic. Because no held-out split or independent metric is introduced, the reported 'agreement' is the optimized fitting residual, not an out-of-sample confirmation; the ECDF plots in Figure 3 simply display the fitted curves.
-
self definitional
[Section II-B1 (camera severity alignment) and Section III, 'Weather Severity Alignment Verification']
"For each image from the nuScenes-mini dataset, a corresponding target style image is randomly sampled from the STF dataset at the same fog level and passed through the TSIT architecture to generate the simulated image. ... For images, we use the Fréchet Inception Distance (FID) & Kernel Inception Distance (KID) metrics between simulated nuScenes foggy images and real STF images with the same fog level."
The STF images that condition the TSIT style transfer are part of the same STF set used as the real reference distribution for FID/KID. The image-quality scores are therefore computed against the very inputs used to generate the fog, so a low score is partly by construction and does not independently certify sensor-aligned realism.
full rationale
The paper's rain/snow pipeline is substantially self-contained: particle positions are sampled in a common world coordinate system and transformed into each sensor, and the re-projection check is an independent geometric test; the mAP differences in Table II are real measurements, not fitted outputs. No load-bearing self-citation occurs (Weather-edit, TSIT, STF, and the fusion models are external). However, the fog half of the central claim is partially circular. First, the LiDAR fog parameters are selected by minimizing D_KS over all STF samples and the same minimized D_KS is then presented as the validation statistic; this is a fit residual, not an out-of-sample confirmation. Second, the camera fog is generated by randomly sampling STF style images, and the same STF set is used as the target distribution for FID/KID, so the image-quality scores are evaluated against the conditioning inputs rather than an independent reference. Third, the paper's inference that per-sensor marginal similarity to STF 'implicitly produces aligned weather simulations in both sensors' is an unverified assumption, not a derived result: no cross-modal fog alignment metric is reported, and the fused-feature MMD check is undercut by the identical rain MMD values in Table I. In addition, the headline 'overly optimistic' claim is directly tested only for rain and snow (Table II); no unaligned-fog baseline appears in Table III. These gaps make the fog alignment validation partly circular and leave the general conclusion only partially supported, but the independent rain/snow measurements keep the paper from being wholly circular.
Assumptions & free parameters
free parameters (6)
- fog attenuation α =
0.01 light, 0.04 dense
- fog reflectivity β =
1.23e-4 light, 6.13e-4 dense
- snowfall rate =
2.0 mm/hr
- snow terminal velocity D =
1.5 m/s in z
- rainfall rate =
50.0 mm/hr
- rain terminal velocity =
2.0 m/s
assumptions (7)
- domain assumption The Rasshofer LiDAR impulse-response model (Eq. 1) adequately captures weather effects on LiDAR.
- domain assumption Fog is a homogeneous medium with step-function response parameterized by α and β [9].
- domain assumption Snow/rain particles can be modeled as discrete occlusion/reflection impulses from [10][11].
- domain assumption TSIT and Weather-edit produce camera weather images whose style/particles are faithful to real weather.
- domain assumption STF provides aligned, real LiDAR-camera fog data representative enough to serve as a reference for nuScenes-mini.
- domain assumption MMD of features from a weather-classification fusion model measures cross-sensor alignment realism.
- domain assumption LiDAR is largely uncorrelated with image background changes [15], justifying using only particle positions from weather-edit.
Cite this review
Pith. "Pith review of Multi-Sensor Alignment for Weather Simulations." pith.science (2026). https://pith.science/paper/XTYV37UN
@misc{pith2026260725612,
author = {Pith},
title = {Pith review of: Multi-Sensor Alignment for Weather Simulations},
year = {2026},
howpublished = {\url{https://pith.science/paper/XTYV37UN}},
note = {Machine review of arXiv:2607.25612}
}
read the original abstract
Perception tasks for autonomous vehicles need to work satisfactorily in adverse weather conditions. Due to lack of real-world weather datasets, weather simulations are a promising alternative. To ensure simulations closely mirror real-world weather data, it's crucial that they represent the same weather characteristics, including severity and particle positioning, across different sensors. To achieve this, we propose the Reference Dataset Alignment Method (ReDAM) for weather intensity alignment in fog and Unified-weather-edit (inspired by Weather-edit[1]) for particle positioning alignment in rain and snow. We validate both alignment methods using statistical and geometrical tests, respectively. We find that 3D detection models for non-aligned versions tend to be overly optimistic as compared to aligned versions. We also show the aligned-multi-sensor simulation's effectiveness for achieving robustness for 3D object detection task by finetuning existing sensor fusion models on it.
Figures
Reference graph
Works this paper leans on
-
[1]
Weatheredit: Controllable weather editing with 4d gaussian field,
C. Qian, W. Li, Y . Guo, and G. Markkula, “Weatheredit: Controllable weather editing with 4d gaussian field,” 2025. [Online]. Available: https://arxiv.org/abs/2505.20471
arXiv 2025
-
[2]
Y . Huang, K. Yu, Q. Guo, F. Juefei-Xu, X. Jia, T. Li, G. Pu, and Y . Liu, “Improving robustness of lidar-camera fusion model against weather corruption from fusion strategy perspective,”arXiv preprint arXiv:2402.02738, 2024
arXiv 2024
-
[3]
Benchmarking robustness of 3d object detection to common corruptions,
Y . Dong, C. Kang, J. Zhang, Z. Zhu, Y . Wang, X. Yang, H. Su, X. Wei, and J. Zhu, “Benchmarking robustness of 3d object detection to common corruptions,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1022–1032
2023
-
[4]
Msc-bench: Benchmarking and analyzing multi-sensor corruption for driving perception,
X. Hao, G. Liu, Y . Zhao, Y . Ji, M. Wei, H. Zhao, L. Kong, R. Yin, and Y . Liu, “Msc-bench: Benchmarking and analyzing multi-sensor corruption for driving perception,”arXiv preprint arXiv:2501.01037, 2025
arXiv 2025
-
[5]
Lidar snowfall simulation for robust 3d object detection,
M. Hahner, C. Sakaridis, M. Bijelic, F. Heide, F. Yu, D. Dai, and L. Van Gool, “Lidar snowfall simulation for robust 3d object detection,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 16 364–16 374
2022
-
[6]
Fog simulation on real lidar point clouds for 3d object detection in adverse weather,
M. Hahner, C. Sakaridis, D. Dai, and L. Van Gool, “Fog simulation on real lidar point clouds for 3d object detection in adverse weather,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 15 283–15 292
2021
-
[7]
V . W. Berger and Y . Zhou,Kolmogorov–Smirnov Test: Overview. John Wiley & Sons, Ltd, 2014. [Online]. Available: https://onlinelibrary. wiley.com/doi/abs/10.1002/9781118445112.stat06558
-
[8]
Influences of weather phenomena on automotive laser radar systems,
R. H. Rasshofer, M. Spies, and H. Spies, “Influences of weather phenomena on automotive laser radar systems,”Advances in Radio Science, vol. 9, pp. 49–60, Jul. 2011
2011
Show all 23 references
-
[9]
Fog simulation on real lidar point clouds for 3d object detection in adverse weather,
M. Hahner, C. Sakaridis, D. Dai, and L. V . Gool, “Fog simulation on real lidar point clouds for 3d object detection in adverse weather,”
-
[10]
Lidar snowfall simulation for robust 3d object detection,
M. Hahner, C. Sakaridis, M. Bijelic, F. Heide, F. Yu, D. Dai, and L. V . Gool, “Lidar snowfall simulation for robust 3d object detection,” 2022. [Online]. Available: https://arxiv.org/abs/2203.15118
2022 arXiv
-
[11]
Lidar light scattering augmentation (lisa): Physics-based simulation of adverse weather conditions for 3d object detection,
V . Kilic, D. Hegde, V . Sindagi, A. B. Cooper, M. A. Foster, and V . M. Patel, “Lidar light scattering augmentation (lisa): Physics-based simulation of adverse weather conditions for 3d object detection,”
-
[12]
Tsit: A simple and versatile framework for image-to-image translation,
L. Jiang, C. Zhang, M. Huang, C. Liu, J. Shi, and C. C. Loy, “Tsit: A simple and versatile framework for image-to-image translation,” in European conference on computer vision. Springer, 2020, pp. 206– 222
2020
-
[13]
Available: https://arxiv.org/abs/2107.07004
[Online]. Available: https://arxiv.org/abs/2107.07004
-
[14]
nuscenes: A multimodal dataset for autonomous driving,
H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” 2020. [Online]. Available: https://arxiv.org/abs/1903.11027
2020 arXiv
-
[15]
Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather,
M. Bijelic, T. Gruber, F. Mannan, F. Kraus, W. Ritter, K. Dietmayer, and F. Heide, “Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather,” inThe IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020
2020
-
[16]
Gans trained by a two time-scale update rule converge to a local nash equilibrium,
M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” 2018. [Online]. Available: https://arxiv.org/abs/1706.08500
2018 arXiv
-
[17]
Weitkamp, Ed.,Lidar: Range-Resolved Optical Remote Sensing of the Atmosphere, 1st ed., ser
C. Weitkamp, Ed.,Lidar: Range-Resolved Optical Remote Sensing of the Atmosphere, 1st ed., ser. Springer Series in Optical Sciences. New York, NY: Springer, 2005
2005
-
[18]
A kernel two-sample test,
A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Sch ¨olkopf, and A. Smola, “A kernel two-sample test,”The journal of machine learning research, vol. 13, no. 1, pp. 723–773, 2012
2012
-
[19]
Demystifying mmd gans,
M. Bi ´nkowski, D. J. Sutherland, M. Arbel, and A. Gretton, “Demystifying mmd gans,” 2021. [Online]. Available: https://arxiv.org/ abs/1801.01401
2021 arXiv
-
[20]
Deepinterac- tion: 3d object detection via modality interaction,
Z. Yang, J. Chen, Z. Miao, W. Li, X. Zhu, and L. Zhang, “Deepinterac- tion: 3d object detection via modality interaction,”Advances in Neural Information Processing Systems, vol. 35, pp. 1992–2005, 2022
1992
-
[21]
Bevfusion: A simple and robust lidar-camera fusion frame- work,
T. Liang, H. Xie, K. Yu, Z. Xia, Z. Lin, Y . Wang, T. Tang, B. Wang, and Z. Tang, “Bevfusion: A simple and robust lidar-camera fusion frame- work,”Advances in Neural Information Processing Systems, vol. 35, pp. 10 421–10 434, 2022
2022
-
[23]
Cross modal transformer: Towards fast and robust 3d object detection,
J. Yan, Y . Liu, J. Sun, F. Jia, S. Li, T. Wang, and X. Zhang, “Cross modal transformer: Towards fast and robust 3d object detection,” in Proceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 18 268–18 278
2023
-
[2021]
Available: https://arxiv.org/abs/2108.05249
[Online]. Available: https://arxiv.org/abs/2108.05249
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.