REVIEW 2 major objections 5 minor 31 references
How Hard Is Snow? A Paired Domain Adaptation Dataset for Clear and Snowy Weather: CADC+
T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Snow is not just noise for LiDAR detection; it is a separate data domain, and CADC+ is the first paired dataset that measures both effects.
desk verdict CADC+ is a genuinely useful paired clear/snow LiDAR dataset and deserves peer review, but the snow-effect experiments need a pairing-quality analysis before the aleatoric/epistemic claim is solid. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the paired dataset CADC+, built by matching each CADC snowy sequence to a clear sequence from the same road network and recording period. The mechanical core is a coverage-based matching procedure: a function measures what fraction of a snowy trajectory lies within a given lateral distance of a candidate clear trajectory, candidates are ranked by coverage quality, and endpoint sampling resamples the chosen clear sequence to at least 100 frames at 2 to 10 hertz. A second mechanism, sparse labelling with semi-supervised pseudo-labels, makes full-sequence training affordable while matching the performance of fully human-annotated training.
What would settle it
Restrict evaluation to the 53 closely matched pairs and compute the clear-versus-snowy average-precision gap, then compute the same gap on the 21 manually matched pairs; if the loose pairs show a markedly larger gap, residual scene mismatch is contaminating the snow measurement, whereas equal gaps would support the pairing assumption. A second check: train a detector on real clear data and evaluate it on de-snowed snowy frames and on real clear frames from the same pairs; equal performance would show de-snowing is a valid substitute, which the paper predicts is false.
Extended reading notes
Core claim
The central claim is that snow is not a single failure mode for LiDAR-based 3D detection but a combination of two. One component is aleatoric: snow physically corrupts the measurement, adding noise, attenuation, and occlusion, so the same object is simply harder to sense. The other is epistemic: snowy point clouds form their own statistical domain, so a detector trained only on clear data performs worse on snow, and adding snowy training data monotonically improves snowy detection. The paper also finds that de-snowing real snowy clouds with a statistical filter (DROR) or a learned method (LiSnowNet) does not recreate clear weather from a detector's point of view: models trained on de-snowed data behave on clear evaluation data much like models trained on snow, which the authors take as evidence that synthetic clear data is not a substitute for real paired clear-snow data.
Load-bearing premise
The load-bearing assumption is that each paired clear and snowy drive is similar enough in route, layout, traffic, and timing that the remaining detection gap is caused by snow; the authors state in the matching section that none of the ideal pairing requirements can be fully met, and 21 of the 74 snowy sequences were matched manually to different locations or road layouts.
Editorial extensions
If this is right
- Detectors trained only on clear weather will systematically underperform on snow, and the gap cannot be closed by more clear data alone.
- Training near half snowy and half clear data gives near-maximum performance on both domains, suggesting a practical recipe for all-weather detectors.
- De-snowed point clouds are not a reliable substitute for real clear data when training or evaluating snow-effect studies; downstream detection performance, not point-cloud appearance, is the right yardstick.
- Sparse labelling with semi-supervised learning makes a paired winter dataset feasible at roughly 10 percent annotation cost without losing detection accuracy.
- CADC+ provides a controlled testbed for weather domain adaptation, letting future detectors be trained and evaluated on the same routes in both weathers.
Reading between the lines
- One consequence the authors do not draw explicitly: if snowy point clouds are a distinct domain, then weather-invariant detectors could be trained by feature-space alignment between the CADC+ pairs, using the paired geometry as supervision.
- The 21 manually matched pairs form a natural stress test: a detector that is truly snow-robust should show the same clear-versus-snowy gap on close and loose pairs; divergence would mean route mismatch is contaminating the measurement.
- The same matching protocol, coverage under lateral thresholds plus endpoint sampling, transfers to other seasonal or weather pairings whenever clear and adverse runs were collected on a shared route network.
- The paper's own results suggest an operating point that it does not elevate into a rule: roughly balanced snowy/clear training appears to maximize worst-case performance across the two domains.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces CADC+, an extension of the Canadian Adverse Driving Conditions (CADC) dataset with clear-weather sequences intended to match each of CADC's 74 snowy sequences. The stated goal is to provide the first paired real-data domain adaptation dataset for LiDAR-based 3D object detection in winter conditions, minimizing domain shift from factors other than snow. The authors describe a sequence-matching algorithm based on spatial interpolation and a coverage function, a sparse-labelling strategy with semi-supervised learning, and preliminary experiments comparing models trained on mixtures of snowy and clear data as well as on de-snowed synthetic clear data. They report that snow degrades detection performance and interpret this as evidence that snow introduces both aleatoric and epistemic uncertainty.
Significance. If the matching is as effective as claimed, CADC+ addresses a real gap: existing snowy driving datasets either lack paired clear data, lack sufficient labelled 3D boxes, or rely on synthetic de-snowing that introduces its own domain shift. The distribution comparisons in Fig. 6 (point counts per box, objects per frame) are useful evidence that the two domains are broadly similar in scene content. The de-snowing comparison in Table II is also a valuable negative result, showing that DROR and LiSnowNet do not produce realistic clear data under this evaluation protocol. The contribution is primarily a dataset, and the experimental section is explicitly preliminary, so the bar for the qualitative uncertainty claims should be adjusted accordingly. The paper's central weakness is that the pairing quality, which is load-bearing for attributing the observed performance gap to snow, is not quantified beyond qualitative descriptions.
major comments (2)
- [Section III-B and III-D, Fig. 7, Table II] The central attribution of the snowy-vs-clear performance gap to snow is not supported because pairing quality is never quantified. The paper states in Section III-B that "none of these requirements can be fully met" and in Section III-D that 21 of 74 CADC sequences could not be closely matched, with 15 manually paired to a different location and 6 to a different road-agent composition, yet no values of cover(s,c,theta) at the selected theta, no d_max distribution, and no per-pair breakdown of the matching subset are reported. Figure 3d-e also show that endpoint selection can extend matched sequences beyond the closest matching frames to reach 100 frames, increasing the maximum distance discrepancy. Without quantitative pairing statistics, the roughly 3-4 AP gap in Table II and the trends in Fig. 7 may partly reflect scene or layout mismatch rather than snow. Please report the coverage and distance statistics for all 74 pairs and repeat the analysis on the subset of closely matched pairs to show that the conclusions are robust.
- [Section IV, Fig. 7, Table II] All experimental results are single-run, with no error bars, confidence intervals, or statistical tests. The aleatoric and epistemic uncertainty conclusions rest on small differences (for example, L1 AP gaps of 1-4 points in Table II and the non-monotonic peak at 75% snowy training data in Fig. 7), which could shift with random seed or training noise. Please provide multiple seeds or otherwise quantify variability, or temper the wording of the uncertainty conclusions so that they are presented as observations from a preliminary single-run study rather than established trends.
minor comments (5)
- [Figure 7] The x-axis is labelled "fraction of training data" but the tick labels and the words "snowy" and "clear" suggest it is specifically the fraction of snowy training data; please state this explicitly in the caption.
- [Section III-B, Eqs. (1)-(3)] The notation c*_{s,theta} is defined as an argmax and therefore denotes a set, but the surrounding prose sometimes refers to it in the singular; a brief clarification of set-valued notation would improve readability.
- [Table I] The Delta% column reports differences in percentage points (e.g., -2.76), not relative percentages; please label the column as difference in percentage points or adjust the values.
- [Section IV] The statement that training epochs are "normalized with respect to the amount of data" is imprecise; please report the actual epoch counts or effective iteration counts for each training configuration.
- [Section III-D] The distribution comparisons in Figs. 5 and 6 are helpful, but the paper would benefit from explicitly stating whether the reported statistics include the 21 manually matched pairs, since those pairs may behave differently from the closely matched ones.
Circularity Check
No significant circularity: CADC+ is a dataset construction with empirical evaluations; the sparse-labelling self-citation is independently supported by Table I.
full rationale
The paper's central contribution is a paired dataset, not a derived prediction, and its experimental conclusions are empirical trends rather than quantities forced by construction. The matching procedure defined by cover(s,c,theta) is a design objective for selecting clear sequences; it is not a fitted parameter later renamed as a prediction. The claim that snow introduces aleatoric and epistemic uncertainty is supported by cross-evaluation results in Fig. 7 and Table II, which are observed outcomes of trained models, not algebraic consequences of the dataset construction. The only notable self-citation is [25], used to justify the sparse labelling and semi-supervised labelling strategy. That reliance is not circular because the paper independently validates the strategy on CADC in Table I: SSL 10% achieves 70.78 L1 AP versus 70.15 for Base 100%, demonstrating that the semi-supervised approach performs comparably on the closely related dataset. Thus the self-citation is corroborated evidence rather than an unverified load-bearing premise. The acknowledged pairing limitations (e.g., 'none of these requirements can be fully met' in Section III-B, and 21 of 74 sequences manually matched in Section III-D) are potential confounding-factor concerns for validity, not circularity: they do not show that any result is equivalent to its own input by definition. No step in the paper reduces a claimed prediction to a fitted value, a renamed known result, or a self-citation chain, so the honest finding is no significant circularity.
Assumptions & free parameters
free parameters (4)
- Matching lateral distance threshold set Θ
- Coverage threshold 0.95 for rule 2 =
0.95
- Minimum clear sequence length =
100 frames
- Training sequence label fraction =
0.1 (every tenth frame)
assumptions (3)
- domain assumption Clear sequences recorded on the same roads and in the same period as CADC are similar enough in environment and traffic that remaining differences are negligible compared with the snow effect.
- domain assumption The semi-supervised sparse labelling strategy from [25] transfers to CADC and produces labels close to human quality.
- domain assumption LiDAR returns affected by snow behave as noise plus attenuation, as described in the cited sensor studies.
Cite this review
Pith. "Pith review of How Hard Is Snow? A Paired Domain Adaptation Dataset for Clear and Snowy Weather: CADC+." pith.science (2026). https://pith.science/paper/INJJA5FY
@misc{pith2026250616531,
author = {Pith},
title = {Pith review of: How Hard Is Snow? A Paired Domain Adaptation Dataset for Clear and Snowy Weather: CADC+},
year = {2026},
howpublished = {\url{https://pith.science/paper/INJJA5FY}},
note = {Machine review of arXiv:2506.16531}
}
read the original abstract
The impact of snowfall on 3D object detection performance remains underexplored. Conducting such an evaluation requires a dataset with sufficient labelled data from both weather conditions, ideally captured in the same driving environment. Current driving datasets with LiDAR point clouds either do not provide enough labelled data in both snowy and clear weather conditions, or rely on de-snowing methods to generate synthetic clear weather. Synthetic data often lacks realism and introduces an additional domain shift that confounds accurate evaluations. To address these challenges, we present CADC+, the first paired weather domain adaptation dataset for autonomous driving in winter conditions. CADC+ extends the Canadian Adverse Driving Conditions dataset (CADC) using clear weather data that was recorded on the same roads and in the same period as CADC. To create CADC+, we pair each CADC sequence with a clear weather sequence that matches the snowy sequence as closely as possible. CADC+ thus minimizes the domain shift resulting from factors unrelated to the presence of snow. We also present some preliminary results using CADC+ to evaluate the effect of snow on 3D object detection performance. We observe that snow introduces a combination of aleatoric and epistemic uncertainties, acting as both noise and a distinct data domain.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[25]
3D object detection with track-based auto-labelling using very sparsely labelled data,
M. Q. Tang, V . Abdelzadet al., “3D object detection with track-based auto-labelling using very sparsely labelled data,” inIEEE International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2024, to appear
work page 2024
-
[1]
Object detection in degraded LiDAR signals by synthetic snowfall noise for autonomous driving,
A. D. Abu-Shaqra, N. J. Abu-Alrub, and N. A. Rawashdeh, “Object detection in degraded LiDAR signals by synthetic snowfall noise for autonomous driving,” inAutonomous Systems: Sensors, Processing and Security for Ground, Air , Sea and Space V ehicles and Infrastruc- ture 2022, vol. 12115, International Society for Optics and Photonics. SPIE, 2022, pp. 121 150K:1–8
work page 2022
-
[2]
Fast statistical outlier removal based method for large 3D point clouds of outdoor environments,
H. Balta, J. Velagicet al., “Fast statistical outlier removal based method for large 3D point clouds of outdoor environments,”IF AC- PapersOnLine, vol. 51, no. 22, pp. 348–353, 2018, 12th IFAC Sym- posium on Robot Control SYROCO 2018
work page 2018
-
[3]
LiDAR point cloud de-noising for adverse weather,
J. Bergius and J. Holmblad, “LiDAR point cloud de-noising for adverse weather,” Master’s thesis, , School of Information Technology, 2022
work page 2022
-
[4]
Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather,
M. Bijelic, T. Gruberet al., “Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 682–11 692
work page 2020
-
[5]
Boreas: A multi-season autonomous driving dataset,
K. Burnett, D. J. Yoonet al., “Boreas: A multi-season autonomous driving dataset,”The International Journal of Robotics Research, vol. 42, no. 1-2, pp. 33–42, 2023
work page 2023
-
[6]
De-noising of lidar point clouds corrupted by snowfall,
N. Charron, S. Phillips, and S. L. Waslander, “De-noising of lidar point clouds corrupted by snowfall,” in2018 15th Conference on Computer and Robot Vision (CRV), 2018, pp. 254–261
work page 2018
-
[7]
V oxelNeXt: Fully sparse V oxelNet for 3D object detection and tracking,
Y . Chen, J. Liuet al., “V oxelNeXt: Fully sparse V oxelNet for 3D object detection and tracking,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 21 674–21 683
work page 2023
Show all 31 references
-
[8]
Ithaca365: Dataset and driving perception under repeated and challenging weather conditions,
C. A. Diaz-Ruiz, Y . Xiaet al., “Ithaca365: Dataset and driving perception under repeated and challenging weather conditions,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 21 383–21 392
2022
-
[9]
Vision meets robotics: The KITTI dataset,
A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, “Vision meets robotics: The KITTI dataset,”The International Journal of Robotics Research, vol. 32, no. 11, pp. 1231–1237, 2013
2013
-
[10]
LiDAR snowfall simulation for robust 3D object detection,
M. Hahner, C. Sakaridiset al., “LiDAR snowfall simulation for robust 3D object detection,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 16 364–16 374
2022
-
[11]
CNN-Based lidar point cloud de-noising in adverse weather,
R. Heinzler, F. Piewak, P. Schindler, and W. Stork, “CNN-Based lidar point cloud de-noising in adverse weather,”IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2514–2521, 2020
2020
-
[12]
Testing and validation of automotive point-cloud sensors in adverse weather conditions,
M. Jokela, M. Kutila, and P. Pyyk ¨onen, “Testing and validation of automotive point-cloud sensors in adverse weather conditions,” Applied Sciences, vol. 9, no. 11, 2019
2019
-
[13]
Lidar light scattering augmentation (LISA): Physics-based simulation of adverse weather conditions for 3D object detection,
V . Kilic, D. Hegdeet al., “Lidar light scattering augmentation (LISA): Physics-based simulation of adverse weather conditions for 3D object detection,” 2021, arXiv: 2107.07004
2021 arXiv
-
[14]
DSOR: A scalable statistical filter for removing falling snow from LiDAR point clouds in severe winter weather,
A. Kurup and J. Bos, “DSOR: A scalable statistical filter for removing falling snow from LiDAR point clouds in severe winter weather,” 2021, arXiv: 2109.07078
2021 arXiv
-
[15]
Winter adverse driving dataset (W ADS): year three,
——, “Winter adverse driving dataset (W ADS): year three,” inAu- tonomous Systems: Sensors, Processing and Security for Ground, Air , Sea and Space V ehicles and Infrastructure 2022, vol. 12115, International Society for Optics and Photonics. SPIE, 2022, pp. 031 207:1–17
2022
-
[16]
Benchmarking automotive LiDAR performance in arctic conditions,
M. Kutila, P. Pyyk ¨onenet al., “Benchmarking automotive LiDAR performance in arctic conditions,” inIEEE International Conference on Intelligent Transportation Systems (ITSC), 2020, pp. 1–8
2020
-
[17]
De-snowing lidar point clouds with intensity and spatial-temporal features,
B. Li, J. Liet al., “De-snowing lidar point clouds with intensity and spatial-temporal features,” inInternational Conference on Robotics and Automation (ICRA), 2022, pp. 2359–2365
2022
-
[18]
GROUNDED: The localizing ground penetrating radar evaluation dataset
T. Ort, I. Gilitschenski, and D. Rus, “GROUNDED: The localizing ground penetrating radar evaluation dataset.” inRobotics: Science and Systems, vol. 2, 2021
2021
-
[19]
Fast and accurate desnowing algorithm for LiDAR point clouds,
J.-I. Park, J. Park, and K.-S. Kim, “Fast and accurate desnowing algorithm for LiDAR point clouds,”IEEE Access, vol. 8, pp. 160 202– 160 212, 2020
2020
-
[20]
Boosting LiDAR-based semantic label- ing by cross-modal training data generation,
F. Piewak, P. Pinggeraet al., “Boosting LiDAR-based semantic label- ing by cross-modal training data generation,” inEuropean Conference on Computer Vision (ECCV), September 2018
2018
-
[21]
Canadian adverse driving conditions dataset,
M. Pitropov, D. E. Garciaet al., “Canadian adverse driving conditions dataset,”The International Journal of Robotics Research, vol. 40, no. 4-5, pp. 681–690, 2021
2021
-
[22]
Influences of weather phenomena on automotive laser radar systems,
R. H. Rasshofer, M. Spies, and H. Spies, “Influences of weather phenomena on automotive laser radar systems,”Advances in Radio Science, vol. 9, pp. 49–60, 2011
2011
-
[23]
4DenoiseNet: Adverse weather denoising from adjacent point clouds,
A. Sepp ¨anen, R. Ojala, and K. Tammi, “4DenoiseNet: Adverse weather denoising from adjacent point clouds,”IEEE Robotics and Automation Letters, vol. 8, no. 1, pp. 456–463, 2023
2023
-
[24]
Scalability in perception for au- tonomous driving: Waymo Open Dataset,
P. Sun, H. Kretzschmaret al., “Scalability in perception for au- tonomous driving: Waymo Open Dataset,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
2020
-
[26]
Simulating realistic rain, snow, and fog variations for comprehensive performance characterization of LiDAR perception,
S. Teufel, G. V olk, A. V on Bernuth, and O. Bringmann, “Simulating realistic rain, snow, and fog variations for comprehensive performance characterization of LiDAR perception,” in2022 IEEE 95th V ehicular Technology Conference: (VTC2022-Spring), 2022, pp. 1–7
2022
-
[27]
Snow removal for LiDAR point clouds with spatio-temporal conditional random fields,
W. Wang, T. Yang, Y . Du, and Y . Liu, “Snow removal for LiDAR point clouds with spatio-temporal conditional random fields,”IEEE Robotics and Automation Letters, vol. 8, no. 10, pp. 6739–6746, 2023
2023
-
[28]
A scalable and accurate de-snowing algorithm for LiDAR point clouds in winter,
W. Wang, X. Youet al., “A scalable and accurate de-snowing algorithm for LiDAR point clouds in winter,”Remote Sensing, vol. 14, no. 6, 2022
2022
-
[29]
Denoising framework based on multiframe continuous point clouds for autonomous driving LiDAR in snowy weather,
X. Yan, J. Yanget al., “Denoising framework based on multiframe continuous point clouds for autonomous driving LiDAR in snowy weather,”IEEE Sensors Journal, vol. 24, no. 7, pp. 10 515–10 527, 2024
2024
-
[30]
LiSnowNet: Real-time snow removal for LiDAR point clouds,
M.-Y . Yu, R. Vasudevan, and M. Johnson-Roberson, “LiSnowNet: Real-time snow removal for LiDAR point clouds,” inIEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 6820–6826
2022
-
[31]
Perception and sensing for autonomous vehicles under adverse weather conditions: A survey,
Y . Zhang, A. Carballo, H. Yang, and K. Takeda, “Perception and sensing for autonomous vehicles under adverse weather conditions: A survey,”ISPRS Journal of Photogrammetry and Remote Sensing, vol. 196, pp. 146–177, 2023. APPENDIX A. Interpolation Letw t i denote the spatial (G...
2023
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.