REVIEW 4 major objections 5 minor 37 references
Is it Raining Outside? Detection of Rainfall using General-Purpose Surveillance Cameras
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper claims that a fully-convolutional 3D CNN detects rainfall from surveillance cameras only on scenes resembling its training data, failing entirely on an asphalt surface.
desk verdict Genuine dataset and baseline, but the abstract overclaims on the paper's own numbers. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the modified C3D network: a 3D convolutional architecture that takes 16-frame clips as input and ends in two 2D convolutional layers plus global averaging, converting it into a fully-convolutional network that scores every 112×112 crop of the frame. It is trained from scratch as a binary rain classifier. The comparison baseline is Bossu's pipeline, which extracts candidate rain streaks via the photometric constraint, builds a histogram of streak orientations, fits a Gaussian-uniform mixture with expectation-maximization, and uses a Kalman filter to smooth detections. The C3D's temporal receptive field is what lets it in principle see rain streaks moving across frames, while the Bossu method explicitly models streak orientation.
What would settle it
If a disdrometer were placed directly beside one of the cameras, and the current model's predictions disagreed with that local ground truth on a large fraction of rain minutes, the assumption tying distant gauge data to the camera scene would be falsified. Alternatively, manually labeling rain visibility on Crossing2-asphalt and showing that the C3D-FCN predicts no rain for clips with clearly visible streaks would falsify the claim that the network detects rain in that setting.
Extended reading notes
Core claim
On its own terms, the paper establishes that a 3D CNN trained on 16-frame clips of one traffic crossing can detect rainfall in surveillance footage from that crossing with high accuracy (MCC 0.94 on training, 0.58 on validation), comfortably beating the hand-crafted Bossu detector whose MCC stays near zero. The same network, applied as a fully-convolutional network with global averaging, also outperforms Bossu on a brick-textured region of a second crossing but fails completely on an asphalt region of that crossing, where it outputs 'no rain' for every frame. The authors interpret this as evidence that the CNN is learning scene texture rather than a generic rain signature, and conclude that rain detection for general-purpose surveillance cameras is not yet solved. The paper's contribution therefore is a public benchmark (VIRADA), an open-source reimplementation of the baseline, and a careful demonstration of where the data-driven method wins and where it breaks.
Load-bearing premise
The ground truth assumes that the laser disdrometer, situated roughly one kilometer from each camera, records the same rain that is falling at the camera scene, so per-minute labels are accurate for every frame of video.
Editorial extensions
If this is right
- Deploying this detector as a preprocessing stage for rain-removal algorithms is feasible only at sites whose surfaces resemble the training data, or after fine-tuning on the target scene.
- The VIRADA dataset gives the community a shared 215-hour benchmark with laser-disdrometer labels, so future rain-detection methods can be compared on identical surveillance footage.
- For general-purpose camera networks, the practical implication is that a site-specific region of interest (e.g., a brick wall) is a safer choice than a textured surface like asphalt.
- The comparison quantifies how far hand-crafted streak-orientation methods lag behind learned features on this task.
Reading between the lines
- The failure on asphalt hints that the network may be short-cutting the task by recognizing static scene appearance rather than temporal rain dynamics; training on a larger variety of surfaces (or adding synthetic rain) could be a testable extension.
- Because the camera and disdrometer are separated by about a kilometer, some per-minute label mismatches are inevitable; a denser network of local sensors could sharpen both training and evaluation.
- The same fully-convolutional architecture could be repurposed to estimate rain rate rather than binary presence, using camera geometry approaches, if labeled rainfall volume were available.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents the AAU VIRADA dataset, 215 hours of surveillance video from two traffic crossings with rainfall ground truth from a laser disdrometer and a tipping-bucket rain gauge, and compares a reimplementation of the Bossu et al. rain detection method with a 3D convolutional network (C3D) converted to a fully convolutional network. The authors report that the proposed C3D method outperforms Bossu et al. on all metrics for both crossings, and also analyze how the choice of region of interest affects generalization. The dataset and implementations are released publicly.
Significance. If the central comparative claim were supported, this would be a useful benchmark and baseline for rain detection from general-purpose surveillance cameras, and the public dataset plus open-source implementations would be a valuable community resource. The paper's strengths are the scale of the dataset, the explicit use of MCC as a primary metric for imbalanced data, and the public release of code and data. However, the headline claim is contradicted by the paper's own Table 5, and the evaluation protocol contains circularity and label-quality issues that currently undermine the reported comparisons.
major comments (4)
- [Abstract and Section 6] The abstract states that the proposed 3D CNN 'outperforms the previous state-of-the-art method by a large margin on all metrics, for both of the traffic crossings,' but Table 5 contradicts this. On Crossing2-asphalt, C3D-FCN achieves F1=0.0 and MCC=0.0, while Bossu-EM achieves F1=0.136 and MCC=0.008, so Bossu is better on two of the three metrics. The body text acknowledges this failure in Section 6 ('the C3D CNN evaluating the entire frame predicts no rain'), so the abstract and contribution statements must be revised to match the actual results, which support only the more nuanced conclusion in Section 7.
- [Section 5.1 and Table 3] The Bossu parameters are selected by a search on six video snippets from Crossing1 and then the method is evaluated on the full Crossing1-trn and Crossing1-val splits. If those snippets are part of the evaluation sets, the hyperparameters are tuned on the test data, which invalidates the Crossing1 comparison as an estimate of generalization. The authors should clarify whether the tuning snippets are included in the reported splits and should evaluate Bossu on a held-out set not used for parameter selection.
- [Section 5.2 and Table 5] Table 5 reports C3D-FCN and C3D-Center results on Crossing1-trn, which is the training split used to fit the network. Reporting performance on the training set is not evidence of generalization and inflates the apparent superiority of the C3D method. The comparison should focus on Crossing1-val and the Crossing2-test splits, with the training row either removed or clearly labeled as training fit rather than as a benchmark result.
- [Section 3.2 and Table 1] The ground-truth rain labels come from a laser disdrometer located 1230 m (Crossing1) and 970 m (Crossing2) from the cameras, and per-minute gauge readings are converted to per-frame labels. Rainfall is spatially variable at these distances, so the labels may not match the rain condition at the camera scene. This label noise affects every reported metric and should be quantified, for example by comparing the disdrometer and tipping-bucket records, by analyzing the sensitivity of results to label temporal smoothing, or by restricting evaluation to periods of spatially homogeneous rain.
minor comments (5)
- [Throughout] There are several typographical errors, including 'at-glace' in the Figure 1 caption, 'millenium' in Section 2, 'trough' in Section 3.1, and 'Kolomogrov' in Section 4.1; these should be corrected.
- [Section 6] In the text, C3D-Center on Crossing2-asphalt is reported as having an MCC of -0.08, while Table 5 reports -0.0837; the rounding should be consistent.
- [Section 5.2] The paper says the network is trained with a temporal stride of 8 frames and that the method 'analyse 16 frames at a time'; it would be clearer to state explicitly how the 16-frame clips are formed relative to the stride and the per-frame label conversion.
- [Section 6] The phrase 'outperforms the Bossu algorithm on all of the Crossing1 videos' is imprecise because the training split is included; it should be clarified that the strong result is on Crossing1-val or qualified as training-set performance.
- [Table 5] The table would benefit from a column showing the number of predictions for each method, since the C3D methods produce far fewer predictions than the per-frame Bossu methods; this is explained in the text but should be visible in the table.
Circularity Check
No significant circularity: the empirical comparison is self-contained; the abstract overclaim on Crossing2-asphalt is a factual inconsistency, not a circular derivation.
full rationale
This paper is an empirical benchmark rather than a derivation chain, so the circularity patterns are largely inapplicable. The central comparison is based on held-out validation data (Crossing1-val) and an unseen crossing (Crossing2, especially the brick crop), where the C3D model was not trained and the Bossu parameters were not tuned. The only in-sample reporting is the Crossing1-trn row, where Bossu parameters were selected using six snippets from Crossing1 (Section 5.1: "we perform a parameter search on six video snippets from the Crossing1 dataset") and the C3D network was trained on Crossing1-trn (Section 5.2). The paper explicitly acknowledges this overlap in Section 6: "As we trained the C3D CNN on a subset of the Crossing1 dataset and determined the parameters of the Bossu algorithm on the very same dataset, the difference in performance is striking." This is a methodological weakness in using training-set metrics as evidence, but it is not a circular reduction: the Crossing1-val and Crossing2-brick results are independent of the fitted parameters, and they still show C3D outperforming Bossu. The abstract's claim that C3D "outperforms the previous state-of-the-art method by a large margin on all metrics, for both of the traffic crossings" is contradicted by Table 5 on Crossing2-asphalt, where C3D-FCN scores F1=0.0 and MCC=0.0 while Bossu-EM scores F1=0.136 and MCC=0.008. That is an internal factual inconsistency, not a circular step. The paper's own limitations in Section 6 ("The C3D CNN evaluating the entire frame predicts no rain") and Section 7 ("our 3D CNN failed to function") further undercut the abstract but do not create circularity. There is no load-bearing self-citation, no imported uniqueness theorem, and no ansatz smuggled in via citation. Therefore, no circularity is present.
Assumptions & free parameters
free parameters (5)
- c (photometric constraint threshold) =
3
- Maximum BLOB size =
200
- dm (orientation uncertainty scaling) =
0.50
- Dc (KS goodness-of-fit threshold) =
0.19
- Pi_rain (Gaussian ratio threshold) =
0.40
assumptions (4)
- domain assumption Laser disdrometer measurements at 970-1230 m from the camera represent rainfall at the camera scene.
- domain assumption Rain streaks are visible and detectable in general-purpose surveillance video with the given camera settings.
- domain assumption The photometric constraint (Eq. 1-2) and the Gaussian-uniform HOS model (Eq. 7) are valid for rain streak segmentation and detection in the Bossu method.
- domain assumption A 16-frame clip with stride 8 captures enough temporal rain information for classification by C3D.
Cite this review
Pith. "Pith review of Is it Raining Outside? Detection of Rainfall using General-Purpose Surveillance Cameras." pith.science (2026). https://pith.science/paper/5EHMMII7
@misc{pith2026190804034,
author = {Pith},
title = {Pith review of: Is it Raining Outside? Detection of Rainfall using General-Purpose Surveillance Cameras},
year = {2026},
howpublished = {\url{https://pith.science/paper/5EHMMII7}},
note = {Machine review of arXiv:1908.04034}
}
read the original abstract
In integrated surveillance systems based on visual cameras, the mitigation of adverse weather conditions is an active research topic. Within this field, rain removal algorithms have been developed that artificially remove rain streaks from images or video. In order to deploy such rain removal algorithms in a surveillance setting, one must detect if rain is present in the scene. In this paper, we design a system for the detection of rainfall by the use of surveillance cameras. We reimplement the former state-of-the-art method for rain detection and compare it against a modern CNN-based method by utilizing 3D convolutions. The two methods are evaluated on our new AAU Visual Rain Dataset (VIRADA) that consists of 215 hours of general-purpose surveillance video from two traffic crossings. The results show that the proposed 3D CNN outperforms the previous state-of-the-art method by a large margin on all metrics, for both of the traffic crossings. Finally, it is shown that the choice of region-of-interest has a large influence on performance when trying to generalize the investigated methods. The AAU VIRADA dataset and our implementation of the two rain detection algorithms are publicly available at https://bitbucket.org/aauvap/aau-virada.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
P Allamano, A Croci, and F Laio. Toward the camera rain gauge. Water Resources Research, 51(3):1744–1757, 2015. 2, 3
work page 2015
-
[2]
Context-aware fusion of rgb and thermal imagery for traffic monitoring
Thiemo Alldieck, Chris Bahnsen, and Thomas Moeslund. Context-aware fusion of rgb and thermal imagery for traffic monitoring. Sensors, 16(11):1947, 2016. 1
work page 1947
-
[3]
Chris H Bahnsen and Thomas B Moeslund. Rain removal in traffic surveillance: Does it matter? IEEE Transactions on Intelligent Transportation Systems, 2018. 1, 2
work page 2018
-
[4]
Analysis of rain and snow in frequency space
Peter C Barnum, Srinivasa Narasimhan, and Takeo Kanade. Analysis of rain and snow in frequency space. International journal of computer vision, 86(2-3):256, 2010. 2
work page 2010
-
[5]
J ´er´emie Bossu, Nicolas Hautiere, and Jean-Philippe TAREL. Utilisation d’un mod `ele probabiliste d’orientation de seg- ments pour d ´etecter des hydrom ´et´eores dans des s ´equences vid´eo. In XXIIe colloque GRETSI (traitement du signal et des images), Dijon (FRA), 8-11 septembre 2009 . GRETSI, Groupe dEtudes du Traitement du Signal et des Images,
work page 2009
-
[6]
Rain or snow detection in image sequences through use of a histogram of orientation of streaks
J ´er´emie Bossu, Nicolas Hauti `ere, and Jean-Philippe Tarel. Rain or snow detection in image sequences through use of a histogram of orientation of streaks. International Journal of Computer Vision, 93(3):348–367, Jul 2011. 2, 3, 4, 5
work page 2011
-
[7]
A re- view of computer vision techniques for the analysis of urban traffic
Norbert Buch, Sergio A Velastin, and James Orwell. A re- view of computer vision techniques for the analysis of urban traffic. IEEE Transactions on Intelligent Transportation Sys- tems, 12(3):920–939, 2011. 1
work page 2011
-
[8]
Jared Casper, Jon Barker, and Bryan Catanzaro. Nvvl: Nvidia video loader. https://github.com/NVIDIA/ nvvl, 2018. 6
work page 2018
Show all 37 references
-
[9]
Error- optimized sparse representation for single image rain re- moval
Bo-Hao Chen, Shih-Chia Huang, and Sy-Yen Kuo. Error- optimized sparse representation for single image rain re- moval. Industrial Electronics, IEEE Transactions on , 64(8):6573–6581, 2017. 2
2017
-
[10]
Ten quick tips for machine learning in com- putational biology
Davide Chicco. Ten quick tips for machine learning in com- putational biology. BioData Mining, 10(1):35, Dec 2017. 7
2017
-
[11]
Measurements of rainfall rates from videos
Rong Dong, Juan Liao, Bo Li, Huiyu Zhou, and Danny Crookes. Measurements of rainfall rates from videos. In 2017 10th International Congress on Image and Signal Pro- cessing, BioMedical Engineering and Informatics (CISP- BMEI), pages 1–9. IEEE, 2017. 2, 3
2017
-
[12]
Photometric model of a rain drop
Kshitiz Garg and Shree K Nayar. Photometric model of a rain drop. In CMU Technical Report. 2003. 2
2003
-
[13]
Kshitiz Garg and Shree K. Nayar. Vision and rain. Inter- national Journal of Computer Vision, 75(1):3–27, Oct. 2007. 2
2007
-
[14]
Sam- pling errors of tipping-bucket rain gauge measurements
Emad Habib, Witold F Krajewski, and Anton Kruger. Sam- pling errors of tipping-bucket rain gauge measurements. Journal of Hydrologic Engineering, 6(2):159–166, 2001. 3
2001
-
[15]
Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh. Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet? In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 6546– 6555, 2018. 5
2018
-
[16]
Meteorological conditions processing for vision-based traffic monitoring
Nicolas Hautiere, Erwan Bigorgne, J ´er´emie Bossu, and Didier Aubert. Meteorological conditions processing for vision-based traffic monitoring. In The Eighth International Workshop on Visual Surveillance-VS2008, 2008. 2
2008
-
[17]
Advancing opportunistic sensing in hydrology: a novel ap- proach to measuring rainfall with ordinary surveillance cam- eras
Shijie Jiang, Vladan Babovic, Yi Zheng, and Jianzhi Xiong. Advancing opportunistic sensing in hydrology: a novel ap- proach to measuring rainfall with ordinary surveillance cam- eras. Water Resources Research, 2019. 3
2019
-
[18]
A novel tensor-based video rain streaks removal approach via utilizing discriminatively in- trinsic priors
Tai-Xiang Jiang, Ting-Zhu Huang, Xi-Le Zhao, Liang-Jian Deng, and Yao Wang. A novel tensor-based video rain streaks removal approach via utilizing discriminatively in- trinsic priors. In Computer Vision and Pattern Recognition, IEEE Conference on, 2017. 2
2017
-
[19]
Auto- matic single-image-based rain streaks removal via image decomposition
Li-Wei Kang, Chia-Wen Lin, and Yu-Hsiang Fu. Auto- matic single-image-based rain streaks removal via image decomposition. IEEE Transactions on Image Processing , 21(4):1742–1755, 2012. 2
2012
-
[20]
Video deraining and desnowing using temporal correlation and low-rank matrix completion
Jin-Hwan Kim, Jae-Young Sim, and Chang-Su Kim. Video deraining and desnowing using temporal correlation and low-rank matrix completion. IEEE Transactions on Image Processing, 24(9):2658–2670, 2015. 2
2015
-
[21]
Sin- gle image deraining: A comprehensive benchmark analysis
Siyuan Li, Iago Breno Araujo, Wenqi Ren, Zhangyang Wang, Eric K Tokuda, Roberto Hirata Junior, Roberto Cesar- Junior, Jiawan Zhang, Xiaojie Guo, and Xiaochun Cao. Sin- gle image deraining: A comprehensive benchmark analysis. arXiv preprint arXiv:1903.08558, 2019. 1
1903 arXiv
-
[22]
Erase or fill? deep joint recurrent rain removal and recon- struction in videos
Jiaying Liu, Wenhan Yang, Shuai Yang, and Zongming Guo. Erase or fill? deep joint recurrent rain removal and recon- struction in videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 3233– 3242, 2018. 2
2018
-
[23]
Pixel based temporal analysis using chromatic property for remov- ing rain from videos
Peng Liu, Jing Xu, Jiafeng Liu, and Xianglong Tang. Pixel based temporal analysis using chromatic property for remov- ing rain from videos. Computer and Information Science , 2(1):53, 2009. 2
2009
-
[24]
An optical disdrometer for measuring size and velocity of hydrometeors
Martin L ¨offler-Mang and J ¨urg Joss. An optical disdrometer for measuring size and velocity of hydrometeors. Journal of Atmospheric and Oceanic Technology, 17(2):130–139, 2000. 3
2000
-
[25]
Madsen, P.S
H. Madsen, P.S. Mikkelsen, D. Rosbjerg, and P. Harremos. Estimation of regional intensity-duration-frequency curves for extreme precipitation. Water Science and Technology , 37(11):29 – 36, 1998. Use of Historical Rainfall Series for Hydrological Modelling. 3
1998
-
[26]
Mikkelsen, H
P.S. Mikkelsen, H. Madsen, K. Arnbjerg-Nielsen, H.K. Jr- gensen, D. Rosbjerg, and P. Harremos. A rationale for using local and regional point rainfall data for design and analysis of urban storm drainage systems. Water Science and Tech- nology, 37(11):7 – 14, 1998. Use of Hist...
1998
-
[27]
Rasmussen
Jesper Ellerbæk Nielsen, Søren Liedtke Thorndahl, and Michael R. Rasmussen. Improving weather radar precipi- tation estimates by combining two types of radars. Atmo- spheric Research, 139(March):3645, 2014. 3
2014
-
[28]
Rasmussen
Jesper Ellerbæk Nielsen, Søren Liedtke Thorndahl, and Michael R. Rasmussen. A numerical method to generate high temporal resolution precipitation time series by com- bining weather radar measurements with a nowcast model. Atmospheric Research, 138(March):1–12, 2014. 3
2014
-
[29]
Automatic differentiation in pytorch
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Al- ban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In NIPS-W, 2017. 6
2017
-
[30]
Video desnowing and deraining based on ma- trix decomposition
Weihong Ren, Jiandong Tian, Zhi Han, Antoni Chan, and Yandong Tang. Video desnowing and deraining based on ma- trix decomposition. In Computer Vision and Pattern Recog- nition, IEEE Conference on, July 2017. 2
2017
-
[31]
Utilizing local phase information to remove rain from video
Varun Santhaseelan and Vijayan K Asari. Utilizing local phase information to remove rain from video. International Journal of Computer Vision, 112(1):71–89, 2015. 2
2015
-
[32]
Chris Stauffer and W. Eric L. Grimson. Learning patterns of activity using real-time tracking. IEEE Transactions on pattern analysis and machine intelligence , 22(8):747–757,
-
[33]
Learning spatiotemporal features with 3d convolutional networks
Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learning spatiotemporal features with 3d convolutional networks. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), ICCV ’15, pages 4489–4497, Washington, DC, USA, 201...
2015
-
[34]
D. Tran, H. Wang, L. Torresani, J. Ray, Y . LeCun, and M. Paluri. A closer look at spatiotemporal convolutions for ac- tion recognition. In 2018 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 6450–6459, June 2018. 5
2018
-
[35]
Video post processing: low-latency spatiotemporal approach for detection and re- moval of rain
AK Tripathi and S Mukhopadhyay. Video post processing: low-latency spatiotemporal approach for detection and re- moval of rain. IET Image Processing, 6(2):181–196, 2012. 2
2012
-
[36]
Non-local neural networks
Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaim- ing He. Non-local neural networks. CVPR, 2018. 5
2018
-
[37]
Deep joint rain detection and removal from a single image
Wenhan Yang, Robby T Tan, Jiashi Feng, Jiaying Liu, Zong- ming Guo, and Shuicheng Yan. Deep joint rain detection and removal from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition, pages 1357–1366, 2017. 2
2017
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.