REVIEW 4 major objections 4 minor 30 references
Unsupervised Raindrop Removal from a Single Image using Conditional Diffusion Models
T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read DropWiper proposes to remove raindrops from a single image by generating pseudo-masks and filling them with a diffusion inpainter; its own experiments show the masks are too poor for effective background reconstruction.
desk verdict Honest negative result, but the method never works and Eq. 8 is backwards; desk-reject. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the masked diffusion inpainting update, Equation 8, which merges a forward-noised version of the known pixels with the model's reverse-diffusion prediction for the masked pixels: $$x_{t-1} = m \odot $x^{{\text{known}}$}_{t-1} + (1-m) \odot $x^{{\text{unknown}}$}_{t-1}.$$ This is the standard masked-inpainting trick from diffusion-based inpainting, letting a DDPM trained only on clean images fill arbitrary masked regions. The second essential component is the pseudo-mask itself: the paper experiments with residual masks (computed from the difference between the raindrop image and the clean image) and with a lightweight CNN detector trained on synthetic raindrops. The argument depends on the mask being accurate enough that the inpainter only has to fill true raindrop areas; when the mask fails, the whole pipeline fails.
What would settle it
Manually annotate raindrop masks on a sample of the Raindrop test set and run the same DDPM inpainter with those exact masks, comparing outputs to the ground-truth clean images. If reconstruction quality is still poor, Equation 8's update is the weak link; if quality is good, the detector's synthetic-to-real transfer is the weak link.
Extended reading notes
Core claim
The central claim, on the paper's own terms, is that raindrop removal from a single image can be reformulated as a conditional inpainting problem: detect the raindrops, then use a DDPM to regenerate the detected regions while preserving the known background. The mechanism is the masked inpainting update of Equation 8, which at each reverse step replaces the known pixels with their forward-diffused values and leaves the masked pixels to the denoiser. The paper further claims that the detector can be trained without real ground-truth masks by synthesizing raindrops with a non-parametric refraction model on Cityscapes images. In reporting, the authors find the detector overfits to synthetic data and produces masks that fail on the Raindrop dataset, and consequently that the background reconstruction step cannot be evaluated as intended. The paper's own conclusion is that the two-step architecture as implemented does not achieve effective raindrop removal.
Load-bearing premise
The load-bearing premise is that synthetic raindrops, rendered on Cityscapes by simulating how a water droplet refracts the background, are similar enough to real raindrops that a detector trained on them will produce masks usable for inpainting on the Raindrop dataset.
Editorial extensions
If this is right
- If the pipeline worked as claimed, raindrop removal would reduce to choosing a mask and letting a clean-image-trained diffusion model fill it, with no need for a GAN discriminator or paired clean images at inference time.
- The method would tolerate imperfect masks because the DDPM can reconstruct regions even when the masked areas are larger than the true raindrops.
- Training the detector on synthetic raindrops would remove the need for manually annotated raindrop masks, which the Raindrop dataset does not provide.
- The same masking trick could be reused for any single-image restoration task where the corruption can be localized, since the DDPM itself is task-agnostic.
Reading between the lines
- The paper's negative result suggests the bottleneck is the mask, not the inpainter: a better detector (fine-tuned on real raindrop images, or trained with domain adaptation) might make the same Equation 8 reconstruction effective.
- The residual-mask branch requires the clean image at test time, so it is not a genuine single-image method; the paper's comparison between residual and learned masks therefore does not directly test the unsupervised scenario it claims to address.
- A natural next step would be to evaluate the DDPM inpainter with ground-truth raindrop masks (manually annotated) on the Raindrop test set, isolating inpainter quality from mask quality.
- The synthetic-to-real failure is a useful data point for the broader deraining literature, where synthetic training data is common: the refraction model renders physically plausible drops, but the domain gap to real photographs remains large enough to break the downstream pipeline.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DropWiper, a two-step pipeline for removing raindrops from a single image: a lightweight CNN raindrop detector produces a pseudo-mask, and a DDPM-based inpainting module (following the RePaint-style blending of forward and reverse diffusion) reconstructs the background in masked regions. The detector is trained on synthetic raindrops rendered on Cityscapes via a refraction model, and the DDPM is trained on clean Raindrop-dataset images. The experiments compare two mask-generation approaches (residual masks from thresholding the difference between a raindrop image and its clean counterpart, and detector masks) and report that detector masks on the Raindrop dataset are of poor quality and that effective background reconstruction is not achieved.
Significance. The intended contribution is a diffusion-based alternative to GAN-based raindrop removal, combining a small detector with a pre-trained DDPM. This idea is not without interest, but the manuscript does not deliver a working method: no quantitative evaluation is reported, the only mask branch that the paper itself considers 'more solid' requires the ground-truth clean image, and the paper's own conclusion states that effective background reconstruction was not achieved. As a result, the claimed novelty is not demonstrated, and the significance of the work as presented is low.
major comments (4)
- [Section V and Section IV-C3] The paper's central claim of raindrop removal is directly contradicted by its own conclusions. Section IV-C3 states that the detector masks on the Raindrop dataset are 'poor quality' and attributes this to domain shift, and Section V states 'we are not able to perform an effective background reconstruction' and that forcing reconstruction on poor masks leads to 'unstable reconstructions'. The abstract and introduction claim a novel technique for raindrop removal from a single image, but the experiments section offers no end-to-end restoration result, no PSNR/SSIM or any other quantitative metric, and no figure showing an actual raindrop-removed output compared with ground truth. The paper as written does not support its title claim.
- [Section III-A1, Eqs. (4)-(7)] The residual mask branch uses the ground-truth clean image B to compute R = A - B (or variants). This is a load-bearing limitation: for a single-image, unsupervised method, B is not available at test time. The residual masks therefore cannot serve as an unsupervised mask generator for the method; they are usable only in a paired-data setting, which contradicts the stated task and also undermines the conclusion in Section V that residual masks are 'more solid' than detector masks, since that comparison is not an independent test of a deployable pipeline.
- [Section III-B, Eq. (8)] Equation (8) inverts the mask definitions stated just above it. The text defines m ⊙ x as 'the missing pixels' and (1 - m) ⊙ x as 'the known pixels', but Eq. (8) computes x_{t-1} = m ⊙ x_known + (1 - m) ⊙ x_unknown. With the stated definitions, the known-pixel term is multiplied by the raindrop mask and the unknown term by its complement, which is the opposite of the standard RePaint-style harmonic update. As written, this equation would corrupt known pixels and would not restore masked regions. This is not a mere presentation typo: it is the core reconstruction equation of the proposed method, and the manuscript's own definitions render it internally inconsistent.
- [Section IV-C and Figure 10] No quantitative evaluation is reported anywhere in the experiments. The only results shown are visual: residual masks (Fig. 6), synthetic raindrop samples (Fig. 7), training/validation losses (Fig. 8), detector masks judged to be poor (Fig. 9), and DDPM samples of clean images (Fig. 10). Figure 10 does not show a single raindrop-removal result. Without any quantitative comparison against the existing GAN-based baselines cited in Section II-A, or even against the ground-truth clean images the paper says it plans to use, the claimed advantage over prior work is unsubstantiated.
minor comments (4)
- [Section III-A1] The text says 'The notations H and W represent the image's height and weight respectively'; 'weight' should be 'width'.
- [Section III-B] The notation in the background-reconstruction section is confusing: x is introduced as the ground-truth image, but Eq. (8) and the surrounding text use x_{t-1} both as the partially denoised latent and as the image being constructed, and the sentence about 'alter the known pixels in (1-m) ⊙ x_{t-1}' is hard to follow. The relationship between the forward-diffusion sample x_known and the reverse-diffusion sample x_unknown should be stated more explicitly.
- [Title and Section II-B] The method is called 'conditional diffusion' but no conditioning mechanism beyond the mask-blending is described; the reconstruction module is essentially the RePaint inpainting procedure of [5]. The paper would be more accurate to describe this as an application of RePaint rather than a new conditional diffusion model.
- [Section V] The phrase 'residual mask generators provide more solid masks than the raindrop detection model' is misleading because the residual masks are computed with access to the ground-truth clean image, whereas the detector is evaluated on unseen raindrop images. The comparison is therefore not apples-to-apples.
Circularity Check
Reconstruction equation injects ground truth and residual masks are computed from the target, so the claimed unsupervised raindrop removal is not an independent prediction.
-
self definitional
[Section III-A1, Eq. (4) and Section V]
"We are given two images A,B ... where A is an image with raindrops while B is the corresponding clean image. ... R = A − B (4) ... From our experiments, we found that the residual mask generators provide more solid masks than the raindrop detection model."
The residual pseudo-mask is computed by thresholding the difference between the raindrop image A and the clean target B. Thus the mask is, by construction, a function of the very ground-truth image the method is supposed to recover. Concluding that these residual masks are 'more solid' than the detector masks is therefore not an independent evaluation: the residual branch cannot be used in a genuine single-image unsupervised pipeline because it requires the clean image at inference. The mask quality is self-defined by the target, not measured against it independently.
-
self definitional
[Section III-B, Eq. (8)]
"we denote x as the ground-truth image, m⊙x as the missing pixels, and (1−m)⊙x as the known pixels. ... xt−1 = m ⊙ xknown t−1 + (1−m) ⊙ xunknown t−1 (8) where xknown t−1 ∼ N (√ αx0, (1−αt)I)"
Eq. (8) constructs the denoised latent from x0, which the paper explicitly defines as the ground-truth clean image, and places this known/ground-truth term on m, the mask previously defined as the missing raindrop pixels. Consequently the 'restored' masked areas are filled from the target by construction; if the mask and complement were swapped, the known-pixel term would still use ground-truth x0 rather than the observed raindrop input. Either way, the background reconstruction is not an unsupervised prediction from a single raindrop image but a direct injection of the target into the output. The paper's own Section V admission that no effective reconstruction was achieved further confirms that no independent, non-circular result is demonstrated.
full rationale
The paper contains no problematic self-citation chain: it cites prior work on synthetic raindrops (Hao et al.) and DDPM inpainting (RePaint, Dhariwal et al.) as external support, and the detector branch is trained independently on synthetic Cityscapes data. However, two load-bearing components reduce by construction to the target. First, the residual mask branch computes R = A − B from the ground-truth clean image, so any mask quality claim for that branch is self-referential rather than an independent measure. Second, Eq. (8) defines the background reconstruction using x0, explicitly the ground-truth image, in the known-pixel term, and assigns that term to the raindrop mask m; a successful reconstruction under this equation would be a copy of the target in the masked regions. The paper also honestly states in Section V that poor detector masks prevented effective background reconstruction, so no external quantitative evaluation rescues the central claim. These issues make the central derivation partially circular, though the paper stops short of reporting a successful forced result. Score 6 reflects that one or more claimed predictions reduce by construction while the paper itself acknowledges the pipeline's failure.
Assumptions & free parameters
free parameters (2)
- Residual mask threshold =
30, 80, 120, 200
- Refraction model camera parameters =
Unspecified
assumptions (4)
- domain assumption Rain streaks are absent from all input images.
- domain assumption Synthetic raindrops rendered with the refraction model on Cityscapes are a sufficient proxy for real raindrops in the Raindrop dataset.
- ad hoc to paper The RePaint-style blending in Equation 8 correctly combines forward and reverse diffusion to restore only masked pixels.
- domain assumption A DDPM trained on 128x128 center-cropped clean Raindrop images learns a distribution sufficient to inpaint masked background regions.
Cite this review
Pith. "Pith review of Unsupervised Raindrop Removal from a Single Image using Conditional Diffusion Models." pith.science (2026). https://pith.science/paper/VV3N7C46
@misc{pith2026250508190,
author = {Pith},
title = {Pith review of: Unsupervised Raindrop Removal from a Single Image using Conditional Diffusion Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/VV3N7C46}},
note = {Machine review of arXiv:2505.08190}
}
read the original abstract
Raindrop removal is a challenging task in image processing. Removing raindrops while relying solely on a single image further increases the difficulty of the task. Common approaches include the detection of raindrop regions in the image, followed by performing a background restoration process conditioned on those regions. While various methods can be applied for the detection step, the most common architecture used for background restoration is the Generative Adversarial Network (GAN). Recent advances in the use of diffusion models have led to state-of-the-art image inpainting techniques. In this paper, we introduce a novel technique for raindrop removal from a single image using diffusion-based image inpainting.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Deep learning for seeing through window with raindrops
Yuhui Quan, Shijie Deng, Yixin Chen, and Hui Ji. Deep learning for seeing through window with raindrops. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 2463–2471, 2019
work page 2019
-
[2]
Attentive generative adversarial network for raindrop removal from a single image
Rui Qian, Robby T Tan, Wenhan Yang, Jiajun Su, and Jiaying Liu. Attentive generative adversarial network for raindrop removal from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2482–2491, 2018
2018
-
[3]
Weiss, Niru Maheswaranathan, and Surya Ganguli
Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep Unsupervised Learning using Nonequilibrium Thermodynamics, November 2015. arXiv:1503.03585 [cond-mat, q-bio, stat]
arXiv 2015
-
[4]
Diffusion models beat gans on image synthesis
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems , 34:8780–8794, 2021
2021
-
[5]
RePaint: Inpainting using Denoising Diffusion Probabilistic Models
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. RePaint: Inpainting using Denoising Diffusion Probabilistic Models. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11451–11461, New Orleans, LA, USA, June 2022. IEEE
work page 2022
-
[6]
The cityscapes dataset for semantic urban scene understanding
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016
2016
-
[7]
Changbao Xu, Jipu Gao, Qi Wen, and Bo Wang. Generative adversarial network for image raindrop removal of transmission line based on unmanned aerial vehicle inspection. Wireless Communications and Mobile Computing , 2021, 2021
work page 2021
-
[8]
Raindrop-aware gan: Unsupervised learning for raindrop-contaminated coastal video enhancement
Jinah Kim, Dong Huh, Taekyung Kim, Jaeil Kim, Jeseon Yoo, and Jae-Seol Shim. Raindrop-aware gan: Unsupervised learning for raindrop-contaminated coastal video enhancement. Remote Sensing, 12(20):3461, 2020
work page 2020
Show all 30 references
-
[9]
Raindrop removal from a single image using a two-step generative adversarial network
Haiying Xia, Yang Lan, Shuxiang Song, and Haisheng Li. Raindrop removal from a single image using a two-step generative adversarial network. Signal, Image and Video Processing , 16(3):677–684, April 2022
2022
-
[10]
Unfairgan: An enhanced generative adversarial network for raindrop removal from a single image
Duc Manh Nguyen and Sang-Woong Lee. Unfairgan: An enhanced generative adversarial network for raindrop removal from a single image. arXiv preprint arXiv:2110.05523 , 2021
2021 arXiv
-
[11]
Cycle-spinning gan for raindrop removal from images
¨Ulk¨u Uzun and Alptekin Temizel. Cycle-spinning gan for raindrop removal from images. In 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) , pages 1–6. IEEE, 2019
2019
-
[12]
Restoring raindrops using attentive generative adversarial networks
Hee-Deok Yang. Restoring raindrops using attentive generative adversarial networks. Applied Sciences, 11(15):7034, 2021
2021
-
[13]
Selective generative adversarial network for raindrop removal from a single image
Mingwen Shao, Le Li, Hong Wang, and Deyu Meng. Selective generative adversarial network for raindrop removal from a single image. Neurocomputing, 426:265–273, 2021
2021
-
[14]
Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models, November 2022
Ozan ¨Ozdenizci and Robert Legenstein. Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models, November 2022. arXiv:2207.14626 [cs]
2022 arXiv
-
[15]
Denoising diffusion probabilistic models
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020
2020
-
[16]
High-resolution image synthesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022
2022
-
[17]
Photorealistic text-to-image diffusion models with deep language understanding
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv...
2022 arXiv
-
[18]
Hierarchical text-conditional image generation with clip latents
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 , 2022
2022 arXiv
-
[19]
Anoddpm: Anomaly detection with denoising diffusion probabilistic models using simplex noise
Julian Wyatt, Adam Leach, Sebastian M Schmon, and Chris G Willcocks. Anoddpm: Anomaly detection with denoising diffusion probabilistic models using simplex noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 650–656, 2022
2022
-
[20]
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. ArXiv, abs/1502.03167, 2015
2015 arXiv
-
[21]
Empirical evaluation of rectified activations in convolutional network
Bing Xu, Naiyan Wang, Tianqi Chen, and Mu Li. Empirical evaluation of rectified activations in convolutional network. ArXiv, abs/1505.00853, 2015
2015 arXiv
-
[22]
Deep Residual Learning for Image Recognition, December
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition, December
-
[23]
Halimeh and Martin Roser
Jad C. Halimeh and Martin Roser. Raindrop detection on car windshields using geometric-photometric environment construction and intensity-based correlation. 2009 IEEE Intelligent Vehicles Symposium , pages 610–615, 2009
2009
-
[24]
Realistic Modeling of Water Droplets for Monocular Adherent Raindrop Recognition Using B ´ezier Curves
Martin Roser, Julian Kurz, and Andreas Geiger. Realistic Modeling of Water Droplets for Monocular Adherent Raindrop Recognition Using B ´ezier Curves. In Reinhard Koch and Fay Huang, editors, Computer Vision – ACCV UNSUPERVISED RAINDROP REMOV AL FROM A SINGLE IMAGE USING CONDI...
2010
-
[25]
Rendering Physically Correct Raindrops on Windshields for Robustness Verification of Camera-based Object Recognition
Alexander von Bernuth, Georg V olk, and Oliver Bringmann. Rendering Physically Correct Raindrops on Windshields for Robustness Verification of Camera-based Object Recognition. 2018 IEEE Intelligent Vehicles Symposium (IV) , pages 922–927, 2018
2018
-
[26]
Tan, Rei Kawakami, Yasuhiro Mukaigawa, and Katsushi Ikeuchi
Shaodi You, Robby T. Tan, Rei Kawakami, Yasuhiro Mukaigawa, and Katsushi Ikeuchi. Waterdrop Stereo. ArXiv, abs/1604.00730, 2016
2016 arXiv
-
[27]
Learning From Synthetic Photorealistic Raindrop for Single Image Raindrop Removal
Zhixiang Hao, Shaodi You, Yu Li, Kunming Li, and Feng Lu. Learning From Synthetic Photorealistic Raindrop for Single Image Raindrop Removal. In 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pages 4340–4349, Seoul, Korea (South), October 2019. IEEE
2019
-
[28]
PyTorch Lightning, 3 2019
William Falcon and The PyTorch Lightning team. PyTorch Lightning, 3 2019
2019
-
[29]
Decoupled weight decay regularization
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2017
2017
-
[2015]
arXiv:1512.03385 [cs]
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.