REVIEW 3 major objections 4 minor 41 references
Shadow Removal via Shadow Image Decomposition
T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A shadow removal method decomposes each image into a relit layer and a soft matte, cutting shadow-area error on ISTD from 13.3 to 7.9 RMSE.
desk verdict A plausible shadow-removal method built on a classic linear illumination model and matting decomposition, but the headline 40% RMSE improvement rests on a color-corrected test set whose baseline numbers appear to come from the original, unadjusted ground truth. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing identity is the decomposition $I_{\text{shadow-free}} = I_{\text{shadow}} \cdot \alpha + I_{\text{relit}} \cdot (1-\alpha)$, with $I_{\text{relit}} = w \cdot I_{\text{shadow}} + b$. The shadow parameters $(w, b)$ are three-channel scale and offset values that model how shadows darken and color-shift pixels; SP-Net regresses them from the input image and mask. The matte $\alpha$ blends shadow and relit pixels per location, with $\alpha$ near 1 outside the shadow and near 0 in its umbra, and it is predicted by M-Net. This split lets network outputs be low-dimensional (six parameters plus a matte) instead of full images, which the paper argues avoids blur and artifacts.
What would settle it
Re-run the baseline methods (Gong et al. and Wang et al.) on the color-corrected ISTD test ground truth used in this paper and recompute shadow-area RMSE; if either baseline reaches 7.9 or lower, the state-of-the-art claim is not supported.
Extended reading notes
Core claim
The central claim is that a shadow image can be decomposed as $I_{\text{shadow-free}} = I_{\text{shadow}} \cdot \alpha + I_{\text{relit}} \cdot (1-\alpha)$, where $I_{\text{relit}} = w \cdot I_{\text{shadow}} + b$ and $\alpha$ is a per-pixel matte. The vectors $w$ and $b$ are assumed constant across the umbra for each RGB channel, capturing the linear relation between lit and shadowed intensities under direct plus ambient illumination. The paper trains SP-Net to regress $w$ and $b$ from the shadow image and mask, and M-Net to predict $\alpha$ from the image, relit image, and mask. With this two-network system the paper reports state-of-the-art RMSE of 7.9 on the shadow area of the ISTD test set, compared with 13.3 for the previous best, and 7.4 after augmenting training data with synthetic shadows generated by varying the recovered parameters.
Load-bearing premise
The comparison assumes that the published RMSE values of earlier methods were computed on the same color-corrected ground truth as the paper's numbers, but the paper does not show that the baselines were rerun on the corrected images.
Editorial extensions
If this is right
- If the decomposition is correct, shadow removal no longer requires a network to hallucinate full shadow-free images; predicting six illumination parameters and a matte suffices.
- Because the relit image is computed analytically from the parameters, the output resolution is limited only by the input, not by the network's output resolution.
- The same decomposition gives a shadow-editing tool: modifying $w$ and $b$ and recombining through Eq. 8 generates new shadow images of controllable strength, which the paper uses to augment training.
- The matte layer absorbs errors in the predicted parameters, because M-Net learns to down-weight pixels that SP-Net over- or under-relights.
- On the ISTD benchmark, the full system lowers shadow-area RMSE from 13.3 to 7.9, and to 7.4 with the augmented training set.
Reading between the lines
- The reported margin depends on the color-corrected ground truth being fair to all methods; if the published baseline numbers were computed on the original ground truth, the comparison is not apples-to-apples and the 40% figure would shrink. The paper does not show that baselines were rerun on the corrected images.
- The constant $(w, b)$ assumption should break on shadows that cross multiple materials with different spectral responses, since a single per-channel affine map cannot relight both; testing on such scenes would reveal the model's boundary.
- The same decomposition could be applied to other spatially varying illumination effects, such as specular highlights or vignetting, as long as a linear relighting model and a matte can be defined.
- The augmentation scheme suggests a controllable data-generation recipe for shadow removal, but the synthetic shadows share the same linear-model biases as the training data, so gains might not transfer to shadows that violate the model.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a shadow-removal framework built on a physical illumination model in which the shadow-free image is a channel-wise affine function of the shadow image, combined with a per-pixel shadow matte. Two deep networks, SP-Net and M-Net, are trained to predict the affine shadow parameters (w, b) and the matte layer α, and the output is formed by Eq. 5. The authors evaluate on the ISTD dataset, reporting RMSE 7.9 on the shadow area versus 13.3 for Gong et al., and further report 7.4 after augmenting the training set with synthetic shadow images generated by the model itself. The main claim is state-of-the-art shadow removal on ISTD.
Significance. If the comparison is made valid, the paper is a useful contribution: it combines a physically motivated low-dimensional illumination model with learned parameter and matte estimators, leading to a compact and interpretable decomposition (Eqs. 4–6). The augmentation idea in Sec. 5.3 is also interesting because it permits targeted synthesis of shadow variations without an external renderer. The paper clearly states its assumptions, including the color-correction protocol, and it provides an explicit failure-case discussion. The central architectural idea is sound and the reported internal ablations (SP-Net alone vs. SP+M-Net) support the value of the matte network. However, the headline quantitative claim is currently tied to a non-standard adjusted test set, and the paper does not provide the re-evaluations needed to make the numbers comparable.
major comments (3)
- [§5.2, Table 1] The claim that 'All numbers are computed on the adjusted testing images so that they are directly comparable' is not supported by the evidence in the paper. The values reported for Gong et al. (13.3) and Wang et al. (13.4) are identical to the numbers in the original publications, which evaluated on the unadjusted ISTD ground truth, and the paper does not report any re-run of these baselines on the corrected ground truth. In fact, Table 1 shows the Input Image row with Non-Shadow RMSE 2.6, which is exactly the value the correction is designed to produce on the non-shadow area, while Wang et al. is listed as 7.7 on the same column; a method that leaves the non-shadow area unchanged would score near 2.6 on the corrected ground truth, so the listed baseline number cannot have been computed on the corrected testing images. The headline 40% reduction from 13.3 to 7.9 is therefore not established unless every baseline is re-evaluated on the same corrected ground truth. I request that the authors either run all baseline methods on the corrected test set and report those numbers, or remove the direct comparison and restate the contribution as an evaluation under a modified protocol.
- [§5.1] The color-correction step changes the ground-truth images by a per-channel linear regression that maps the non-shadow area of the shadow-free image onto the shadow image. This is a legitimate response to the known color-inconsistency problem of ISTD, but it changes the evaluation metric itself, so RMSE values against the corrected ground truth are not comparable to RMSE values against the original ground truth. The paper should report results under both protocols (original and corrected) or, at minimum, provide the original-GT numbers for the proposed method so readers can assess the effect of the correction. Without this, the magnitude of the claimed improvement cannot be separated from the effect of modifying the ground truth.
- [§5.3, Table 2] The augmentation experiment is interesting, but the 7.4 result is only reported on the corrected test set. Since the synthetic images are generated using the same decomposition model that is being trained (Eqs. 8–9), it is possible that the augmented training set biases the model toward the specific form of the model's assumptions. The paper should report the augmented-model results on the original ISTD ground truth as well, and ideally on a second shadow-removal benchmark, to show that the improvement is not an artifact of the matched evaluation protocol.
minor comments (4)
- [Acknowledgments] The text 'National Geographic/Microsoft AI for Earth program' contains a typo: 'Neographic' should be 'Geographic'.
- [Eq. 7] Equation 7 divides by I_shadow − I_relit, which can be zero or near-zero for pixels where the shadow has no effect; the paper should state how such pixels are handled in the numerical computation of the training target for M-Net.
- [§5.1, §5.3] Both the color-correction regression and the synthetic-shadow generation use linear-regression and inverse operations that can produce out-of-range pixel values; the paper should specify whether clipping or other range enforcement is applied and whether this affects the reported metrics.
- [Fig. 8] The qualitative comparison figure would be easier to read if the shadow region were outlined or enlarged, since the differences between methods are mostly visible at boundaries and in the penumbra.
Circularity Check
No circularity: the shadow-free image is produced from network-predicted illumination parameters and a matte, supervised by held-out ISTD ground truth; comparison-protocol concerns are not circularity.
full rationale
The derivation is self-contained. The shadow-free output is computed by Eq. 5 from a shadow image, network-predicted shadow parameters (w,b), and a network-predicted matte alpha, where the relit image is defined by Eq. 6. The linear illumination model in Eq. 4 is an openly stated physical assumption rather than a restatement of the target output, and the target shadow-free image is external supervisory data from the ISTD dataset. SP-Net is trained with regression targets computed from training pairs, and M-Net is trained with a reconstruction loss equivalent to supervised matte estimation; both are standard supervised-learning setups, not fitted quantities later renamed as predictions. The augmentation procedure in Sec. 5.3 synthesizes additional training images from estimated decompositions and is a data-generation step, not a claim that a prediction reduces to its input. Self-citations, such as the authors' prior A+D Net work, appear only in related-work context and are not load-bearing for the central derivation. The concern that baseline RMSE values in Table 1 may not have been recomputed on the authors' color-adjusted ground truth is a potential evaluation-fairness issue, but it is not a circularity: no fitted parameter or definitional equation is being presented as an independent prediction.
Assumptions & free parameters
free parameters (3)
- augmentation factor k =
[0.8, 0.9, 1.1, 1.2]
- shadow mask threshold =
0.95
- shadow mask erosion =
5 pixels
assumptions (4)
- domain assumption The linear illumination relation Ishadow-free = w * Ishadow + b holds per color channel, and w,b are constant across the umbra.
- ad hoc to paper The test-set color inconsistency between shadow and shadow-free images can be removed by a per-channel linear regression on the non-shadow area.
- domain assumption The shadow matte alpha can be represented by a U-Net output conditioned on the shadow image, relit image, and binary mask.
- domain assumption The shadow detection network (Zhu et al.) fine-tuned on ISTD provides sufficiently accurate masks at test time.
Cite this review
Pith. "Pith review of Shadow Removal via Shadow Image Decomposition." pith.science (2026). https://pith.science/paper/5NPRC6YI
@misc{pith2026190808628,
author = {Pith},
title = {Pith review of: Shadow Removal via Shadow Image Decomposition},
year = {2026},
howpublished = {\url{https://pith.science/paper/5NPRC6YI}},
note = {Machine review of arXiv:1908.08628}
}
read the original abstract
We propose a novel deep learning method for shadow removal. Inspired by physical models of shadow formation, we use a linear illumination transformation to model the shadow effects in the image that allows the shadow image to be expressed as a combination of the shadow-free image, the shadow parameters, and a matte layer. We use two deep networks, namely SP-Net and M-Net, to predict the shadow parameters and the shadow matte respectively. This system allows us to remove the shadow effects on the images. We train and test our framework on the most challenging shadow removal dataset (ISTD). Compared to the state-of-the-art method, our model achieves a 40% error reduction in terms of root mean square error (RMSE) for the shadow area, reducing RMSE from 13.3 to 7.9. Moreover, we create an augmented ISTD dataset based on an image decomposition system by modifying the shadow parameters to generate new synthetic shadow images. Training our model on this new augmented ISTD dataset further lowers the RMSE on the shadow area to 7.4.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Harry G. Barrow and J. Martin Tenenbaum. Recovering in- trinsic scene characteristics from images. Computer Vision Systems, pages 3–26, 1978. 1, 2
work page 1978
-
[2]
Joachim M. Buhmann. Weakly supervised structured output learning for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, CVPR ’12, 2012. 7
work page 2012
-
[3]
Yung-Yu Chuang, Dan B Goldman, Brian Curless, David H. Salesin, and Richard Szeliski. Shadow matting and com- positing. ACM Transactions on Graphics , 22(3):494–500, July 2003. Sepcial Issue of the SIGGRAPH 2003 Proceed- ings. 1, 2, 3
work page 2003
-
[4]
R. Dennis Cook. Influential observations, high leverage points, and outliers in linear regression. Statistical Science, pages 393–397, 1986. 4
work page 1986
-
[5]
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei- Fei. ImageNet: A Large-Scale Hierarchical Image Database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2009. 4
work page 2009
-
[6]
Mark S. Drew. Recovery of chromaticity image free from shadows via illumination invariance. In In IEEE Workshop on Color and Photometric Methods in Computer Vision, ICCV03, pages 32–39, 2003. 2, 4
work page 2003
-
[7]
Rank-based camera spectral sensitivity estimation
Graham Finlayson, Maryam Mohammadzadeh Darrodi, and Michal Mackiewicz. Rank-based camera spectral sensitivity estimation. J. Opt. Soc. Am. A, 33(4):589–599, Apr 2016. 3
work page 2016
-
[8]
Entropy minimization for shadow removal
Graham Finlayson, Mark Drew, and Cheng Lu. Entropy minimization for shadow removal. International Journal of Computer Vision, 2009. 2, 4
work page 2009
Show all 41 references
-
[9]
Graham Finlayson and Mark S. Drew. 4-sensor camera cal- ibration for image representation invariant to shading, shad- ows, lighting, and specularities. In Proceedings of the Inter- national Conference on Computer Vision , volume 2, pages 473–480 vol.2, July 2001. 2
2001
-
[10]
Hordley, Cheng Lu, and M.S
Graham Finlayson, S.D. Hordley, Cheng Lu, and M.S. Drew. On the removal of shadows from images.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006. 1, 2
2006
-
[11]
Hordley, and Mark S
Graham Finlayson, Steven D. Hordley, and Mark S. Drew. Removing shadows from images. In Proceedings of the Eu- ropean Conference on Computer Vision , ECCV ’02, pages 823–836, London, UK, UK, 2002. Springer-Verlag. 2, 4
2002
-
[12]
Interactive removal and ground truth for difficult shadow scenes
Han Gong and Darren Cosker. Interactive removal and ground truth for difficult shadow scenes. J. Opt. Soc. Am. A, 33(9):1798–1811, 2016. 2, 4, 6, 7, 8
2016
-
[13]
Paired regions for shadow detection and removal
Ruiqi Guo, Qieyun Dai, and Derek Hoiem. Paired regions for shadow detection and removal. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012. 1, 2, 4, 6, 7, 8
2012
-
[14]
Tumblin, and L
Xiang Huang, Gang Hua, J. Tumblin, and L. Williams. What characterizes a shadow boundary under the sun and sky? In Proceedings of the International Conference on Computer Vision, 2011. 1, 4
2011
-
[15]
Image-to-image translation with conditional adver- sarial networks
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adver- sarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. 2, 3, 6
2017
-
[16]
Progressive growing of GANs for improved quality, stabil- ity, and variation
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of GANs for improved quality, stabil- ity, and variation. In International Conference on Learning Representations, 2018. 2
2018
-
[17]
Weakly labeling the antarctic: The penguin colony case
Hieu Le, Bento Goncalves, Dimitris Samaras, and Heather Lynch. Weakly labeling the antarctic: The penguin colony case. In The IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR) Workshops, June 2019. 7
2019
-
[18]
Yago Vicente, Vu Nguyen, Minh Hoai, and Dimitris Samaras
Hieu Le, Tomas F. Yago Vicente, Vu Nguyen, Minh Hoai, and Dimitris Samaras. A+D Net: Training a shadow detector with adversarial shadow attenuation. In Proceedings of the European Conference on Computer Vision, 2018. 1, 7
2018
-
[19]
Co-localization with category-consistent features and geodesic distance propagation
Hieu Le, Chen-Ping Yu, Gregory Zelinsky, and Dimitris Samaras. Co-localization with category-consistent features and geodesic distance propagation. In ICCV 2017 Workshop on CEFRL: Compact and Efficient Feature Representation and Learning in Computer Vision, 2017. 7
2017
-
[20]
Levin, D
A. Levin, D. Lischinski, and Y . Weiss. A closed-form solu- tion to natural image matting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(2):228–242, 2008. 2
2008
-
[21]
Fashion parsing with weak color-category labels
Si Liu, Jiashi Feng, Csaba Domokos, Hui Xu, Junshi Huang, Zhenzhen Hu, and Shuicheng Yan. Fashion parsing with weak color-category labels. IEEE Transactions on Multime- dia, 16:253–265, 2014. 7
2014
-
[22]
Weakly- supervised dual clustering for image semantic segmenta- tion
Yang Liu, Zechao Li, Jinhui Tang, and Hanqing Lu. Weakly- supervised dual clustering for image semantic segmenta- tion. 2013 IEEE Conference on Computer Vision and Pattern Recognition, pages 2075–2082, 2013. 7
2013
-
[23]
De- convolution and checkerboard artifacts
Augustus Odena, Vincent Dumoulin, and Chris Olah. De- convolution and checkerboard artifacts. Distill, 2016. 2
2016
-
[24]
Compositing digital images
Thomas Porter and Tom Duff. Compositing digital images. Proceedings of the ACM SIGGRAPH Conference on Com- puter Graphics, 18(3), January 1984. 2, 3
1984
-
[25]
Liangqiong Qu, Jiandong Tian, Shengfeng He, Yandong Tang, and Rynson W. H. Lau. Deshadownet: A multi-context embedding deep network for shadow removal. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. 1, 3
2017
-
[26]
Ronneberger, P.Fischer, and T
O. Ronneberger, P.Fischer, and T. Brox. U-net: Convo- lutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Im- age Computing and Computer Assisted Intervention, volume 9351 of LNCS, pages 234–241, 2015. 5
2015
-
[27]
Clustering-based shadow edge detection in a single color image
Wang Shiting and Zheng Hong. Clustering-based shadow edge detection in a single color image. In International Con- ference on Mechatronic Sciences, Electric Engineering and Computer, pages 1038–1041, Dec 2013. 1
2013
-
[28]
The shadow meets the mask: Pyramid-based shadow removal.Computer Graphics Forum, 27(2):577–586, April 2008
Yael Shor and Dani Lischinski. The shadow meets the mask: Pyramid-based shadow removal.Computer Graphics Forum, 27(2):577–586, April 2008. 1, 2, 3, 4
2008
-
[29]
Learning from sim- ulated and unsupervised images through adversarial training
Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Josh Susskind, Wenda Wang, and Russ Webb. Learning from sim- ulated and unsupervised images through adversarial training. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. 7
2016
-
[30]
Alvy Ray Smith and James F. Blinn. Blue screen matting. In Proceedings of the ACM SIGGRAPH Conference on Com- puter Graphics, 1996. 3
1996
-
[31]
Yago Vicente, Minh Hoai, and Dimitris Samaras
Tomas F. Yago Vicente, Minh Hoai, and Dimitris Samaras. Leave-one-out kernel optimization for shadow detection and removal. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 40(3):682–695, 2018. 2
2018
-
[32]
Yago Vicente, Le Hou, Chen-Ping Yu, Minh Hoai, and Dimitris Samaras
Tomas F. Yago Vicente, Le Hou, Chen-Ping Yu, Minh Hoai, and Dimitris Samaras. Large-scale training of shadow detec- tors with noisily-annotated shadow examples. In Proceed- ings of the European Conference on Computer Vision, 2016. 1, 3, 6
2016
-
[33]
Yago Vicente and Dimitris Samaras
Tomas F. Yago Vicente and Dimitris Samaras. Single im- age shadow removal via neighbor-based region relighting. In Proceedings of the European Conference on Computer Vi- sion Workshops, 2014. 2
2014
-
[34]
Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal
Jifeng Wang, Xiang Li, and Jian Yang. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition ,
-
[35]
High-resolution image syn- thesis and semantic manipulation with conditional gans
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High-resolution image syn- thesis and semantic manipulation with conditional gans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018. 2
2018
-
[36]
Digital compositing for film and video
Steve Wright. Digital compositing for film and video. In Focal Press, 2001. 3
2001
-
[37]
Aggregated residual transformations for deep neural networks
Saining Xie, Ross Girshick, Piotr Dollar, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. In The IEEE Conference on Computer Vi- sion and Pattern Recognition (CVPR), July 2017. 4
2017
-
[38]
Shadow removal using bilateral filtering
Qingxiong Yang, Kar Han Tan, and Narendra Ahuja. Shadow removal using bilateral filtering. IEEE Transactions on Im- age Processing, 21:4361–4368, 2012. 6, 7, 8
2012
-
[39]
Zhang, Q
L. Zhang, Q. Zhang, and C. Xiao. Shadow remover: Image shadow removal based on illumination recovering optimiza- tion. IEEE Transactions on Image Processing, 24(11), Nov
-
[40]
Colorful image colorization
Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. In ECCV, 2016. 2, 6
2016
-
[41]
Bidirectional feature pyramid network with recurrent attention residual modules for shadow detection
Lei Zhu, Zijun Deng, Xiaowei Hu, Chi-Wing Fu, Xuemiao Xu, Jing Qin, and Pheng-Ann Heng. Bidirectional feature pyramid network with recurrent attention residual modules for shadow detection. In Proceedings of the European Con- ference on Computer Vision, 2018. 4, 6
2018
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.