REVIEW 4 major objections 4 minor 1 cited by
Regional Attention for Shadow Removal
T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Shadow removal works best when each shadow region reads its immediate non-shadow neighbors, and a 5.2M-parameter network implementing this idea reaches 40.73 dB PSNR on ISTD+ shadow regions, beating all prior models with far less compute.
desk verdict A workmanlike shadow-removal paper with a genuine efficiency-accuracy trade-off, undermined a bit by test-set tuning and missing error bars. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The regional attention module (RAM) carries the argument: for each token $i$ on the feature map, it computes dot-product attention between the query $Q_i$ and the keys of the $r$ tokens in a surrounding region, adds a relative position bias, and applies a softmax-scaled weighted sum over the corresponding values (Eqs. 3-5). With region size 11 and dilation factor 2, the effective receptive field expands without increasing parameters or FLOPs, following the dilated-neighborhood idea. RAM replaces the window-attention or self-attention blocks used by prior shadow-removal transformers, and is paired with a Squeeze-and-Excitation-style channel attention module so that global channel statistics are still captured cheaply. The loss is a weighted combination of Charbonnier content loss and VGG-19 perceptual loss.
What would settle it
Generate paired shadow/shadow-free images in which the same local neighborhood around a shadow is rendered under two different distant lighting conditions, so the correct unshadowed appearance differs while the nearby context is identical. If RASM cannot reproduce the two different ground truths, the proximity assumption is false. A complementary test: in ISTD+ test images, corrupt the immediate non-shadow surround and measure how much RASM's shadow-region PSNR drops; a small drop would indicate that nearby context is not actually the deciding signal.
Extended reading notes
Core claim
The central discovery is that shadow restoration is best framed as a region-context problem, not a global-context problem. RASM treats the shadow mask as a guide and lets every shadowed token attend to the tokens in its immediate neighborhood using a regional attention module with region size 11 and dilation 2; the surrounding non-shadow information is what reconstructs the shadowed pixels. On ISTD+, this yields 40.73 dB PSNR and 0.993 SSIM in the shadow region, surpassing the previous SOTA by 0.90 dB and also leading in all-image PSNR (36.16 dB) and RMSE (2.53). On SRD, the model achieves the best all-image RMSE (3.37) and the best non-shadow-region metrics, though its shadow-region PSNR (37.91 dB) trails two competitors, which the authors attribute to stricter adherence to the predicted shadow mask. The same network is an order of magnitude smaller than diffusion-based alternatives, using 5.2M parameters and 25.2 GFLOPs, so the paper's claim is that accuracy and efficiency are jointly achievable once attention is aligned with the geometry of shadows.
Load-bearing premise
The model rests on the premise that the information needed to restore a shadowed area is concentrated in the non-shadowed pixels immediately surrounding it and becomes less important with distance; if shadow appearance is governed by global illumination or distant scene context, regional attention will miss the deciding cues.
Editorial extensions
If this is right
- If regional attention is the right inductive bias, shadow-removal systems can drop global self-attention and diffusion components entirely, making real-time on-device shadow removal practical.
- The ISTD+ numbers imply that a 5.2M-parameter model can outperform a 55.2M-parameter diffusion model by 0.90 dB in shadow regions, so the task does not require large models once the attention geometry matches the corruption pattern.
- The window-vs-regional ablation (40.06 vs 40.73 dB) shows the gain comes from aligning attention with shadow boundaries, not from added capacity, predicting that other mask-guided restoration tasks will benefit from the same substitution.
- The dilation experiment shows receptive-field range can be enlarged at zero FLOPs cost by increasing dilation, decoupling how far the model looks from how much it computes.
Reading between the lines
- The proximity hypothesis could be tested directly: corrupt or erase the immediate non-shadow surround in ISTD+ test images and measure the PSNR drop; a large drop would confirm that nearby context is the load-bearing signal, while a small drop would suggest global cues do more work than the paper assumes.
- Because the method needs a shadow mask at inference, its practical reach depends on mask quality; a mask-free variant or joint detection module would be a natural extension the paper does not address.
- The efficiency claim is metered at 256x256; at higher resolutions the fixed 11x11 region covers a smaller fraction of the scene, so resolution scaling behaviour is an open question.
- Regional attention may transfer to other partial-corruption tasks such as reflection removal or object inpainting, since those share the property that the correct output is locally inferable from clean surroundings.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a lightweight shadow removal model, RASM, built around a regional attention mechanism that lets each shadowed token attend to its surrounding non-shadowed tokens, with a dilation option to enlarge the receptive field. The architecture is a U-shaped encoder-decoder with channel attention and a Regional Attention Module at the bottleneck, trained with Charbonnier and VGG-based perceptual losses. The authors evaluate on ISTD+ and SRD, reporting state-of-the-art PSNR/SSIM/RMSE on ISTD+ and strong results on SRD, with 5.2M parameters and 25.2 GFLOPs. The central claim, stated in the abstract, is that the method delivers superior performance over other state-of-the-art models in both accuracy and efficiency.
Significance. If the empirical claims hold, the paper makes a useful contribution: it identifies a simple inductive bias for shadow removal—information from adjacent non-shadow regions is especially informative—and packages it in a lightweight, efficient network. The strengths include publicly available code, comparisons on two widely used datasets with a consistent evaluation protocol, and a clear ablation isolating the regional attention mechanism versus window attention. The regional attention design is plausible and the qualitative attention visualizations support the mechanism's behavior. However, the headline "superior performance" claim is currently not fully supported: the ISTD+ results appear to be selected through test-set hyperparameter tuning, no uncertainty measures are reported, and on SRD the shadow-region accuracy trails two prior methods. The significance of the contribution is therefore conditional on additional validation that the reported margins are not artifacts of selection bias.
major comments (4)
- [Sec. 4.3, Tab. 5] The final model configuration (region size r=11, dilation d=2) is chosen by evaluating variants on the ISTD+ test set, and the same test set is used to report the headline results in Tab. 1. The reported 40.73 dB PSNR is thus a selected maximum over multiple test-set evaluations, which introduces optimistic bias and can inflate the margin over ShadowFormer and ShadowDiffusion. Please either select hyperparameters on a held-out validation split (or via cross-validation) and then evaluate a single configuration on the test set, or report the selection procedure explicitly and provide significance testing, such as paired bootstrap confidence intervals over images, for the key comparisons in Tab. 1.
- [Sec. 4.2.1, Tab. 2] The abstract and introduction claim superior performance over state-of-the-art models, but on the SRD dataset the shadow-region PSNR of RASM (37.91 dB) is below ShadowDiffusion (38.72 dB) and Li et al. (39.33 dB), and the paper's own text acknowledges this gap. The superiority claim is therefore dataset- and metric-dependent; please scope the claim to the settings in which it holds, or provide a well-defined multi-objective criterion (e.g., Pareto dominance over accuracy and efficiency) under which RASM is superior on both datasets.
- [Sec. 4.2.1, Tab. 3] The efficiency comparison does not fully support the claim of "superior performance in terms of accuracy and efficiency": BMNet has 0.4M parameters and 11.6 GFLOPs, whereas RASM has 5.2M parameters and 25.2 GFLOPs, so RASM is not the most efficient among the compared methods. The efficiency advantage is relative mainly to large diffusion and transformer models. Please present a Pareto-style analysis of accuracy versus FLOPs/parameters, or explicitly characterize the accuracy-efficiency trade-off that RASM achieves, rather than stating a general efficiency superiority.
- [Sec. 4.2, Tabs. 1 and 2] No error bars, standard deviations, or multiple-seed results are reported for any configuration. Some of the reported margins are very small in absolute terms (e.g., non-shadow PSNR 39.23 vs. 39.15 for Liu et al. on ISTD+; SSIM 0.993 vs. 0.992), so without uncertainty quantification the reader cannot assess whether these differences are statistically meaningful. Please report at least three independent training runs with mean and standard deviation for the main comparisons, or use paired statistical tests if single-seed evaluation is standard for the dataset.
minor comments (4)
- [Sec. 3.2.2, Eq. (5)] Equation (5) is ambiguous about the scope of the softmax: it should read softmax(A_i^r / sqrt(d)) V_i^r, with parentheses around the softmax argument, rather than softmax(A_i^r / sqrt(d) V_i^r).
- [Sec. 4.1] The text states "AdamW optimizer with the momentum as (0.9, 0.999)"; AdamW uses betas, not momentum, so please rename this to betas for accuracy.
- [Sec. 5] In the concluding remarks, "ISRD+" should be "ISTD+".
- [Sec. 4.2.1] The sentence "The competitors are consist of 11 methods" is ungrammatical; please change to "The competitors consist of 11 methods."
Circularity Check
Test-set hyperparameter selection inflates the ISTD+ SOTA claim; no definitional circularity in the architecture.
-
fitted input called prediction
[Sec. 4.3, Table 5 (hyperparameter search) vs Sec. 4.2.1, Table 1 (headline SOTA comparison)]
"To balance performance and computational complexity, we choose a regional attention size of 11×11 and a dilation rate of 2 for our model."
The same ISTD+ test set is used both to select the regional-attention hyperparameters and to produce the headline SOTA results. Table 5 searches region size and dilation by ALL PSNR on ISTD+, marking 11×11 with d=2 (36.16 dB) as the final choice; Table 1 then reports that same configuration as 'Ours' with 40.73 dB shadow-region PSNR and 36.16 dB ALL PSNR. The reported superiority over ShadowFormer (39.67 dB) is therefore the maximum over the searched variants rather than an unbiased evaluation of a fixed model. This is test-set fitting rather than independent confirmation, and no held-out validation or error bars are provided. The architectural derivation itself is not definitionally circular, and the SRD results and efficiency numbers remain partly independent.
full rationale
The paper's derivation chain is empirical, not formal: regional attention is an architectural proposal motivated by the proximity assumption, and the reported metrics are measurements rather than quantities derived from the assumption. There is no self-citation chain, no imported uniqueness theorem, and no equation in which an output quantity is defined as the input. The single significant circularity concern is the ISTD+ evaluation protocol: the hyperparameters r and d are tuned on the same test set used to claim SOTA, so the 40.73 dB figure is optimistically biased. Because the SRD benchmark, the efficiency comparison, and the qualitative results are independent of that particular choice, the central contribution still has independent content. Overall score 4 reflects this partial, evaluation-level circularity, not definitional circularity.
Assumptions & free parameters
free parameters (6)
- region_size r =
11
- dilation_factor d =
2
- embedding_dim C =
32
- encoder_depth L =
3
- loss weights alpha1, alpha2 =
0.001, 1
- perceptual layer weights wi =
0.1, 0.1, 1, 1, 1
assumptions (5)
- domain assumption The most useful context for restoring a shadowed pixel is the non-shadow area adjacent to it; importance increases with proximity.
- domain assumption Shadow masks are available at test time and are accurate enough to guide attention.
- domain assumption Benchmark numbers from prior papers are directly comparable to the authors' re-implementations under the same protocol.
- domain assumption Neighborhood attention (NAT) and dilated neighborhood attention (DiNAT) are reliable building blocks for image restoration.
- standard math VGG-19 pretrained features provide a suitable perceptual loss for shadow removal.
Cite this review
Pith. "Pith review of Regional Attention for Shadow Removal." pith.science (2026). https://pith.science/paper/C7E67ECD
@misc{pith2026241114201,
author = {Pith},
title = {Pith review of: Regional Attention for Shadow Removal},
year = {2026},
howpublished = {\url{https://pith.science/paper/C7E67ECD}},
note = {Machine review of arXiv:2411.14201}
}
read the original abstract
Shadow, as a natural consequence of light interacting with objects, plays a crucial role in shaping the aesthetics of an image, which however also impairs the content visibility and overall visual quality. Recent shadow removal approaches employ the mechanism of attention, due to its effectiveness, as a key component. However, they often suffer from two issues including large model size and high computational complexity for practical use. To address these shortcomings, this work devises a lightweight yet accurate shadow removal framework. First, we analyze the characteristics of the shadow removal task to seek the key information required for reconstructing shadow regions and designing a novel regional attention mechanism to effectively capture such information. Then, we customize a Regional Attention Shadow Removal Model (RASM, in short), which leverages non-shadow areas to assist in restoring shadow ones. Unlike existing attention-based models, our regional attention strategy allows each shadow region to interact more rationally with its surrounding non-shadow areas, for seeking the regional contextual correlation between shadow and non-shadow areas. Extensive experiments are conducted to demonstrate that our proposed method delivers superior performance over other state-of-the-art models in terms of accuracy and efficiency, making it appealing for practical applications.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
DenseSR: Image Shadow Removal as Dense Prediction
DenseSR uses depth, normal, and DINO priors plus a split smoothing/detail decoder to remove shadows from single images, reporting SOTA on five benchmarks.
Reference graph
Works this paper leans on
-
[1]
Zipei Chen, Chengjiang Long, Ling Zhang, and Chunxia Xiao. 2021. Canet: A context-aware network for shadow removal. In ICCV. 4743–4752
work page 2021
-
[2]
Xiaodong Cun, Chi-Man Pun, and Cheng Shi. 2020. Towards Ghost-Free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN. In AAAI. AAAI Press, 10680–10687
work page 2020
-
[3]
Bin Ding, Chengjiang Long, Ling Zhang, and Chunxia Xiao. 2019. Argan: Atten- tive recurrent generative adversarial network for shadow detection and removal. In ICCV. 10213–10222
work page 2019
-
[4]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In ICLR. OpenReview.net
work page 2021
-
[5]
Graham D. Finlayson, Mark S. Drew, and Cheng Lu. 2009. Entropy Minimization for Shadow Removal. IJCV 85, 1 (2009), 35–57
work page 2009
-
[6]
Graham D. Finlayson, Steven D. Hordley, and Mark S. Drew. 2002. Removing Shadows from Images. In ECCV, Vol. 2353. 823–836
work page 2002
-
[7]
Graham D. Finlayson, Steven D. Hordley, Cheng Lu, and Mark S. Drew. 2006. On the Removal of Shadows from Images. IEEE TPAMI 28, 1 (2006), 59–68
work page 2006
-
[8]
Lan Fu, Changqing Zhou, Qing Guo, Felix Juefei-Xu, Hongkai Yu, Wei Feng, Yang Liu, and Song Wang. 2021. Auto-exposure fusion for single-image shadow removal. In CVPR. 10571–10580
work page 2021
Show all 41 references
-
[9]
Han Gong and Darren Cosker. 2014. Interactive Shadow Removal and Ground Truth for Variable Scene Categories. In BMVC
2014
-
[10]
Lanqing Guo, Siyu Huang, Ding Liu, Hao Cheng, and Bihan Wen. 2023. Shadow- Former: Global Context Helps Image Shadow Removal. In AAAI
2023
-
[11]
Lanqing Guo, Chong Wang, Wenhan Yang, Siyu Huang, Yufei Wang, Hanspeter Pfister, and Bihan Wen. 2023. Shadowdiffusion: When degradation prior meets diffusion model for shadow removal. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 14049–14058
2023
-
[12]
Ruiqi Guo, Qieyun Dai, and Derek Hoiem. 2012. Paired regions for shadow detection and removal. IEEE TPAMI 35, 12 (2012), 2956–2967
2012
-
[13]
Ali Hassani and Humphrey Shi. 2022. Dilated Neighborhood Attention Trans- former. (2022). arXiv:2209.15001 [cs.CV] https://arxiv.org/abs/2209.15001
2022 arXiv
-
[14]
Ali Hassani, Steven Walton, Jiachen Li, Shen Li, and Humphrey Shi. 2023. Neigh- borhood Attention Transformer. In CVPR. IEEE, 6185–6194
2023
-
[15]
Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-Excitation Networks. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . https: //doi.org/10.1109/cvpr.2018.00745
2018
-
[16]
Xiaowei Hu, Chi-Wing Fu, Lei Zhu, Jing Qin, and Pheng-Ann Heng. 2020. Direction-Aware Spatial Context Features for Shadow Detection and Removal. IEEE TPAMI 42, 11 (2020), 2795–2808
2020
-
[17]
Xiaowei Hu, Yitong Jiang, Chi-Wing Fu, and Pheng-Ann Heng. 2019. Mask- ShadowGAN: Learning to remove shadows from unpaired data. In ICCV. 2472– 2481
2019
-
[18]
Yeying Jin, Ruoteng Li, Wenhan Yang, and Robby T. Tan. 2023. Estimating Reflectance Layer from a Single Image: Integrating Reflectance Guidance and Shadow/Specular Aware Learning. In AAAI. AAAI Press, 1069–1077
2023
-
[19]
Yeying Jin, Aashish Sharma, and Robby T Tan. 2021. DC-ShadowNet: Single- Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network. In CVPR. 5027–5036
2021
-
[20]
Yeying Jin, Wei Ye, Wenhan Yang, Yuan Yuan, and Robby T. Tan. 2024. DeS3: Adaptive Attention-Driven Self and Soft Shadow Removal Using ViT Similarity. In AAAI. AAAI Press, 2634–2642
2024
-
[21]
Salman Hameed Khan, Mohammed Bennamoun, Ferdous Ahmed Sohel, and Roberto Togneri. 2016. Automatic Shadow Detection and Removal from a Single Image. IEEE TPAMI 38, 3 (2016), 431–446
2016
-
[22]
Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang. 2019. Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks. IEEE TPAMI 41, 11 (2019), 2599–2613
2019
-
[23]
Le and Dimitris Samaras
Hieu M. Le and Dimitris Samaras. 2019. Shadow Removal via Shadow Image Decomposition. In ICCV. IEEE, 8577–8586. https://doi.org/10.1109/ICCV.2019. 00867
2019 doi
-
[24]
Tsang, and Song Wang
Xiaoguang Li, Qing Guo, Rabab Abdelfattah, Di Lin, Wei Feng, Ivor W. Tsang, and Song Wang. 2023. Leveraging Inpainting for Single-Image Shadow Removal. In ICCV. IEEE, 13009–13018
2023
-
[25]
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. 2021. SwinIR: Image Restoration Using Swin Transformer. In ICCVW. IEEE, 1833–1844
2021
-
[26]
Yuhao Liu, Zhanghan Ke, Ke Xu, Fang Liu, Zhenwei Wang, and Rynson W. H. Lau. 2024. Recasting Regional Lighting for Shadow Removal. In AAAI. AAAI Press, 3810–3818
2024
-
[27]
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. In ICCV. IEEE, 9992–10002
2021
-
[28]
Zhihao Liu, Hui Yin, Xinyi Wu, Zhenyao Wu, Yang Mi, and Song Wang. 2021. From Shadow Generation to Shadow Removal. In CVPR
2021
-
[29]
Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In ICLR (Poster). OpenReview.net
2019
-
[30]
Sohail Nadimi and Bir Bhanu. 2004. Physical Models for Moving Shadow and Object Detection in Video. IEEE TPAMI 26, 8 (2004), 1079–1087
2004
-
[31]
Liangqiong Qu, Jiandong Tian, Shengfeng He, Yandong Tang, and Rynson W. H. Lau. 2017. DeshadowNet: A Multi-context Embedding Deep Network for Shadow Removal. In CVPR. IEEE Computer Society, 2308–2316. https://doi.org/10.1109/ CVPR.2017.248
2017
-
[32]
Jifeng Wang, Xiang Li, and Jian Yang. 2018. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In CVPR. 1788–1797
2018
-
[33]
Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li. 2022. Uformer: A General U-Shaped Transformer for Image Restoration. In CVPR. IEEE, 17662–17672
2022
-
[34]
Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick
-
[35]
Dauphin, and David Lopez-Paz
Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz. 2018. mixup: Beyond Empirical Risk Minimization. In ICLR (Poster). OpenReview.net
2018
-
[36]
Ling Zhang, Qing Zhang, and Chunxia Xiao. 2015. Shadow Remover: Image Shadow Removal Based on Illumination Recovering Optimization. IEEE TIP 24, 11 (2015), 4623–4636
2015
-
[37]
Yiqi Zhong, Xianming Liu, Deming Zhai, Junjun Jiang, and Xiangyang Ji. 2022. Shadows can be Dangerous: Stealthy and Effective Physical-world Adversarial Attack by Natural Phenomenon. In CVPR. 15324–15333
2022
-
[38]
Yurui Zhu, Jie Huang, Xueyang Fu, Feng Zhao, Qibin Sun, and Zheng-Jun Zha
-
[39]
Yurui Zhu, Zeyu Xiao, Yanchi Fang, Xueyang Fu, Zhiwei Xiong, and Zheng-Jun Zha. 2022. Efficient Model-Driven Network for Shadow Removal. In AAAI
2022
-
[2019]
https://github.com/facebookresearch/detectron2
Detectron2. https://github.com/facebookresearch/detectron2
-
[2022]
Bijective Mapping Network for Shadow Removal. In CVPR. 5627–5636
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.