{"id":"4e7d2999-b453-4358-8087-85c0017b4de0","arxiv_id":"1908.08628","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A deep learning pipeline that models shadows with a linear lighting change and a matte layer reduces shadow-area RMSE on ISTD from 13.3 to 7.9 (claimed), evaluated on a color-corrected test set.","lead":"The paper presents a shadow removal system that first estimates how lighting changes inside a shadow and where the soft boundary is, then relights the shadowed area. It reports a large drop in error on the ISTD benchmark, though the evaluation uses a color-corrected test set that makes comparisons with earlier numbers uncertain.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed state-of-the-art result is not yet supported because Table 1 baseline RMSEs appear to be taken from papers that evaluated on the original ISTD ground truth, not the authors' color-corrected test set.","rationale":"The reader's conditional verdict is well-founded. The single most load-bearing assumption is the comparability of baseline RMSE values, and the concrete test would settle it. I see no reason to move the verdict: if the baseline numbers were re-run and the improvement holds, the paper is a solid contribution; if not, the SOTA claim fails but the method may still be useful. The architecture itself is coherent, and the color-correction is arguably a legitimate step to handle known ISTD inconsistencies, but it must be applied to all methods.","tokens_in":11185,"tokens_out":5010,"duration_ms":49513,"concrete_test":"Obtain the released implementations or outputs of Gong et al., Wang et al., Yang et al., and Guo et al.; run them on the authors' adjusted ISTD test set; recompute RMSE in LAB space on the shadow region, non-shadow region, and full image, then compare with Table 1. If any baseline RMSE changes relative to the published value, the claim of a 40% improvement is not established; if all values remain within rounding error, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the 40% RMSE reduction (13.3 to 7.9) in Table 1. Section 5.2 states 'All numbers are computed on the adjusted testing images so that they are directly comparable,' but the paper never reports re-running any baseline on the adjusted test set. The values for Gong et al. (13.3) and Wang et al. (13.4) are identical to those in the original papers, which evaluated on the unadjusted ISTD ground truth. Because the authors modify the ground truth by a per-channel linear regression (Sec. 5.1), RMSE values computed against the corrected GT are not comparable to published numbers computed against the original GT unless the baselines are re-evaluated. This is not a minor detail: the headline 40% improvement could be entirely due to the GT change. The method itself is a reasonable architecture, but the evaluation protocol as reported cannot substantiate the SOTA claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a shadow-removal framework built on a physical illumination model in which the shadow-free image is a channel-wise affine function of the shadow image, combined with a per-pixel shadow matte. Two deep networks, SP-Net and M-Net, are trained to predict the affine shadow parameters (w, b) and the matte layer α, and the output is formed by Eq. 5. The authors evaluate on the ISTD dataset, reporting RMSE 7.9 on the shadow area versus 13.3 for Gong et al., and further report 7.4 after augmenting the training set with synthetic shadow images generated by the model itself. The main claim is state-of-the-art shadow removal on ISTD.","tokens_in":11352,"tokens_out":2435,"duration_ms":27330,"significance":"If the comparison is made valid, the paper is a useful contribution: it combines a physically motivated low-dimensional illumination model with learned parameter and matte estimators, leading to a compact and interpretable decomposition (Eqs. 4–6). The augmentation idea in Sec. 5.3 is also interesting because it permits targeted synthesis of shadow variations without an external renderer. The paper clearly states its assumptions, including the color-correction protocol, and it provides an explicit failure-case discussion. The central architectural idea is sound and the reported internal ablations (SP-Net alone vs. SP+M-Net) support the value of the matte network. However, the headline quantitative claim is currently tied to a non-standard adjusted test set, and the paper does not provide the re-evaluations needed to make the numbers comparable.","major_comments":[{"comment":"The claim that 'All numbers are computed on the adjusted testing images so that they are directly comparable' is not supported by the evidence in the paper. The values reported for Gong et al. (13.3) and Wang et al. (13.4) are identical to the numbers in the original publications, which evaluated on the unadjusted ISTD ground truth, and the paper does not report any re-run of these baselines on the corrected ground truth. In fact, Table 1 shows the Input Image row with Non-Shadow RMSE 2.6, which is exactly the value the correction is designed to produce on the non-shadow area, while Wang et al. is listed as 7.7 on the same column; a method that leaves the non-shadow area unchanged would score near 2.6 on the corrected ground truth, so the listed baseline number cannot have been computed on the corrected testing images. The headline 40% reduction from 13.3 to 7.9 is therefore not established unless every baseline is re-evaluated on the same corrected ground truth. I request that the authors either run all baseline methods on the corrected test set and report those numbers, or remove the direct comparison and restate the contribution as an evaluation under a modified protocol.","section":"§5.2, Table 1"},{"comment":"The color-correction step changes the ground-truth images by a per-channel linear regression that maps the non-shadow area of the shadow-free image onto the shadow image. This is a legitimate response to the known color-inconsistency problem of ISTD, but it changes the evaluation metric itself, so RMSE values against the corrected ground truth are not comparable to RMSE values against the original ground truth. The paper should report results under both protocols (original and corrected) or, at minimum, provide the original-GT numbers for the proposed method so readers can assess the effect of the correction. Without this, the magnitude of the claimed improvement cannot be separated from the effect of modifying the ground truth.","section":"§5.1"},{"comment":"The augmentation experiment is interesting, but the 7.4 result is only reported on the corrected test set. Since the synthetic images are generated using the same decomposition model that is being trained (Eqs. 8–9), it is possible that the augmented training set biases the model toward the specific form of the model's assumptions. The paper should report the augmented-model results on the original ISTD ground truth as well, and ideally on a second shadow-removal benchmark, to show that the improvement is not an artifact of the matched evaluation protocol.","section":"§5.3, Table 2"}],"minor_comments":[{"comment":"The text 'National Geographic/Microsoft AI for Earth program' contains a typo: 'Neographic' should be 'Geographic'.","section":"Acknowledgments"},{"comment":"Equation 7 divides by I_shadow − I_relit, which can be zero or near-zero for pixels where the shadow has no effect; the paper should state how such pixels are handled in the numerical computation of the training target for M-Net.","section":"Eq. 7"},{"comment":"Both the color-correction regression and the synthetic-shadow generation use linear-regression and inverse operations that can produce out-of-range pixel values; the paper should specify whether clipping or other range enforcement is applied and whether this affects the reported metrics.","section":"§5.1, §5.3"},{"comment":"The qualitative comparison figure would be easier to read if the shadow region were outlined or enlarged, since the differences between methods are mostly visible at boundaries and in the penumbra.","section":"Fig. 8"}],"recommendation":"major_revision","confidential_remarks":"The core method is reasonable and the paper is generally well written, but the central quantitative claim rests entirely on comparability of RMSE values across two different ground-truth protocols. The authors should be asked to re-evaluate all baselines on the corrected test set and to also report results on the original ground truth; the paper should not be accepted until that evidence is provided."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a sensible engineering contribution that combines a known linear illumination model with a classic matting decomposition, and uses two networks to estimate the unknowns. The idea of generating augmented training data by editing the shadow parameters is neat and could be useful beyond this paper. The math is coherent and the method is described well enough to reproduce.\n\nThe soft spot is the evaluation, and it is not a minor one. The authors color-correct the ISTD ground truth via per-channel linear regression because the shadow and shadow-free images have color inconsistency. That is a reasonable thing to do, but it changes the test set. They then say “all numbers are computed on the adjusted testing images so that they are directly comparable.” The problem is that the baseline RMSEs in Table 1 (Gong et al. 13.3, Wang et al. 13.4) match the values in the original publications, which evaluated on the unadjusted ISTD ground truth. If the baselines were not re-run on the adjusted ground truth, the comparison is unfair and the claimed 40% reduction is unsupported. The paper never states that the baselines were re-run, and the matching numbers are suspicious. This is exactly the kind of thing a referee should catch.\n\nThat said, the core method is not the problem. The decomposition is physically motivated, the loss functions make sense, and the failure cases are shown honestly. The augmentation trick is a nice contribution. The concerns about the shadow detector being used at test time are present but minor, since they also report a version with a jointly trained detector.\n\nFor a reader working on shadow removal or image editing, this paper is worth knowing about, but the SOTA claim should be treated as unverified until the baselines are re-evaluated on the same adjusted ground truth or the results are reported on the original ground truth. I would send this to peer review, but with a clear request to fix the evaluation protocol before acceptance.\n\nMy verdict: engage with the work, cite with caution, and ask the authors for matched comparisons and error bars.","headline":"A plausible shadow-removal method built on a classic linear illumination model and matting decomposition, but the headline 40% RMSE improvement rests on a color-corrected test set whose baseline numbers appear to come from the original, unadjusted ground truth.","tokens_in":11877,"tokens_out":1428,"would_cite":false,"duration_ms":16701,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A shadow removal method decomposes each image into a relit layer and a soft matte, cutting shadow-area error on ISTD from 13.3 to 7.9 RMSE.","keywords":["shadow removal","image decomposition","illumination model","shadow matting","deep learning","ISTD dataset","data augmentation","linear relighting"],"falsifier":"Re-run the baseline methods (Gong et al. and Wang et al.) on the color-corrected ISTD test ground truth used in this paper and recompute shadow-area RMSE; if either baseline reaches 7.9 or lower, the state-of-the-art claim is not supported.","tokens_in":10942,"feed_emoji":"🌓","tokens_out":6870,"duration_ms":56498,"temperature":0.7,"pith_summary":"The paper proposes a shadow-removal method that avoids making a network predict shadow-free pixels directly. Instead, it decomposes a shadow image into a relit image and a soft matte, combining them through a classic compositing equation. The relit image is produced by a per-channel linear transformation of the shadow image, whose parameters are predicted by one network while a second network predicts the matte. On the ISTD benchmark the authors report reducing shadow-area RMSE from 13.3 to 7.9, a 40% drop over the prior state of the art, and further to 7.4 when trained on synthetically re-shadowed data. The significance, if correct, is that physically grounded decomposition can match or beat end-to-end generative approaches while avoiding typical generative artifacts.","feed_headline":"Splitting shadows into relit layer and matte cuts error to 7.9","feed_subtitle":"It models shadowed pixels as a linear transform plus a soft matte, beating the prior ISTD state of the art.","key_machinery":"The load-bearing identity is the decomposition $I_{\\text{shadow-free}} = I_{\\text{shadow}} \\cdot \\alpha + I_{\\text{relit}} \\cdot (1-\\alpha)$, with $I_{\\text{relit}} = w \\cdot I_{\\text{shadow}} + b$. The shadow parameters $(w, b)$ are three-channel scale and offset values that model how shadows darken and color-shift pixels; SP-Net regresses them from the input image and mask. The matte $\\alpha$ blends shadow and relit pixels per location, with $\\alpha$ near 1 outside the shadow and near 0 in its umbra, and it is predicted by M-Net. This split lets network outputs be low-dimensional (six parameters plus a matte) instead of full images, which the paper argues avoids blur and artifacts.","core_discovery":"The central claim is that a shadow image can be decomposed as $I_{\\text{shadow-free}} = I_{\\text{shadow}} \\cdot \\alpha + I_{\\text{relit}} \\cdot (1-\\alpha)$, where $I_{\\text{relit}} = w \\cdot I_{\\text{shadow}} + b$ and $\\alpha$ is a per-pixel matte. The vectors $w$ and $b$ are assumed constant across the umbra for each RGB channel, capturing the linear relation between lit and shadowed intensities under direct plus ambient illumination. The paper trains SP-Net to regress $w$ and $b$ from the shadow image and mask, and M-Net to predict $\\alpha$ from the image, relit image, and mask. With this two-network system the paper reports state-of-the-art RMSE of 7.9 on the shadow area of the ISTD test set, compared with 13.3 for the previous best, and 7.4 after augmenting training data with synthetic shadows generated by varying the recovered parameters.","pith_inferences":["The reported margin depends on the color-corrected ground truth being fair to all methods; if the published baseline numbers were computed on the original ground truth, the comparison is not apples-to-apples and the 40% figure would shrink. The paper does not show that baselines were rerun on the corrected images.","The constant $(w, b)$ assumption should break on shadows that cross multiple materials with different spectral responses, since a single per-channel affine map cannot relight both; testing on such scenes would reveal the model's boundary.","The same decomposition could be applied to other spatially varying illumination effects, such as specular highlights or vignetting, as long as a linear relighting model and a matte can be defined.","The augmentation scheme suggests a controllable data-generation recipe for shadow removal, but the synthetic shadows share the same linear-model biases as the training data, so gains might not transfer to shadows that violate the model."],"forward_implications":["If the decomposition is correct, shadow removal no longer requires a network to hallucinate full shadow-free images; predicting six illumination parameters and a matte suffices.","Because the relit image is computed analytically from the parameters, the output resolution is limited only by the input, not by the network's output resolution.","The same decomposition gives a shadow-editing tool: modifying $w$ and $b$ and recombining through Eq. 8 generates new shadow images of controllable strength, which the paper uses to augment training.","The matte layer absorbs errors in the predicted parameters, because M-Net learns to down-weight pixels that SP-Net over- or under-relights.","On the ISTD benchmark, the full system lowers shadow-area RMSE from 13.3 to 7.9, and to 7.4 with the augmented training set."],"supporting_citations":[{"why":"Supplies the ISTD dataset and the ST-CGAN baseline whose error numbers frame the comparison.","marker":"[34]"},{"why":"Provides the previous state-of-the-art interactive method at 13.3 shadow-area RMSE that the paper claims to beat.","marker":"[12]"},{"why":"Introduces the affine shadow illumination model that the paper's linear transformation builds on.","marker":"[28]"},{"why":"Establishes shadow matting and compositing, the basis for handling penumbra via a matte layer.","marker":"[3]"},{"why":"Provides the alpha compositing equation used as the image decomposition in Eq. 5.","marker":"[24]"},{"why":"Gives the shadow detection network used at test time to produce input masks for SP-Net and M-Net.","marker":"[41]"},{"why":"Supplies the ResNeXt architecture that SP-Net adapts for parameter regression.","marker":"[37]"}],"fun_headline_variants":["Dual-network shadow decomposition cuts RMSE to 7.9","Linear relight plus matte drops shadow error to 7.9","Shadow removal via parameter and matte nets scores 7.9 RMSE","Two-step shadow decomposition achieves RMSE 7.9 on ISTD"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes that the published RMSE values of earlier methods were computed on the same color-corrected ground truth as the paper's numbers, but the paper does not show that the baselines were rerun on the corrected images.","fun_headline_variants_meta":{"raw":{"variants":["Dual-network shadow decomposition cuts RMSE to 7.9","Linear relight plus matte drops shadow error to 7.9","Shadow removal via parameter and matte nets scores 7.9 RMSE","Two-step shadow decomposition achieves RMSE 7.9 on ISTD"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000314,"raw_usage":{"total_tokens":1784,"prompt_tokens":952,"completion_tokens":832,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":753}},"tokens_in":568,"tokens_out":832,"duration_ms":7734,"temperature":1.0,"reasoning_tokens":753,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:33:59.756635+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the baseline methods (Gong et al. and Wang et al.) on the color-corrected ISTD test ground truth used in this paper and recompute shadow-area RMSE; if either baseline reaches 7.9 or lower, the state-of-the-art claim is not supported.","supporting_citations":[{"cited_title":"Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal","cited_arxiv_id":null,"evidence_quote":"Supplies the ISTD dataset and the ST-CGAN baseline whose error numbers frame the comparison."},{"cited_title":"Interactive removal and ground truth for difficult shadow scenes","cited_arxiv_id":null,"evidence_quote":"Provides the previous state-of-the-art interactive method at 13.3 shadow-area RMSE that the paper claims to beat."},{"cited_title":"The shadow meets the mask: Pyramid-based shadow removal.Computer Graphics Forum, 27(2):577–586, April 2008","cited_arxiv_id":null,"evidence_quote":"Introduces the affine shadow illumination model that the paper's linear transformation builds on."},{"cited_title":"Salesin, and Richard Szeliski","cited_arxiv_id":null,"evidence_quote":"Establishes shadow matting and compositing, the basis for handling penumbra via a matte layer."},{"cited_title":"Compositing digital images","cited_arxiv_id":null,"evidence_quote":"Provides the alpha compositing equation used as the image decomposition in Eq. 5."},{"cited_title":"Bidirectional feature pyramid network with recurrent attention residual modules for shadow detection","cited_arxiv_id":null,"evidence_quote":"Gives the shadow detection network used at test time to produce input masks for SP-Net and M-Net."},{"cited_title":"Aggregated residual transformations for deep neural networks","cited_arxiv_id":null,"evidence_quote":"Supplies the ResNeXt architecture that SP-Net adapts for parameter regression."}],"review_version":1}