{"id":"e7257b17-c671-40fd-bffc-3aa96ccf1b1a","arxiv_id":"2411.13042","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A learned attention-filtering module, AC-Attention, improves cloud removal quality when added to existing networks, with top scores on RICE-I, RICE-II, and SEN12MS-CR benchmarks.","lead":"The authors introduce AC-Attention, a small learned filter that makes attention mechanisms in cloud removal networks ignore irrelevant image regions. Plugged into several existing networks, it reports the best reconstruction scores on three public cloud removal benchmarks, which could make satellite imagery restoration more reliable.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SEN12MS-CR half of the central claim rests on imported baseline numbers from UnCRtainTS under an unstated evaluation protocol; until those baselines are retrained or matched, 'consistently outperforms all baselines' is not established.","rationale":"The reader's weakest assumption correctly identifies the SEN12MS-CR comparison as the load-bearing point. The paper's own Table III footnote concedes that four of the six baselines on this dataset are imported from UnCRtainTS, and no evidence is given that the evaluation protocols match. Since the claimed margins over UnCRtainTS on SEN12MS-CR are small, an uncontrolled comparison is enough to make the 'all datasets' headline conditional. I do not see a reason to reject the paper: the AC-Attention formulation is well specified, the RICE-I/RICE-II comparisons appear more controlled, and the central idea is plausible. But because the strongest competitive evidence is concentrated in a comparison that is not protocol-matched, the appropriate verdict remains CONDITIONAL, not ACCEPT. The numerical inconsistencies (average gain percentages in Section IV.B.2 and the 0.2093 MAE in Table IV) are secondary: they do not by themselves overturn Table III, but they strengthen the need for an independent rerun or a released evaluation script.","tokens_in":20086,"tokens_out":7382,"duration_ms":67957,"concrete_test":"Retrain SpA GAN, DSen2-CR, GLF-CR, and UnCRtainTS on SEN12MS-CR using the exact ACA-CRNet protocol: 256x256 random crops, batch size 6, 30 epochs, the same train/test split, and the same metric script. If any of these baselines then achieves a better MAE, SAM, PSNR, or SSIM than the corresponding ACA-CRNet entry in Table III, the 'consistently outperforms' claim fails for SEN12MS-CR. As a secondary check, run ACA-CRNet through the UnCRtainTS evaluation pipeline (official split and official metric code) and compare; if the imported baseline numbers shift by more than the reported margins, the original comparison was protocol-sensitive.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that ACA-CRNet 'consistently outperforms all baseline methods across the four evaluated metrics on all three datasets' is only as strong as Table III. For SEN12MS-CR, the rows for SpA GAN, DSen2-CR, GLF-CR, and UnCRtainTS are marked with an asterisk and, per the table footnote, the metric values are 'sourced from the UnCRtainTS paper [10]'. Those numbers were produced under UnCRtainTS's own training and evaluation protocol, whereas ACA-CRNet is trained with 256x256 crops, batch size 6, for 30 epochs (Section IV.A.3). The paper does not state whether the imported values use the same 7,899-image test split, the same input scaling, or the same metric implementation as the ACA-CRNet evaluation. The margins on this dataset are small (MAE 0.025 vs 0.027, SSIM 0.896 vs 0.880, PSNR 29.78 vs 28.90), so a modest protocol mismatch could change the ranking. Because the SEN12MS-CR portion is the only place where the claimed superiority over the strongest attention-based baselines is tested on real-world multitemporal data, this uncontrolled comparison is the load-bearing weakness. The miscalculated average gains in Section IV.B.2 and the 0.2093 MAE anomaly in Table IV support a general caution about numerical rigor, but the decisive fix is a controlled SEN12MS-CR comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Attentive Contextual Attention (AC-Attention), a patch-based attention mechanism that adds a learned per-query linear transformation to the similarity scores of contextual attention, with the stated goal of suppressing irrelevant long-range dependencies in cloud removal. The authors embed AC-Attention into the DSen2-CR residual architecture to form ACA-CRNet, replacing two residual blocks with residual AC-attention blocks. They report quantitative results on RICE-I, RICE-II, and SEN12MS-CR, claiming that ACA-CRNet outperforms six baselines on all four metrics on all three datasets, and they provide ablations in which AC-Attention is inserted into three existing methods and three network architectures. The paper also presents visualizations of attention similarity maps and cloud-free outputs.","tokens_in":20416,"tokens_out":5150,"duration_ms":51009,"significance":"If the central claim holds, the contribution is a practical and reasonably simple module: a learned gating/bias mechanism on attention scores that improves reconstruction quality in cloud removal, with demonstrated gains over several published baselines. The strengths of the paper include evaluation on public benchmarks, use of standard metrics, availability of code, and an ablation design that tests the module across multiple architectures. However, the significance is currently tempered by an uncontrolled comparison on the SEN12MS-CR dataset, a lack of error bars or significance tests, and several numerical inconsistencies in the reported averages and ablation tables. The method itself is not circular: the central mechanism is a learned module trained on the target data, and the evaluation uses external benchmarks.","major_comments":[{"comment":"The SEN12MS-CR comparison is not controlled. The table footnote states that the metric values for SpA GAN, DSen2-CR, GLF-CR, and UnCRtainTS are sourced from the UnCRtainTS paper, whereas ACA-CRNet is trained under the authors' own protocol described in Section IV.A.3 (256x256 crops, batch size 6, 30 epochs). The paper does not establish that the imported values use the same test split, input scaling, preprocessing, or metric implementation as the ACA-CRNet evaluation. Because the SEN12MS-CR margins are small (MAE 0.025 vs. 0.027, PSNR 29.78 vs. 28.90, SSIM 0.896 vs. 0.880), a modest protocol mismatch could change the ranking. This is the load-bearing evidence for the claim that ACA-CRNet 'consistently outperforms all baseline methods' on all three datasets, so the authors should either retrain the baselines under their own protocol or obtain and report official numbers produced under a matched evaluation protocol, for the SEN12MS-CR rows at minimum.","section":"Table III and Section IV.B.1"},{"comment":"The claimed average improvements over UnCRtainTS do not match the numbers in Table III. For MAE, the per-dataset relative improvements are approximately 36.4%, 20.0%, and 7.4%, whose arithmetic mean is about 21.3%, not the reported 30.05%. For SAM, the corresponding mean is approximately 16.6%, not 21.05%; for PSNR it is approximately 6.4%, not 5.95%. The SSIM average is close to the reported 1.58%, but the others are not. The authors should either correct the averages or state the exact formula used to aggregate them.","section":"Section IV.B.2"},{"comment":"The ablation table contains a clearly anomalous entry: on RICE-II, ACA-EDNet with AC-Attention reports MAE = 0.2093, whereas the base encoder-decoder reports 0.0218 and the CA version reports 0.021. This single entry contradicts the paper's statement that replacing CA with AC-Attention leads to consistent improvements across all architectures, and it is almost certainly a typo or an evaluation error. The authors should verify the entry and rerun or report the corrected value, and they should re-examine the surrounding numbers for similar transcription errors.","section":"Table IV"},{"comment":"As written, the operation in Eq. (6) is not query-dependent in the way the text claims. Because each row of the softmax matrix Sp sums to 1, the row average Sp,avg is identically 1/Np for every query. Subtracting Sp,avg therefore subtracts a constant shift from every row of Sp, so the 'adjusted similarity matrix' Sp,ad does not capture query-specific average similarity. The query-dependent behavior must come entirely from the learned W and B terms in Eq. (7). The authors should either correct the description of Eq. (6) or clarify the intended computation, and they should also specify how the pixel-level W and B tensors from Eq. (5) are broadcast or reshaped to match the patch-level matrix Sp.","section":"Section III.A.2, Eq. (6)"}],"minor_comments":[{"comment":"The reference to 'Evel et al.' for the UnCRtainTS method should be 'Ebel et al.' to match the cited author in [10].","section":"Section II.B"},{"comment":"The implementation details state that the base channel size C is 256, but Table I shows that the input channel count Cin varies with the dataset; the paper should clarify how the multispectral 13-band SEN12MS-CR data are mapped to the network's input channels.","section":"Section IV.A.3"},{"comment":"In Table V, Ours-STGAN on RICE-I shows a SAM increase from 1.701 to 2.223 relative to the base STGAN, while the text says the modifications produced substantial improvements in all metrics; this exception should be acknowledged or explained.","section":"Table V"},{"comment":"The phrase 'significant improvements' is used for the quantitative gains, but no statistical significance tests or error bars are reported; the authors should soften the wording or provide variance estimates across multiple runs or test sets.","section":"Section IV.B.2"},{"comment":"The notation in Fig. 2 is dense and partly illegible, particularly the expressions for the patch counts and reshaped dimensions; a cleaner definition of the patch index mapping would improve reproducibility.","section":"Section III.A.2"}],"recommendation":"major_revision","confidential_remarks":"The core idea is plausible and the internal RICE-I/RICE-II comparisons are internally consistent, but the SEN12MS-CR evidence for the headline claim is not yet controlled. The numerical discrepancies in Section IV.B.2 and Table IV also suggest that a careful re-verification of the experimental records is needed before the paper can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: AC-Attention is a small but real architectural tweak to Contextual Attention, and the paper provides decent evidence that it helps. The headline claim, though, is stronger than the SEN12MS-CR experiment supports, because the four key baselines on that dataset were not retrained under the authors' protocol.\n\nWhat's new: instead of softmax similarity scores, AC-Attention subtracts the per-query mean, applies a learned affine transform (weight and bias from the query features), and ReLUs the result. That is a clean, cheap way to zero out low-scoring matches. The paper then integrates it into three network families (DSen2-CR, an encoder-decoder, Restormer) and shows consistent small gains over CA. The similarity-map visualizations show the mechanism doing what it claims. This is honest, incremental engineering, and the code is promised.\n\nWhere it wobbles: the SEN12MS-CR rows for SpA GAN, DSen2-CR, GLF-CR, and UnCRtainTS are imported from the UnCRtainTS paper. The margins there are small—PSNR 29.78 vs 28.90, SSIM 0.896 vs 0.880—so a difference in crop size, test split, or metric scaling could change the ranking. The paper doesn't retrain those baselines under its own 256x256, batch-size-6, 30-epoch protocol. That needs to be fixed before the 'consistently outperforms all baselines' claim is credible. I also noticed the reported average gains over UnCRtainTS (30.05% MAE, 21.05% SAM) don't match Table III—I get roughly 21% and 17%—and the ablation table has a clear typo (MAE 0.2093 for ACA-EDNet on RICE-II). The patch size s is never specified. None of this sinks the core idea, but it suggests the numbers were not checked carefully.\n\nOverall: the RICE-I/II results are probably fine, and the module is worth having. The SEN12MS-CR comparison is the load-bearing part for the strongest claim, and it's under-controlled. A serious referee should send this to major revision: retrain or at least justify the baselines, add error bars, correct the arithmetic, and specify the missing hyperparameter.\n\nI'd bring it to the reading group, but I'd tell people to treat the SEN12MS-CR table as provisional.","headline":"AC-Attention is a small but plausible tweak to Contextual Attention with decent evidence on RICE, but the SEN12MS-CR claim rests on uncontrolled imported baselines.","tokens_in":20937,"tokens_out":3727,"would_cite":false,"duration_ms":29918,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that softmax attention weights are never exactly zero, so cloud-removal outputs inherit noise from irrelevant cloud-covered patches, and that AC-Attention fixes this by learning a per-query rescaling that sets useless…","keywords":["cloud removal","attention mechanism","remote sensing imagery","image restoration","Attentive Contextual Attention","ACA-CRNet","DSen2-CR","SEN12MS-CR"],"falsifier":"Re-run all six baselines on the same SEN12MS-CR split, preprocessing, crop size, and evaluation script used for ACA-CRNet, without importing numbers from another paper; if any baseline then matches or beats ACA-CRNet on MAE, SAM, PSNR, or SSIM, the claimed consistent superiority on that dataset is not established.","tokens_in":19860,"feed_emoji":"🌤️","tokens_out":10647,"duration_ms":98087,"temperature":0.7,"pith_summary":"Clouds cover much of the Earth's surface in remote sensing imagery, and the deep learning models used to remove them can return blurry, inconsistent images. This paper argues that one culprit is the attention mechanism itself: softmax-based similarity scores are never exactly zero, so every output patch is a weighted average that includes noise from cloud-covered or unrelated patches. The proposed answer, Attentive Contextual Attention (AC-Attention), learns a per-query weight and bias that rescale the similarity matrix and then applies ReLU, zeroing out irrelevant connections before they reach the output. Built into the DSen2-CR residual network as ACA-CRNet, the module reports better reconstruction on all four metrics across RICE-I, RICE-II, and SEN12MS-CR than six comparison methods. The same module also improves existing networks when dropped into them, so the paper's claim is not just a single tuned architecture but a reusable correction to how attention is computed.","feed_headline":"Attention that learns what to ignore improves cloud removal","feed_subtitle":"ACA-CRNet beats six baselines on MAE, SAM, PSNR, and SSIM across three datasets.","key_machinery":"The load-bearing object is the AC-Attention module, a patch-based attention block. After the input feature $F$ is projected to query, key, and value embeddings and split into $s\\times s$ patches, it computes the patch similarity matrix $S_p = \\mathrm{softmax}(Q_p K_p^T / \\sqrt{d})$. Two small convolutional modules read the query $Q$ and emit a weight $W$ and bias $B$; the attentive score matrix is $S_{att} = \\mathrm{ReLU}[(S_p - \\mathrm{Avg}(S_p)) W + B]$, and the output is $O_p = S_{att} V_p$, where $\\mathrm{Avg}$ is the row-wise average. This transformation is what carries the argument: it lets the network learn, for each query, which long-range matches to keep and which cloud-corrupted or irrelevant matches to set to zero.","core_discovery":"The central claim is that the failure mode of global attention in cloud removal is not a small receptive field but an indiscriminate weighting of every patch. For a query patch $q_i$, the vanilla softmax similarity $s_{i,j}$ is positive for all key patches $j$, so patches with no real relationship—often cloud-covered regions—still contribute to the reconstruction and blur the result. AC-Attention replaces this with $S_{att} = \\mathrm{ReLU}[(S_p - \\mathrm{Avg}(S_p)) W + B]$, where the row-average subtraction centers the similarity matrix and the query-dependent $W$ and $B$ rescale it, so negative and near-zero relations are discarded. The paper embeds this mechanism in two Residual AC-Attention Blocks inside the DSen2-CR residual network, yielding ACA-CRNet; on RICE-I, RICE-II, and SEN12MS-CR it reports the best MAE, SAM, PSNR, and SSIM among the compared methods, and ablations show similar gains when AC-Attention replaces ordinary blocks in SpA GAN, STGAN, encoder-decoder, and U-shaped architectures.","pith_inferences":["Beyond cloud removal, the same softmax-never-zero problem appears in image inpainting and dehazing, where distant irrelevant patches also get small nonzero weights; a straightforward extension is to install AC-Attention in those tasks and run the same base-versus-CA-versus-AC ablation protocol.","Because $W$ and $B$ are produced solely from the query, the module needs no temporal or SAR inputs; I infer it could be combined with temporal-SAR fusion models to get both long-range context and cloud-patch filtering, a configuration the paper does not test.","The paper's visualization suggests a quantitative check: for queries located inside cloud-covered regions, the learned row sums of $S_{att}$ over cloud patches should be near zero while rows for clear-sky queries concentrate on relevant patches; measuring this across SEN12MS-CR would test whether the mechanism behaves as described."],"forward_implications":["ACA-CRNet reports the best scores on all four metrics on RICE-I, RICE-II, and SEN12MS-CR among the six compared methods; the reported PSNR values are 36.66, 33.70, and 29.78 dB on the three datasets.","Swapping AC-Attention into existing cloud-removal models (SpA GAN, STGAN, and DSen2-CR) improves their MAE, SAM, PSNR, and SSIM by substantial average margins, so the module is a drop-in component for those architectures, not only for the authors' own network.","AC-Attention also improves encoder-decoder and U-shaped networks compared with both their base versions and versions using Contextual Attention, supporting the claim that the selection step, not the surrounding architecture, drives the gain.","The similarity-score visualizations show AC-Attention concentrates high scores on semantically matching ground regions and assigns zero to cloud-covered areas, meaning the mechanism's behavior can be inspected directly in a trained model."],"supporting_citations":[{"why":"Supplies the Contextual Attention baseline that AC-Attention modifies and that serves as the comparison in the ablation tables.","marker":"[12]"},{"why":"Defines the vanilla softmax dot-product attention whose near-zero nonzero scores are the paper's stated failure mode.","marker":"[49]"},{"why":"Provides the DSen2-CR residual backbone and the SEN12MS-CR dataset; ACA-CRNet is this network with two Residual AC-Attention Blocks.","marker":"[3]"},{"why":"Provides UnCRtainTS, the temporal-attention baseline and the source of the imported SEN12MS-CR metrics for SpA GAN, DSen2-CR, GLF-CR, and UnCRtainTS.","marker":"[10]"},{"why":"Provides the RICE-I and RICE-II datasets used for the RGB cloud-removal comparisons and ablations.","marker":"[13]"},{"why":"Provides GLF-CR, a global-local fusion baseline evaluated on SEN12MS-CR.","marker":"[46]"},{"why":"Provides SpA GAN, an attention-GAN baseline that is compared and also modified in the ablation study.","marker":"[11]"},{"why":"Provides STGAN, a spatiotemporal GAN baseline that is compared and also modified in the ablation study.","marker":"[9]"},{"why":"Provides the encoder-decoder architecture used to test AC-Attention's generality in the ablation study.","marker":"[56]"},{"why":"Provides the U-shaped Restormer architecture used to test AC-Attention's generality in the ablation study.","marker":"[57]"}],"fun_headline_variants":["Attention that learns to ignore clouds sharpens satellite images","New attention filter blocks cloud noise, beats 6 baselines","AC-Attention: selective focus improves cloud removal across models","Smarter attention picks relevant context, clears clouds better","Cloud removal: attention that discards junk yields sharper results"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The SEN12MS-CR comparison assumes that metric values taken from the UnCRtainTS paper for SpA GAN, DSen2-CR, GLF-CR, and UnCRtainTS are directly comparable to ACA-CRNet numbers produced under the authors' own cropping, batch size, and epoch settings, even though those baselines were not retrained under the same protocol.","fun_headline_variants_meta":{"raw":{"variants":["Attention that learns to ignore clouds sharpens satellite images","New attention filter blocks cloud noise, beats 6 baselines","AC-Attention: selective focus improves cloud removal across models","Smarter attention picks relevant context, clears clouds better","Cloud removal: attention that discards junk yields sharper results"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000673,"raw_usage":{"total_tokens":3126,"prompt_tokens":1065,"completion_tokens":2061,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":681,"completion_tokens_details":{"reasoning_tokens":1980}},"tokens_in":681,"tokens_out":2061,"duration_ms":14109,"temperature":1.0,"reasoning_tokens":1980,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:54:45.990216+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run all six baselines on the same SEN12MS-CR split, preprocessing, crop size, and evaluation script used for ACA-CRNet, without importing numbers from another paper; if any baseline then matches or beats ACA-CRNet on MAE, SAM, PSNR, or SSIM, the claimed consistent superiority on that dataset is not established.","supporting_citations":[{"cited_title":"Generative image inpainting with contextual attention,","cited_arxiv_id":null,"evidence_quote":"Supplies the Contextual Attention baseline that AC-Attention modifies and that serves as the comparison in the ablation tables."},{"cited_title":"Attention is all you need,","cited_arxiv_id":null,"evidence_quote":"Defines the vanilla softmax dot-product attention whose near-zero nonzero scores are the paper's stated failure mode."},{"cited_title":"Cloud removal in sentinel-2 imagery using a deep residual neural network and sar-optical data fusion,","cited_arxiv_id":null,"evidence_quote":"Provides the DSen2-CR residual backbone and the SEN12MS-CR dataset; ACA-CRNet is this network with two Residual AC-Attention Blocks."},{"cited_title":"Uncrtaints: Uncertainty quantification for cloud removal in optical satellite time series,","cited_arxiv_id":null,"evidence_quote":"Provides UnCRtainTS, the temporal-attention baseline and the source of the imported SEN12MS-CR metrics for SpA GAN, DSen2-CR, GLF-CR, and UnCRtainTS."},{"cited_title":"Cloud Removal in Satellite Images Using Spatiotemporal Generative Networks","cited_arxiv_id":"1912.06838","evidence_quote":"Provides STGAN, a spatiotemporal GAN baseline that is compared and also modified in the ablation study."},{"cited_title":"Context adaptive network for image inpainting,","cited_arxiv_id":null,"evidence_quote":"Provides the encoder-decoder architecture used to test AC-Attention's generality in the ablation study."},{"cited_title":"Restormer: Efficient transformer for high-resolution image restoration,","cited_arxiv_id":null,"evidence_quote":"Provides the U-shaped Restormer architecture used to test AC-Attention's generality in the ablation study."}],"review_version":1}