{"id":"b359c850-28b9-44e6-8da9-40d82938cbda","arxiv_id":"2412.03413","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"A U-Net trained on MODIS nighttime SST with four previous days as input reconstructs artificially occluded pixels with lower RMSE than DINCAE in one comparison, but the reported 50 percent advantage is not supported by the paper's own numbers.","lead":"This paper trains U-Net and vision-transformer models to fill cloud-covered areas in satellite sea surface temperature images, testing them on artificial clouds over the Italian seas. The authors claim their best model cuts reconstruction error by 50 percent versus existing gap-filling products, but the evidence in the paper supports a smaller, less directly comparable improvement.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No experiment in the paper measures RMSE under real cloud occlusion against an established method; the abstract's 50% claim is contradicted by Table 7's 22% and Figure 10's cloud-free-only comparison.","rationale":"The reader's REJECT is well-founded. The stress-test pass found an even more immediate problem than the transferability of synthetic masks: the paper's own numbers do not support the abstract's 50% claim. Table 7 shows a 22% improvement over DINCAE on mismatched protocols; Figure 10 compares on cloud-free pixels where the model preserves observations, so it cannot demonstrate skill under clouds. The only direct test of gap-filling is the synthetic generator, whose transfer is explicitly qualified by the authors. This combination means the central claim is unverified. The concrete test proposed would settle the matter: a common-protocol comparison on real cloudy pixels against in situ data. If that test passes, the paper could be revised to a stronger claim; as submitted, rejection is appropriate. I partially agree with the reader's weakest assumption: the generator transfer is a concern, but the primary failure is the absence of any valid head-to-head cloudy-pixel comparison in the paper.","tokens_in":12258,"tokens_out":4154,"duration_ms":41109,"concrete_test":"Run a single head-to-head evaluation on a common held-out test set: use MODIS-Aqua nighttime scenes from 2021-2023 over the full Italian-seas region, real cloud masks from the operational L3 product, and in situ SST from drifters or buoys as ground truth for pixels flagged cloudy. Compute RMSE for U-Net64, DINCAE (evaluated on the same region and period), and the Copernicus L4 product separately on cloudy pixels. If U-Net64's RMSE is not at least 50% lower than both baselines on those cloudy pixels, the abstract's headline claim fails. As an internal check, also recompute Table 7 using the same generator for both models on the same region and dates; if the ratio changes materially from 0.42/0.54, the reported improvement needs correction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim ('50% lower RMSE over established gap-filling methods') requires a valid head-to-head comparison on cloud-covered pixels. The paper provides none. Table 7 compares U-Net64 (RMSE 0.42) with DINCAE (0.54) but on different test protocols: DINCAE is evaluated on the Northern Adriatic, fixed artificial masks, and MODIS-TERRA 2003-2016, while U-Net is evaluated on its own configurable generator over the full Italian seas. Figure 10, used to claim superiority over L4, computes RMSE on visible (cloud-free) pixels only; since the model preserves observed values there, this measures fidelity to the input, not gap-filling skill under clouds. The synthetic-mask evaluation (Section 3.3) is the only evidence of actual reconstruction, but the paper itself notes the bias risk: 'the sea temperature under clouds is usually different from the temperature under a clear sky' and only asserts it is lower at night. The masks are made by superimposing clouds from a different day, and the evaluation region excludes pixels already cloudy in the real scene, potentially selecting easier cases. Thus the headline improvement is not established by any experiment reported in the paper, independent of whether the generator is representative.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript investigates deep learning models, primarily U-Net and vision transformer architectures, for filling cloud-occluded pixels in nighttime MODIS Aqua sea surface temperature (SST) fields over the Italian seas. The authors describe an artificial cloud generator that superimposes masks from other days to create training and evaluation targets, compare several U-Net/ViT configurations, and report that their best U-Net64 with four days of temporal input outperforms DINCAE and the Copernicus L4 product. The central claim, stated in the abstract, is that the best architecture achieves 50% lower root mean square errors than established gap-filling methods.","tokens_in":12623,"tokens_out":3395,"duration_ms":31692,"significance":"If the headline claim were supported by the evidence, the paper would provide a practical, open-source deep-learning alternative for SST gap filling, with possible operational value for Copernicus Marine products. The manuscript includes useful engineering details: a configurable cloud mask generator, systematic ablation of temporal input length and spatial tiling, and public code availability. However, the reported evaluations do not currently establish the claimed advantage for real cloud-covered pixels: the DINCAE comparison is not a controlled head-to-head, the L4 comparison is restricted to cloud-free pixels, and the synthetic-mask evaluation relies on an assumption of representativeness that the authors themselves flag as a bias risk. The significance of the work therefore hinges on additional validation that is not present.","major_comments":[{"comment":"The abstract's claim of \"50% lower root mean square errors over established gap-filling methods\" is contradicted by the paper's own Table 7, which reports U-Net64 RMSE of 0.42 versus DINCAE 0.54, i.e. about 22% lower. Figure 10, the comparison with the L4 product, computes RMSE only on visible (cloud-free) pixels, so it does not measure skill at filling cloud-occluded areas. The central quantitative claim of the paper is therefore not supported by the reported experiments.","section":"Abstract and Table 7"},{"comment":"The synthetic cloud generator evaluates reconstruction on pixels that are artificially occluded by a mask taken from a different day, and the evaluation region excludes pixels that were already cloudy in the real scene. The paper itself states that \"the sea temperature under clouds is usually different from the temperature under a clear sky, but this bias is lower during the night,\" which is an explicit caveat that the synthetic scenario may not represent real cloud occlusion. No experiment in the manuscript measures RMSE on actual cloud-covered pixels against any ground truth (L4, reanalysis, or in situ data). Consequently, the core claim of improved gap-filling accuracy under real cloud occlusion is not established.","section":"Section 3.3"},{"comment":"The seasonal climatology used to compute anomalies is described as \"the time mean SST across all the 21 years dataset,\" which includes the 2021-2023 test period. Since the model is trained to predict residuals with respect to this climatology, the test-period mean SST is embedded in the input baseline, leaking information from the test set into both training and evaluation. The climatology should be computed using only data before the test period, and the evaluation should be repeated to assess the impact of this leakage.","section":"Section 2.4"},{"comment":"The DINCAE comparison is not a controlled head-to-head. Table 7 reports DINCAE RMSE on the Northern Adriatic with fixed artificial masks, MODIS-TERRA data from 2003-2016, and a different evaluation protocol, while the U-Net models are evaluated on the configurable generator over the full Italian seas and a different data period. Differences in region, input data source, mask distribution, and temporal coverage could explain the reported RMSE gap as much as architectural superiority. A valid comparison would require both methods to be evaluated on the same test set and the same occlusion masks.","section":"Section 4.1 and Table 7"},{"comment":"The comparison with the L4 product in Figure 10 is computed over visible sea pixels only, where the U-Net reconstruction coincides with the input L3 values (the model preserves observed values in cloud-free areas). This measures the model's fidelity to its input, not its ability to reconstruct cloud-covered SST. The qualitative comparison in Figure 11 is likewise limited to cloud-free regions. These experiments therefore do not support the conclusion that the U-Net outperforms L4 at gap filling.","section":"Section 5 and Figure 10"}],"minor_comments":[{"comment":"The domain coordinates are stated as \"latitude between 35.33° and 46.0°E and longitude between 7.92° and 18.58°N\"; latitude and longitude units are swapped and should be corrected (latitude in °N, longitude in °E).","section":"Section 1"},{"comment":"\"outlied\" should be \"outlined\" in the description of the ViT architecture.","section":"Section 3.2"},{"comment":"The text contains a typo: \"Donwnsamping\" should be \"Downsampling.\"","section":"Section 3.1"},{"comment":"\"Our best-performing architecture show\" should be \"Our best-performing architecture shows.\"","section":"Abstract"},{"comment":"The entries for spatial gradients above 2°C are blank rather than shown as zeros or dashes; the table would be clearer if all cells contained explicit values.","section":"Table 2"},{"comment":"The sentence \"We retrained the model L3S reprocessed data\" is missing a preposition; it should read \"We retrained the model on L3S reprocessed data.\"","section":"Section 5"},{"comment":"\"Our selected best model architectures is made of a U-net64 algorithm\" should be rephrased, for example \"Our selected best model architecture is a U-Net64 algorithm.\"","section":"Section 7"}],"recommendation":"major_revision","confidential_remarks":"The paper's evaluation design contains load-bearing flaws that the authors will need to address with substantial new experiments: a leakage-free climatology, a controlled DINCAE comparison, and a valid test on real cloud-covered pixels. The internal inconsistency between the abstract's 50% claim and Table 7's ~22% should also be resolved. I do not recommend rejection outright because the underlying method (temporal U-Net with anomaly normalization) is plausible and the code is publicly available, but the current manuscript overstates its findings."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read this as a careful engineering paper wrapped in an overclaimed abstract. What's actually new is a well-tuned U-Net for SST gap-filling in the Italian seas: a four-day input window, quadrant tiling, climatology-anomaly targeting, and a thorough ablation of days, width, and tiling. The authors also provide code and are honest about the synthetic-mask limitation. That part is worth something.\n\nThe soft spots are real and they matter. The abstract says 50% lower RMSE over established methods, but Table 7 shows U-Net64 at 0.42 vs DINCAE at 0.54, which is about 22% lower, and that comparison is not a controlled head-to-head: different test area, different satellite, different mask protocol. The L4 comparison in Figure 10 is computed only on cloud-free visible pixels, so it measures fidelity to the input, not gap-filling skill under clouds. The synthetic mask generator is used for both training and evaluation, so the test distribution matches the training distribution. And the climatology used to compute anomalies is averaged over all 21 years, including the 2021–2023 test period; that leaks test-period mean SST into the inputs. The paper itself notes the bias risk of using clear-sky SST as a proxy for cloudy-sky SST, which I read as a fair admission but not a fix.\n\nThese are not minor blemishes. The central claim of the paper — that U-Net is substantially better than existing gap-filling methods under real cloud occlusion — is not established by any experiment reported here. The internal results are plausible and the approach is reasonable, but the evidence stops short of the headline.\n\nWho should read this? Researchers working on SST reconstruction will find the ablations and the code useful, and the paper is a good case study in how evaluation protocol can drive conclusions. I would not cite it for the 50% claim, but I might cite it for the tiling/window design if that holds up after a corrected evaluation.\n\nFor peer review: this deserves a serious referee, but not acceptance as is. The authors should be asked to recompute the climatology using only training years, run a common-protocol comparison with DINCAE on the same test region and masks, and either remove the cloud-free-pixel L4 comparison or reframe it as fidelity to the input. If those fixes land, the paper could become a solid contribution. Right now, it's an honest engineering report with an unsupported headline.","headline":"Solid engineering with reproducible code, but the headline 50% improvement claim is not supported by the paper's own comparisons, and the evaluation protocol has load-bearing flaws.","tokens_in":13107,"tokens_out":2013,"would_cite":false,"duration_ms":20881,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a U-Net trained on four nights of MODIS Aqua SST, with a seasonal climatology subtracted, fills cloud-occluded pixels with roughly half the root-mean-square error of established interpolation products.","keywords":["sea surface temperature reconstruction","cloud occlusion","U-Net","deep learning","satellite remote sensing","optimal interpolation","DINCAE","Mediterranean Sea"],"falsifier":"Evaluate the U-Net64 reconstruction on real cloud-covered pixels using independent nighttime microwave SST retrievals or drifting-buoy measurements that see through clouds; if the RMSE advantage over L4 and DINCAE shrinks or reverses on those points, the central claim fails.","tokens_in":12072,"feed_emoji":"🌊","tokens_out":7862,"duration_ms":67393,"temperature":0.7,"pith_summary":"The paper tries to establish that a convolutional U-Net can reconstruct sea surface temperature under cloud cover more accurately than the interpolation-based products now used operationally. The authors train on nighttime MODIS Aqua images over the Italian seas, feeding the network several previous days of temperature and cloud masks and asking it to predict the current day's cloud-occluded pixels as anomalies from a seasonal climatology. Their best U-Net reports lower root mean square error than both the operational L4 optimal-interpolation product and the DINCAE autoencoder, while leaving cloud-free pixels essentially unchanged. If true, the result would give oceanographers and forecasters a cheap, high-resolution way to fill SST gaps without smoothing out fronts and mesoscale structure.","feed_headline":"Neural net fills cloud-covered seas at half the temperature error","feed_subtitle":"A four-day U-Net with seasonal anomaly targets beats standard interpolation over Italian seas.","key_machinery":"The load-bearing object is a U-Net with residual down- and up-sampling blocks and skip connections, configured as U-Net64 with channel structure [64,128,256,512], fed a temporal stack of the current night and four previous nights, each with its cloud mask, plus a land-sea mask. The target is the anomaly relative to an unbiased daily climatology, built by Gaussian-blur interpolation and temporal smoothing, so the network learns residual deviations rather than absolute temperatures. Training and evaluation rely on an artificial cloud generator that copies real cloud masks from other days onto clear-sky scenes, guaranteeing a controlled visible-sea fraction and defining the region where RMSE is measured. The same generator and the DINCAE baselines anchor the comparison.","core_discovery":"The central discovery is that a moderately sized U-Net, trained with a configurable artificial cloud generator on a residual anomaly target, outperforms both OI-based L4 and DINCAE for nighttime SST reconstruction in the Italian seas. On synthetic occlusions, the U-Net64 with four input days reaches an RMSE of 0.42 degrees Celsius against DINCAE's 0.54, and the paper's abstract states that the best architecture shows about 50 percent lower RMSE than established gap-filling methods. On operational L3S inputs, the U-Net reconstruction error over visible sea pixels is around 0.04 degrees Celsius against 0.14 for the L4 product, and it keeps frontal structures that L4 smooths. The authors interpret this as evidence that learned spatio-temporal priors beat linear correlation-based filling once seasonality is removed.","pith_inferences":["The same recipe—anomaly target, artificial masks, and a U-Net with a short temporal stack—could transfer to other gridded ocean variables such as chlorophyll or sea ice, where a similar generator protocol is already in use.","Because error saturates at four input days, the network likely learns a persistence-plus-climatology prior; ablating the temporal channels would show how much of the gain comes from previous days versus spatial context.","The visible-pixel comparison to L4 is not a true cloud-filling test; the decisive experiment is to compare against independent microwave or in-situ measurements under actual clouds, which the paper does not report."],"forward_implications":["The best U-Net64 configuration with four input days is a practical, high-resolution gap-filling method for the Italian seas.","Adding more than four previous days does not improve reconstruction, so the operational input window can be kept short.","Splitting the 256x256 domain into four quadrant models improves accuracy, so regional training can outperform a single larger model.","The method preserves cloud-free pixel values and frontal structures while interpolating under clouds, reducing the smoothing typical of OI products.","If the reported RMSE advantage holds, data-driven reconstructions could feed operational downstream forcing and reanalysis systems without waiting for multi-day temporal windows."],"supporting_citations":[{"why":"Supplies the DINCAE baseline and the artificial-cloud generator approach used for training and validation.","marker":"Barth et al. (2020)"},{"why":"Extends the DINCAE baseline with chlorophyll inputs; the paper compares against this best DINCAE product.","marker":"Barth et al. (2022)"},{"why":"MAESSTRO masked autoencoder for SST under occlusion, used as a state-of-the-art method and sharing the artificial-mask protocol.","marker":"Goh et al. (2024)"},{"why":"Original U-Net architecture that the paper adapts with residual blocks and temporal input channels.","marker":"Ronneberger et al. (2015)"},{"why":"Describes the OI-based L4 SST product against which the reconstruction error is measured.","marker":"Nardelli et al. (2013)"},{"why":"Documents the operational Mediterranean L4 and NRT products used in the application comparison.","marker":"Pisano et al. (2022)"},{"why":"State-of-the-art multi-scale OI reconstruction whose temporal-window smoothing motivates the deep-learning alternative.","marker":"Chin et al. (2017)"},{"why":"Describes the MODIS instrument and data used to build the training time series.","marker":"Werdell et al. (2013)"}],"fun_headline_variants":["U-Net cuts sea surface temp error by half vs standard interpolation","Neural net fills cloud gaps in sea surface temps with 50% less error","Deep learning reconstructs SST under clouds, halving RMSE","Cloudy seas? U-Net reconstructs SST twice as accurate as OI","AI fills ocean temperature gaps under clouds, error slashed in half"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole comparison rests on the assumption that errors measured under artificial cloud masks represent what happens under real clouds, even though the paper notes that water under clouds is cooler than clear-sky water, a bias that is smaller at night.","fun_headline_variants_meta":{"raw":{"variants":["U-Net cuts sea surface temp error by half vs standard interpolation","Neural net fills cloud gaps in sea surface temps with 50% less error","Deep learning reconstructs SST under clouds, halving RMSE","Cloudy seas? U-Net reconstructs SST twice as accurate as OI","AI fills ocean temperature gaps under clouds, error slashed in half"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00049,"raw_usage":{"total_tokens":2345,"prompt_tokens":814,"completion_tokens":1531,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":430,"completion_tokens_details":{"reasoning_tokens":1435}},"tokens_in":430,"tokens_out":1531,"duration_ms":9830,"temperature":1.0,"reasoning_tokens":1435,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:25:54.045185+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the U-Net64 reconstruction on real cloud-covered pixels using independent nighttime microwave SST retrievals or drifting-buoy measurements that see through clouds; if the RMSE advantage over L4 and DINCAE shrinks or reverses on those points, the central claim fails.","supporting_citations":[],"review_version":1}