{"id":"7784f65f-331e-4938-ad2e-ab52fb4c0a1e","arxiv_id":"2504.20203","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Distortion and blur/noise augmentations give the best floodwater segmentation results on BlessemFlood21; color augmentations give the worst, with small overall differences.","lead":"This paper compares nine groups of image augmentation tricks for teaching segmentation models to spot floodwater in drone photos of a river flood. The gains are small: the best augmentation group improves pixel-level overlap accuracy by about 1 to 2 percent over the basic group.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Color-group collapse at 100 epochs signals training instability, not a stable ranking; single-run results cannot support the central augmentation ranking.","rationale":"The reader identified weak statistical evidence, single runs, and the Color collapse at 100 epochs, but still recommended CONDITIONAL. I view the problem as more severe: the 10.68% Color/DeepLabV3+ result at 100 epochs is a red flag that the experimental pipeline is not measuring only augmentation quality. The paper's own introductory summary says deviations peak at 2% IoU while its table contains a 70-point gap, an internal inconsistency that casts doubt on the reliability of every number in the comparison. To accept the central claim, one must first rule out training instability or data-processing bugs in at least one configuration. The missing baseline without augmentation further weakens the claim that 'all augmentation groups contribute positively,' since the Basic group is used as benchmark but a no-augmentation control was never run. The paper is reproducible in principle (public dataset, fixed library, fixed epochs), but without code, seed control, or reruns, its stated ranking is not established. The recommendation is REJECT because the central comparative claim cannot be evaluated from the reported evidence as it stands, not because the underlying conclusion is necessarily false; a revised manuscript with seed repetitions and error bars could plausibly be accepted.","tokens_in":7400,"tokens_out":1665,"duration_ms":15119,"concrete_test":"Rerun the DeepLabV3+ Color group (and at minimum Distortion, Blur and Noise, and Basic as controls) at 100, 200, and 300 epochs with at least 3 random seeds using the exact stated augmentation probabilities and ImageNet-pretrained initialization. If the Color group at 100 epochs reproducibly lands near 10% IoU, the collapse is a real augmentation effect and the paper's caveat needs rewording; if any run recovers to ~82-84% IoU, the ranking is unstable and single-run claims are unsupported. Also report mean and standard deviation of IoU across seeds; if the standard deviation exceeds the 1-2% inter-group gaps, the claim that Distortion/Blur-and-Noise consistently outperform Basic requires revision.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is the ranking of augmentation groups. The evidence is single training runs per model/group/epoch combination, with no seed repetition, no error bars, and no no-augmentation control. More importantly, the tables contain internal instability that undermines the claimed consistency: in Table 2 the Color group at 100 epochs collapses to 10.68% IoU, far below all other configurations, while at 200 and 300 epochs it recovers to roughly 83%. This is not a small perturbation; it indicates either a training failure, a broken augmentation recipe (e.g., ToGray or Solarize interacting with ImageNet normalization), or a bad initialization, rather than a measurement of the augmentation group's intrinsic quality. The paper itself notes 'remarkable performance across all metrics' and 'deviation peaks at 2% for IoU,' which directly conflicts with this 70-point outlier. If the 100-epoch Color/DeepLabV3+ result is a training artifact, then the 'Color tends to lag behind' conclusion rests on an unreplicated, unverified number. The Distortion and Blur and Noise groups differ from the Basic baseline by roughly 1 to 2 IoU points at 300 epochs (Table 1: 90.56 vs 88.89; Table 2: 88.75 vs 88.69), which is within plausible run-to-run variance for a single seed, especially with stochastic augmentation applied at 50% probability. The claimed stable ranking thus lacks statistical support, and the paper's own text acknowledges the deviations are small. Consequently, the headline comparative claim is not currently supported by the presented evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates the influence of nine groups of image augmentations (Basic, Blur and Noise, Color, Drop, Distortion, Pixel, Quality, Sophisticated, Weather) on flood-water semantic segmentation. Using the BlessemFlood21 dataset and two pretrained segmentation models (UNet++ and DeepLabV3+), the authors report IoU, Dice, and Accuracy at 100, 200, and 300 training epochs. The central claims are that the Distortion and Blur and Noise augmentation groups yield the best performance, that the Color group tends to lag behind, and that longer training generally improves performance.","tokens_in":7724,"tokens_out":2878,"duration_ms":29534,"significance":"If the reported ranking were statistically robust, the paper would offer useful practical guidance for augmentation selection in flood segmentation. The systematic categorization of augmentations and the evaluation across two architectures and three training durations on a publicly available dataset are a reasonable reference setup, and the paper is honest about the small overall effect sizes (1–2% IoU). However, the evidentiary basis for the central claims is currently weak: all results come from single training runs with no variance estimates, no significance testing, and no no-augmentation baseline, and one reported entry (DeepLabV3+ Color at 100 epochs) is a 70-point outlier that contradicts the paper's own prose. The practical significance is modest, and the conclusions therefore require additional experimental support rather than merely editorial changes.","major_comments":[{"comment":"Table 2 reports DeepLabV3+ with the Color augmentation group at 100 epochs as IoU=10.68%, Dice=19.3%, Accuracy=97.59%, a 70-point drop relative to all other configurations. The text in the same section claims 'IoU achieving more than 83%' and 'deviation peaks at 2% for IoU', which is directly contradicted by this entry. Because the conclusion that Color 'tends to lag behind' relies on tables containing this unreplicated outlier, the authors must either explain this run (e.g., a training failure, a broken augmentation recipe, or an initialization issue) and exclude it with justification, or re-run the experiment and report the corrected value. As written, this single number undermines the internal consistency of the paper.","section":"Result and Discussion / Table 2"},{"comment":"All reported results are from single training runs; no random seeds, no repeated runs, and no variance estimates are provided. The central ranking claim is based on differences of roughly 1–2 IoU points at 300 epochs (e.g., Table 1: Distortion 90.56 vs Basic 88.89 for UNet++; Table 2: Blur and Noise 88.75 vs Basic 88.69 for DeepLabV3+). For stochastic augmentations applied with default probabilities (50%, except 30% for blur), such small gaps are well within plausible run-to-run variance. The paper should present means and standard deviations over at least 3–5 seeds, or equivalently perform significance tests, before claiming that one augmentation group 'consistently' outperforms another.","section":"Result and Discussion / Tables 1 and 2"},{"comment":"There is no no-augmentation baseline. The comparison is always against the 'Basic' augmentation group, not against training without any augmentation. Consequently, the conclusion that 'all the augmentation groups contribute positively to some extent' and the broader claim that augmentation benefits flood segmentation are not actually evaluated. The authors should include a baseline trained with no augmentations (or at least with only the essential resizing/cropping operations needed to run the models) to support these claims.","section":"Setup / Result and Discussion"},{"comment":"The claim that 'longer training periods generally improve model performance' is not consistently supported by the tables. In Table 1, performance decreases from 200 to 300 epochs for Basic (89.11 to 88.89), Color (84.22 to 83.61), Pixel (87.23 to 85.86), Quality (89.32 to 89.00), and Sophisticated (88.92 to 88.72); Weather stays flat. Similar non-monotonic behavior appears in Table 2 (e.g., Drop 86.72 at 200 vs 88.44 at 300, but Color 83.45 at 200 vs 82.88 at 300). The authors should either qualify the statement to reflect the observed non-monotonicity or provide an analysis of when and why longer training helps.","section":"Result and Discussion / Epoch Influence"}],"minor_comments":[{"comment":"The phrase 'state-of-the-art' is used repeatedly but no comparison with prior state-of-the-art segmentation results on BlessemFlood21 on which to base the claim is given. Please clarify whether 'state-of-the-art' refers to the architectures themselves (which is fine) or to the obtained performance.","section":"Abstract / Introduction"},{"comment":"The choice of default Albumentations parameters is described, but the specific probability settings for each augmentation within a group (except blur at 30% and the rest at 50%) are not tabulated. Since probability is a free parameter that can affect the ranking, a table listing per-augmentation probabilities would improve reproducibility.","section":"Augmentations / Parameters"},{"comment":"Several references contain typesetting or punctuation errors, for example [3] and [5] have an erroneous extra comma after the author list, and 'R. R¨osch' in [16] is inconsistent with 'Ronald R¨osch' in the author list. Please proofread the bibliography.","section":"References"},{"comment":"The phrase 'the Color augmentation group registers the lowest scores across all metrics for both models' is stated in the text, but Table 1 shows at 300 epochs the Pixel group (85.86 IoU) is lower than Color (83.61? No, Color is lowest at 83.61, Pixel is 85.86, so this is correct). However, the text also says 'the deviations remain relatively insignificant' which is hard to reconcile with the 10.68% IoU entry in Table 2; the prose should be updated consistently after the outlier is addressed.","section":"Result and Discussion"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely within scope for a workshop or applied venue, but the statistical grounding is the main concern. The authors should be encouraged to add multi-seed runs and a no-augmentation baseline; these are feasible with the reported H100 setup. The self-citation of [16], co-authored by several of the present authors, is appropriate in this context and not a concern."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a straightforward empirical scan—nine augmentation groups from Albumentations, two pretrained segmentation models, three training lengths, one flood dataset. There is no new augmentation method and no new theory; the value is in the structured grouping and the up-to-date dataset. As a benchmark it is useful for people training flood segmentation models, and the broad conclusion (distortion and blur/noise help most, color least) is plausible. But the evidence as presented is only moderately convincing, and the paper needs revision before I'd rely on the ranking.\n\nWhat's good: The grouping is sensible and clearly described. Evaluating on BlessemFlood21 rather than the usual FloodNet adds a useful data point. The 100/200/300 epoch comparison is also reasonable, and the tables make the broad pattern easy to read. Credit where due: this is honest, unpretentious engineering work. The fact that the dataset is co-authored by the same group is not a problem here; the augmentation-ranking comparison is not hardwired by dataset construction.\n\nWhere it's soft: First, every number is a single training run. With stochastic augmentation at 50% probability, one seed is not enough to support the headline ranking, especially when the 300-epoch differences between top groups and the Basic group are around 1–2 IoU points. Second, there is no no-augmentation baseline, so \"contribute positively\" is asserted but not demonstrated. Third, the tables contain at least one red flag: DeepLabV3+ shows Color at 10.68% IoU at 100 epochs while everything else is in the 70s–80s, then it recovers to 83% at 200 epochs. That looks like a training failure or a bad run, not a stable property of the augmentation group. The text says \"deviation peaks at 2%\" and \"IoU above 82%,\" which directly contradicts the table; that would need to be resolved or the result removed. The 100-epoch Quality/Pixel numbers also look noisy. Fourth, no code, seeds, or configs are given, so the scan isn't reproducible as-is.\n\nOn balance: the paper's central argument—that augmentation choice matters a little and certain groups are better than others—probably holds in broad brush. But the specific ranking is less solid than the prose claims. This is a paper for a workshop or a short application-focused venue, not a methods contribution. It deserves a serious referee if it is cleaned up, but I would not accept it in its current form.\n\nRecommendation: send it to peer review with \"revise and resubmit\" in mind—require at least repeated runs, a no-augmentation control, and a correction or explanation of the Color outlier. If it's for a top conference, it is too thin; if it's for a remote-sensing workshop or an applied journal, it's a legitimate small contribution.","headline":"A clean but thin empirical scan: useful augmentation ranking on BlessemFlood21, weakened by single runs and one telling outlier.","tokens_in":8216,"tokens_out":2623,"would_cite":false,"duration_ms":26804,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that among nine augmentation families tested for flood-water segmentation on the BlessemFlood21 river-flood dataset, geometric distortion and blur/noise augmentations give the best results, color transforms the worst, and…","keywords":["semantic segmentation","flood detection","remote sensing","image augmentation","data augmentation","UNet++","DeepLabV3+","BlessemFlood21"],"falsifier":"Run each of the nine augmentation groups across multiple random seeds with the same hyperparameters; if the 1-2% IoU ordering between Distortion/Blur-and-Noise and Color does not hold across seeds, the ranking is a single-run artifact rather than a property of the augmentations.","tokens_in":7224,"feed_emoji":"🌊","tokens_out":5622,"duration_ms":52379,"temperature":0.7,"pith_summary":"The paper asks whether the choice of image augmentation meaningfully changes how well deep segmentation models detect flood water in overhead imagery. By sorting augmentations into nine families—basic flips and crops, blur and noise, color shifts, dropout, geometric distortion, pixel-level changes, quality degradation, sophisticated color adjustments, and simulated weather—and training UNet++ and DeepLabV3+ on the BlessemFlood21 river-flood dataset, it finds that geometric distortion and blur-and-noise families help most, while color-only augmentation helps least. The largest spread across families is about 2 percent IoU, so the effect is real but modest. Training for 300 epochs instead of 100 generally raises scores and narrows the gap between augmentation families. A sympathetic reader would take this as guidance for configuring flood-segmentation training rather than as a dramatic accuracy breakthrough.","feed_headline":"Distortion and blur augmentations win for flood segmentation","feed_subtitle":"Nine augmentation families on two segmentation models: distortion and blur help most, color helps least, by a 1-2 percent IoU margin.","key_machinery":"The central machinery is a taxonomy of augmentations plus a controlled comparison protocol: each of the nine families is applied one at a time while training two standard segmentation networks, UNet++ and DeepLabV3+, for 100, 200, and 300 epochs, with the Basic family as a benchmark and IoU, Dice, and Accuracy as metrics. The load-bearing tool is an augmentation library whose default parameters set the probabilities—50% per transform, except 30% for blur—and the modifications include splitting shift-scale-rotate into separate operations with a full 360-degree rotation range. This setup is what lets the paper attribute performance differences to the augmentation families rather than to other training variables.","core_discovery":"On the paper's own terms, the central finding is empirical: under a controlled protocol where each of nine augmentation families is applied alone to the BlessemFlood21 dataset, the Distortion group—optical barrel and pincushion warping, elastic deformation, grid distortion, perspective shifts—delivers the highest IoU and Dice for UNet++ (90.56% IoU at 300 epochs), while the Blur and Noise group is the best for DeepLabV3+ (88.75% IoU). The Color group, built from jitter, gamma changes, grayscale, solarization, inversion, RGB shifting, and equalization, is the worst for both models yet still stays above 82-83% IoU. The paper also claims that extending training from 100 to 300 epochs improves all groups and reduces the performance spread among them. These claims are framed as findings on a single high-resolution river-flood dataset, not as a general theory of augmentation.","pith_inferences":["If the single-run results are taken at face value, combining Distortion with Blur-and-Noise is the most promising next experiment, since the paper leaves group combinations unexplored.","The Color group's collapse to 10.68 IoU for DeepLabV3+ at 100 epochs hints that short schedules amplify bad interactions with aggressive color transforms; testing lower probabilities or longer warm-ups might change the ranking.","The ranking may not be a stable property of the augmentations themselves: without seed repetition, the reported 1-2% margins are within the range that random training variation could flip, so a multi-seed replication is the natural falsifier."],"forward_implications":["Practitioners training flood segmentation on similar RGB imagery should expect geometric-distortion and blur/noise augmentations to give the best accuracy among single-group choices.","Extending training from 100 to 300 epochs should improve IoU, Dice, and accuracy for every augmentation family, making longer schedules a low-risk investment.","Color-only augmentation is the weakest family but is not harmful enough to abandon; it stays above 82% IoU on both models.","The gap between best and worst augmentation family is about 2% IoU, so augmentation choice is a secondary tuning lever compared with architecture and training length."],"supporting_citations":[{"why":"Supplies the BlessemFlood21 dataset on which every experiment is trained and evaluated.","marker":"[16]"},{"why":"One of the two segmentation models whose per-group results are reported.","marker":"[25]"},{"why":"The other segmentation model, used as a comparison architecture.","marker":"[26]"},{"why":"The augmentation library whose default probabilities (50%, with 30% for blur) set the experimental protocol.","marker":"[27]"},{"why":"Frames data augmentation as the standard mechanism for enriching training sets, motivating the study.","marker":"[17]"},{"why":"Provides the pretraining weights from which both models are fine-tuned.","marker":"[29]"},{"why":"Defines the IoU, Dice, and accuracy metrics used to compare augmentation groups.","marker":"[10]"}],"fun_headline_variants":["Distortion beats color for flood segmentation","Blur and noise best for DeepLabV3+ flood maps","Longer training shrinks augmentation gap","Optical distortion tops flood segmentation training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim rests on single training runs for each augmentation group with hand-picked, mostly 50% application probabilities and no random-seed repetition; if those runs are atypical, the observed ranking could be coincidence.","fun_headline_variants_meta":{"raw":{"variants":["Distortion beats color for flood segmentation","Blur and noise best for DeepLabV3+ flood maps","Longer training shrinks augmentation gap","Optical distortion tops flood segmentation training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000393,"raw_usage":{"total_tokens":2011,"prompt_tokens":838,"completion_tokens":1173,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":454,"completion_tokens_details":{"reasoning_tokens":1116}},"tokens_in":454,"tokens_out":1173,"duration_ms":8513,"temperature":1.0,"reasoning_tokens":1116,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:34:14.667483+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run each of the nine augmentation groups across multiple random seeds with the same hyperparameters; if the 1-2% IoU ordering between Distortion/Blur-and-Noise and Color does not hold across seeds, the ranking is a single-run artifact rather than a property of the augmentations.","supporting_citations":[{"cited_title":"Blessemflood21: Advancing flood analysis with a high-resol ution georeferenced dataset for humanitarian aid support,","cited_arxiv_id":null,"evidence_quote":"Supplies the BlessemFlood21 dataset on which every experiment is trained and evaluated."},{"cited_title":"Unet ++: A nested u-net architecture for medical image segmentation,","cited_arxiv_id":null,"evidence_quote":"One of the two segmentation models whose per-group results are reported."},{"cited_title":"Encoder-decoder with atrous separable convolution for semantic image segmentation,","cited_arxiv_id":null,"evidence_quote":"The other segmentation model, used as a comparison architecture."},{"cited_title":"Albumentations: fast and flexible image augmentations,","cited_arxiv_id":null,"evidence_quote":"The augmentation library whose default probabilities (50%, with 30% for blur) set the experimental protocol."},{"cited_title":"Imagenet: A large-scale hierarchical image database,","cited_arxiv_id":null,"evidence_quote":"Provides the pretraining weights from which both models are fine-tuned."},{"cited_title":"Image segmentation using deep learning: A survey,","cited_arxiv_id":null,"evidence_quote":"Defines the IoU, Dice, and accuracy metrics used to compare augmentation groups."}],"review_version":1}