{"id":"3c833c47-9ee6-49c1-a669-7bbfa6405d2f","arxiv_id":"2412.19457","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SCGS generates new training images by inpainting over a classifier's misattended background regions, reducing spurious-correlation reliance without group labels.","lead":"This paper proposes a data augmentation method that generates new training images by redrawing the background regions a classifier mistakenly focuses on, reducing reliance on spurious correlations. It works without group labels and is tested on three standard image benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's core mechanism—that Grad-CAM++ masks from mistaken predictions target spurious regions—is never tested against random masks; without this control, the reported worst-group gains could equally stem from generic inpainting rather than spurious-correlation guidance.","rationale":"The reader's conditional verdict (CONDITIONAL, moderate confidence) is reasonable given the missing code and underspecified details. My stress-test identifies a sharper, more decisive gap: the paper's central mechanism—incorrect-attention masks identifying spurious regions—is not isolated by any control experiment. The existing Table II ablation varies the mask source (Grad-CAM vs Grad-CAM++) and the presence of a mask (img2img), but it does not vary the spatial content of the mask. A random-mask control directly tests the claim that the 'incorrect attended regions' are what guide the debiasing. If random masks work as well, then the method's success is not attributable to spurious-correlation guidance, and the contribution is substantially weakened. This does not invalidate the empirical numbers, but it changes the interpretation of the central claim. Since the paper already lacks code and detailed reproducibility, adding this one experiment is a natural condition for acceptance; hence the verdict remains CONDITIONAL (UNCHANGED). I partially agree with the reader's weakest assumption because both concern the mask's role, but the reader focused on mask quality/Stable Diffusion fidelity, whereas my concern is the absence of a causal control for the mask's spatial information. I did not find an internal inconsistency or a mathematical error; the concern is about the sufficiency of evidence for the proposed mechanism.","tokens_in":8414,"tokens_out":14410,"duration_ms":135715,"concrete_test":"Run the SCGS pipeline on MetaShift with identical hyperparameters, but replace the Grad-CAM++ masks with random binary masks that match the Grad-CAM++ masks in area (and, ideally, in connected-component statistics). Generate new images with Stable Diffusion using these random masks and retrain the downstream classifier. Compare worst-group accuracy to the reported SCGS value (75.3%) and to the img2img baseline (68.8%). If the random-mask variant achieves accuracy close to 75.3%, the specific mask selection is not load-bearing; if it falls to the img2img level, the Grad-CAM++ mask is essential to the method.","verdict_should_be":"UNCHANGED","load_bearing_attack":"SCGS's novelty is the use of incorrect-attention masks to guide image generation: misclassified images are masked by Grad-CAM++ high-attention regions (preserved) while the instance region is regenerated with Stable Diffusion. The central claim depends on the mask's spatial information being causally responsible for the debiasing. However, the only ablation in Table II compares Grad-CAM++ masks to Grad-CAM masks and to maskless img2img; it never compares to random masks of matched size and location. If a random mask preserves a comparable area and produces similar worst-group accuracy, then the method is not actually 'spurious-correlation-guided'—the improvement would be attributable to generic inpainting or dataset diversification, undermining the proposed mechanism and the paper's stated contribution. The paper also provides no generated-image samples, no label-accuracy verification of synthetic images, and no mask-quality metrics, so the semantic correctness of the augmented data is entirely unverified. This is load-bearing because the reported gains (e.g., 13.1% worst-group improvement over ERM on MetaShift) could be reproducible even if the mask selection is irrelevant; the explanation of why SCGS works would then be false, and the method would reduce to a less-novel augmentation trick.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SCGS, a data augmentation method that uses Stable Diffusion to generate new training images aimed at reducing a classifier's reliance on spurious correlations, without requiring group labels. The method first trains an ERM model, identifies misclassified training images, clusters and samples them, computes Grad-CAM++ masks from the incorrect prediction, preserves the high-attention (putatively spurious) regions, and regenerates the remaining regions conditioned on the true class label. The augmented data are added to the training set and used to retrain the classifier. Experiments on MetaShift, Waterbirds, and CelebA report improved worst-group accuracy over ERM and competitive performance when combined with JTT.","tokens_in":8692,"tokens_out":5233,"duration_ms":45546,"significance":"If the reported results hold, SCGS is a practical, group-label-free augmentation approach that improves worst-group accuracy on standard benchmarks, and the paper includes error bars and comparisons with several strong baselines. The method is simple and could be combined with existing debiasing methods. However, the paper does not yet establish that the spurious-attention guidance is causally responsible for the gains, and reproducibility details on generation and clustering are incomplete.","major_comments":[{"comment":"The ablation in Table II compares Grad-CAM++ masks only to Grad-CAM masks and to maskless img2img; it does not include a control with random masks of matched size and location. Without such a control, the reported worst-group gains cannot be attributed to the spurious-correlation guidance: they could arise from generic masked inpainting or added data diversity. This control is load-bearing for the paper's central claim that the incorrect-attention masks are what make the method effective.","section":"Section III.B, Table II"},{"comment":"The Stable Diffusion configuration is not reported (model checkpoint, prompt template, guidance scale, number of steps, inpainting strength, seed), nor are any generated-image examples shown or any automatic or human verification that the generated images preserve the class label and are semantically plausible. Since the entire pipeline rests on the quality of the generated training data, the manuscript should provide these details and at least a label-accuracy check of the synthetic images.","section":"Section II.E and Section III.A"},{"comment":"The clustering and sampling procedure is under-specified: the feature extractor used for k-means is not stated, and Eq. (3) gives a Gaussian density but not how it is used to select a fixed 20% of misclassified images per class across clusters (e.g., by ranking within clusters or by sampling from the density). The manuscript should specify the feature source and the exact sampling protocol to make the method reproducible.","section":"Section II.C"},{"comment":"The number of clusters K, sampled fraction, generation ratio, and mask threshold are fixed ad hoc with no sensitivity analysis; since these parameters directly control the augmentation, a small study on at least one dataset would strengthen the claim that the gains are not an artifact of specific settings.","section":"Section III.A"}],"minor_comments":[{"comment":"The notation H(x) = Grad − CAM(...) is inconsistent with the text, which says Grad-CAM++ is used; please correct this to avoid confusion.","section":"Section II.D, Eq. (5)"},{"comment":"The mask threshold is described only as 'red and orange regions', which is not an objective criterion; please report the numeric threshold or the heatmap percentile used to define the mask.","section":"Section III.A"},{"comment":"The dataset name is misspelled as 'Metashift' in the table header and in several places in the text; it should be 'MetaShift'.","section":"Table I and text"},{"comment":"Reference [21] contains a typo: 'CLIP []]' should be 'CLIP]' or similar.","section":"References"},{"comment":"The symbol D_i(k)_sam is used in Eq. (4) but has not been defined in the text; please define it explicitly.","section":"Section II.C, Eq. (4)"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the journal's scope and addresses a timely problem. The main risk is that the empirical results may not uniquely support the proposed mechanism; the random-mask control is the key experiment to request. I would also encourage the editor to ask for code or at least detailed generation settings, as the current manuscript is not reproducible by the standards of this community."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this is a plausible, group-label-free augmentation method that buys a real improvement on standard spurious-correlation benchmarks, but the paper never tests whether the specific mask locations are doing the work. That missing control is the main thing to fix.\n\nWhat's actually new: the pipeline reroutes ERM misclassifications through clustering, Grad-CAM++ masks, and Stable Diffusion inpainting to generate counter-spurious training images. The combination is new, even though each ingredient is off the shelf. The best result is on MetaShift, where JTT+SCGS reaches 78.4 worst-group accuracy, 4.9 points ahead of the next best and better than group-label methods like Group DRO and DISC. The standalone gains over ERM are also real: 13.1, 9.1, and 9.7 points on the three datasets.\n\nThe paper does some things well. Experiments have error bars. It compares against group-label baselines. The ablation in Table II shows that masks help relative to maskless img2img, and that Grad-CAM++ beats Grad-CAM. The attention map visualization in Fig. 3 supports the claimed mechanism qualitatively.\n\nNow the soft spots, in order of importance.\n\nFirst, there is no random-mask control. The paper argues that the mask's spatial information is what guides debiasing, but it never compares to random masks of matched size and location. The img2img comparison rules out \"any generation helps\" to a degree, but it doesn't rule out \"any mask that preserves the center and regenerates the background helps.\" That's the load-bearing claim, and it's untested. This is fixable; a random-mask condition would directly answer it.\n\nSecond, the semantic quality of the generated data is unverified. No generated images are shown, no label-accuracy check on synthetic images, no mask-quality metrics. If Stable Diffusion produces mislabeled or unrealistic examples, the method could work by adding noise rather than by fixing spurious correlations.\n\nThird, implementation details are underspecified: Stable Diffusion version, prompt construction, mask thresholding, and generation ratios are all vague, and no code is released.\n\nFourth, the CelebA result is modest. SCGS improves worst-group by 9.7 points but drops average accuracy by 3.1; JTT+SCGS gains only 1 point over JTT. The paper's own limitation statement admits weak debiasing on large datasets, which is honest but means the headline claim should be tempered.\n\nNone of these are fatal. The central empirical claim that this pipeline improves worst-group accuracy without group labels is likely correct. The mechanism explanation needs more evidence, but the method is worth a serious referee. I'd send it to review and ask for the random-mask control, generated-image examples, and code. The right audience is the spurious-correlation and data-augmentation community.","headline":"Plausible group-label-free augmentation with real benchmark gains, but the mechanism is untested: no random-mask control, no generated-image verification.","tokens_in":9177,"tokens_out":4548,"would_cite":false,"duration_ms":36076,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A group-label-free data augmentation method, SCGS, uses misclassification-guided image synthesis to reduce classifiers' reliance on spurious background cues, improving worst-group accuracy by 9.1-13.1 points over standard training on…","keywords":["spurious correlations","data augmentation","image generation","Stable Diffusion","Grad-CAM++","worst-group accuracy","group-label-free debiasing","image classification"],"falsifier":"Generate SCGS's new images for a small test split and have annotators check two things: whether the true object and label survive the edit, and whether the masked background was actually replaced; if a random sample reveals many examples with missing objects or unchanged spurious backgrounds, the reported worst-group gains cannot be attributed to the proposed mask-and-repaint mechanism.","tokens_in":8248,"feed_emoji":"🎨","tokens_out":7514,"duration_ms":61697,"temperature":0.7,"pith_summary":"The paper proposes SCGS, a data-augmentation method that weakens a classifier's reliance on spurious correlations without needing group labels. It first trains a standard classifier, collects the training images it misclassifies, and clusters them by the type of background attribute that likely caused the error. It then extracts masks from the misclassified examples using Grad-CAM++ and asks Stable Diffusion to redraw the masked background regions while keeping the foreground object and the true label. The new images are added to the training set so that rare but informative combinations, such as a dog on a bed, become more common. On MetaShift, Waterbirds, and CelebA, SCGS raises worst-group accuracy over standard training by 13.1, 9.1, and 9.7 percentage points, and combined with JTT it exceeds comparison methods on MetaShift by 4.9 points.","feed_headline":"New synthetic images lift worst-group accuracy 9.1-13.1%","feed_subtitle":"A group-label-free pipeline redraws misleading backgrounds with Stable Diffusion, improving three biased benchmarks.","key_machinery":"The central object is the binary mask produced from a Grad-CAM++ class activation map over the classifier's wrong prediction. The mask's white regions mark the spurious background cues the biased model relied on, and its black regions are exactly what Stable Diffusion is asked to redraw, with the true class label as the text prompt. K-means clustering of the misclassified images and Gaussian sampling decide which images enter this pipeline, so different spurious attributes within a class are represented without generating the entire misclassified set. The mask is what turns an ordinary generative model into a targeted debiasing tool: it fixes what to preserve, what to change, and what semantic content the new image should have.","core_discovery":"The paper claims that a classifier's reliance on spurious correlations can be weakened by synthesizing the very image combinations the classifier gets wrong. SCGS runs a standard ERM model on the training set, collects the misclassified images, clusters them per class, and samples a Gaussian-distributed subset. For each sampled image it computes a Grad-CAM++ map over the model's incorrect prediction and thresholds it into a binary mask marking the high-contribution background regions; Stable Diffusion then regenerates the masked-out areas while keeping the object and true label. These new images are added to the training set and the classifier is retrained. The paper reports worst-group accuracy gains over ERM of 13.1 points on MetaShift, 9.1 points on Waterbirds, and 9.7 points on CelebA, and a 4.9-point gain when SCGS is combined with JTT on MetaShift.","pith_inferences":["A direct control the paper leaves implicit is regenerating the same number of images without masks; if that control matched SCGS's worst-group gains, the Grad-CAM++ mask would not be the operative ingredient.","SCGS inherits the failure modes of the base classifier: if a spurious attribute is so dominant that the ERM model's misclassified set is small or homogeneous, clustering and Gaussian sampling may under-represent rare sub-shortcuts, capping the benefit.","Because the pipeline is label-conditioned diffusion, its ceiling should rise as generators render rare object-context combinations more faithfully; the method may also transfer to segmentation or multi-label tasks if the prompt encodes all present labels."],"forward_implications":["Because SCGS needs no group labels or concept bank, it can be applied to datasets where the spurious attributes are unknown or too costly to annotate.","SCGS can be wrapped around existing debiasing methods; on MetaShift, JTT+SCGS beats comparison methods by 4.9 points in worst-group accuracy.","Retrained models show Grad-CAM++ attention moving from background to the instance, indicating the augmentation changes what the classifier uses for prediction.","The gains replicate across three domains—cats/dogs, birds, and faces—so the mechanism is not tuned to a single image type.","The authors note gains shrink on very large datasets, so the method's benefit depends on dataset scale."],"supporting_citations":[{"why":"Supplies the Stable Diffusion model that regenerates the masked background regions to produce new training images.","marker":"[7]"},{"why":"Provides the Grad-CAM++ maps used to locate the high-contribution background regions responsible for misclassification.","marker":"[18]"},{"why":"Baseline method JTT that SCGS is combined with, demonstrating the augmentation transfers to an existing debiasing approach.","marker":"[13]"},{"why":"Supplies the Waterbirds dataset and the Group DRO baseline whose worst-group accuracy is compared.","marker":"[9]"},{"why":"Supplies the MetaShift dataset, where SCGS shows its largest worst-group gains.","marker":"[16]"},{"why":"Supplies the CelebA dataset, the large-scale benchmark where SCGS's gains are smaller.","marker":"[17]"},{"why":"Provides the class activation mapping foundation that Grad-CAM++ improves on for mask generation.","marker":"[15]"}],"fun_headline_variants":["Image synthesis strips spurious cues, lifting worst-group accuracy 13.1%","SCGS: stable diffusion redraws backgrounds to break spurious correlations","Group-label-free image regeneration boosts worst-group accuracy 9-13%","Synthesized images correct classifier focus, cutting spurious correlations","Redraw misleading backgrounds: worst-group accuracy gains up to 13.1%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the attention mask taken from a wrong prediction pinpoints exactly the spurious background, and that the generative model redraws that region while preserving the object and the label, so the synthesized images enlarge the dataset with correct, bias-breaking examples rather than noise.","fun_headline_variants_meta":{"raw":{"variants":["Image synthesis strips spurious cues, lifting worst-group accuracy 13.1%","SCGS: stable diffusion redraws backgrounds to break spurious correlations","Group-label-free image regeneration boosts worst-group accuracy 9-13%","Synthesized images correct classifier focus, cutting spurious correlations","Redraw misleading backgrounds: worst-group accuracy gains up to 13.1%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000264,"raw_usage":{"total_tokens":1574,"prompt_tokens":885,"completion_tokens":689,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":501,"completion_tokens_details":{"reasoning_tokens":601}},"tokens_in":501,"tokens_out":689,"duration_ms":6039,"temperature":1.0,"reasoning_tokens":601,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T00:33:57.085175+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate SCGS's new images for a small test split and have annotators check two things: whether the true object and label survive the edit, and whether the masked background was actually replaced; if a random sample reveals many examples with missing objects or unchanged spurious backgrounds, the reported worst-group gains cannot be attributed to the proposed mask-and-repaint mechanism.","supporting_citations":[{"cited_title":"High- resolution image synthesis with latent diffusion models,","cited_arxiv_id":null,"evidence_quote":"Supplies the Stable Diffusion model that regenerates the masked background regions to produce new training images."},{"cited_title":"Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks,","cited_arxiv_id":null,"evidence_quote":"Provides the Grad-CAM++ maps used to locate the high-contribution background regions responsible for misclassification."},{"cited_title":"Just train twice: Improving group robustness without training group information,","cited_arxiv_id":null,"evidence_quote":"Baseline method JTT that SCGS is combined with, demonstrating the augmentation transfers to an existing debiasing approach."},{"cited_title":"Metashift: A dataset of datasets for evaluating contextual distribution shifts and training conflicts,","cited_arxiv_id":null,"evidence_quote":"Supplies the MetaShift dataset, where SCGS shows its largest worst-group gains."},{"cited_title":"Deep learning face attributes in the wild,","cited_arxiv_id":null,"evidence_quote":"Supplies the CelebA dataset, the large-scale benchmark where SCGS's gains are smaller."},{"cited_title":"Learning deep features for discriminative localization,","cited_arxiv_id":null,"evidence_quote":"Provides the class activation mapping foundation that Grad-CAM++ improves on for mask generation."}],"review_version":1}