{"id":"4b1442b8-f2f2-41d3-9a46-802d53453c18","arxiv_id":"2506.22866","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A threshold on backpropagated gradients improves class activation map resolution and yields near-fully-supervised defect segmentation on KSDD and KSDD2.","lead":"This paper proposes a weakly supervised method for segmenting surface defects in industrial images using only image-level labels. It introduces a gradient-filtering backpropagation step to sharpen class activation maps, then trains a segmentation model on pseudo-labels.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 7.24-point IoU gain over Ablation-CAM comes from single training runs with no variance reporting; on KSDD's 52 defect images this could be seed noise.","rationale":"The reader's verdict is CONDITIONAL and cites lack of error bars and code as reasons. The weakest_assumption field, however, focuses on the gradient-monotonicity premise. In my stress-test, the more load-bearing issue is statistical reliability: the paper's strongest claim is a numerical superiority on a small, imbalanced dataset, and without multiple seeds or confidence intervals the 7.24-point IoU advantage cannot be distinguished from training noise. This is a direct threat to the central claim, whereas the gradient premise is an assumption about the method's internal mechanism that could be wrong even if the empirical result holds. The reader partially anticipated this in the rationale ('lacks error bars'), so my concern agrees with the overall verdict but shifts the emphasis. A multiple-seed rerun is a concrete, low-cost check that would settle whether the reported margins are real. If the margins persist, the conditional acceptance is justified; if they vanish, the paper's central superiority claim would need to be withdrawn or substantially weakened. The verdict stays CONDITIONAL because the paper should be revised to include this evidence before stronger claims are made.","tokens_in":10881,"tokens_out":6513,"duration_ms":73179,"concrete_test":"Retrain the classifier with 5 independent random seeds for RA-CAM and Ablation-CAM (and ideally LayerCAM/FullGrad) on KSDD and KSDD2, keeping the reported preprocessing, Otsu thresholding, and pseudo-label training fixed; report mean±std defect IoU and per-seed distributions. If the mean gap between RA-CAM and Ablation-CAM is smaller than the pooled standard deviation, or if the gap is not reproducible at the default δ=50%, the superiority claim is not established and the verdict should remain conditional on further evidence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that RA-CAM beats Ablation-CAM by 7.24% IoU on KSDD and 3.53% on KSDD2, and that pseudo-label training approaches fully supervised performance. All numbers are point estimates from one training run: the paper reports no seeds, no error bars, and no confidence intervals. KSDD contains only 52 defective images with extreme class imbalance, so IoU is highly sensitive to a few boundary pixels, and the classifier training is stochastic (SGD, random flips). The 7.24-point gap could therefore lie within run-to-run variation. This is not a critique of the mechanism but of the evidence for the headline claim: if the gap collapses under resampling, the claimed superiority is unsupported. The reader's identified assumption about gradient monotonicity is mechanism-level, yet even a flawed mechanism can produce a working heuristic; the numbers themselves are the actual claim. Relatedly, the fully supervised baselines in Table III are not sourced or described, so the '88.6% of DeepLabV3+' statement may also be inflated by weak baselines, but the seed-robustness check is the more fundamental precondition.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a weakly supervised defect segmentation framework consisting of two stages: generating high-resolution class activation maps via a region-aware CAM (RA-CAM) built on filtering-guided backpropagation (FGBP), and then using these maps as pseudo-labels to train a segmentation network. The method is evaluated on KSDD and KSDD2, reporting IoU improvements over existing CAM methods and pseudo-label segmentation performances that approach fully supervised baselines. The authors also demonstrate FGBP as a plug-in module for LayerCAM and FullGrad.","tokens_in":11137,"tokens_out":4610,"duration_ms":45135,"significance":"If the reported gains are robust, the method would be a practically useful contribution to industrial defect inspection with image-level labels only. The paper's main strengths are the simplicity of the proposed filtering idea, the plug-and-play demonstration with two existing CAM methods (Table IV), and the explicit analysis of the δ sensitivity on both training and test splits (Figs. 8–9). The pseudo-label training results (Table III) suggest that the generated heatmaps are informative enough to train a segmentation model to near-full-supervision performance on KSDD2. However, the empirical claims currently rest on single-run evaluations, a benchmark-tuned hyperparameter, and under-specified baselines, which limits the strength of the conclusions.","major_comments":[{"comment":"The headline gains (7.24% IoU over Ablation-CAM on KSDD and 3.53% over LayerCAM on KSDD2) are single-run point estimates. The KSDD defect set contains only 52 images, and the classification model is trained with stochastic SGD and random horizontal/vertical flips, so the reported differences may be within run-to-run variation. Please report mean and standard deviation (or at least multiple seeds) for all compared methods, and confirm that the ranking holds under resampling.","section":"Section IV.C, Tables I and II"},{"comment":"The definition of RA-CAM is not reproducible as written. The left-hand side R^c_l_CAM appears on both sides, the arguments (x, I(∂y^c/∂A^l ≥ δ) ⊗ ∂y^c/∂A^l) are undefined (what is x? what is ⊗?), and the connection between R^c_l_CAM in Eq. (6) and R^c_kl in Eq. (7) is unclear. Please rewrite Eq. (6) to specify exactly how the indicator-filtered gradients are used to weight the feature maps in Eq. (7).","section":"Section III.B, Eq. (6)"},{"comment":"The hyperparameter δ is selected based on the IoU-versus-δ curves on the training sets of the same two datasets, and the default δ=50% is then used for the reported test results. This makes part of the reported superiority dependent on benchmark-specific tuning. Please report test performance for a range of δ values (e.g., 30% to 70%) to show the sensitivity of the claimed gains, and ideally validate δ on a third dataset or via cross-validation.","section":"Section IV.E.1, Figs. 8 and 9"},{"comment":"The fully supervised baselines (UNet, DeepLabV3, DeepLabV3+) are not described or cited; the reader cannot tell what training protocol, backbone, data split, or loss function produced the numbers. Since the claim of '88.6% of DeepLabV3+'s mIoU' depends entirely on these baseline numbers, please specify their training setup or provide a reference with the exact protocol.","section":"Section IV.D, Table III"},{"comment":"The central premise that 'the higher the value of the gradient, the higher the likelihood that the defect will be localized' is asserted but never tested. Because FGBP discards all gradient entries below the δ-th percentile, this assumption could remove genuine low-gradient defect boundaries. Please provide a per-layer analysis (e.g., correlation between gradient magnitudes and ground-truth defect masks) to support the mechanism, or at least report how the segmentation changes when the filtering is replaced by random filtering of the same fraction of gradients.","section":"Section III.A and Fig. 3"}],"minor_comments":[{"comment":"'Develope' should be 'develop'; also use 'filtering-guided backpropagation' consistently instead of 'filtering-guide backpropagation'.","section":"Section I, Contributions"},{"comment":"The caption 'The workflow of weakly-supervised defect segmentation' duplicates the caption of Fig. 2; it should describe the FGBP computational workflow.","section":"Fig. 3 caption"},{"comment":"The table is missing a caption line; add a caption such as 'Comparison of fully supervised models and pseudo-label training'.","section":"Table III"},{"comment":"It is unclear whether FGBP is applied to all layers or only the last layer when used as a plug-in; specify the layer range and the chosen δ for each method.","section":"Section IV.E.2, Table IV"},{"comment":"The text states that IoU scores reach 37.86% and 57.56%, but Table III reports both Defect IoU and mIoU; clarify which metric is being referenced in the prose.","section":"Section IV.D, text and Table III"},{"comment":"The sentence '[35] designed STDC-Net' appears to misattribute the STDC-Net work; reference [35] is a welding defect segmentation paper, so please check the citation or rephrase.","section":"Section II.B, reference [35]"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a practical and timely problem, and the core idea (gradient-magnitude filtering for CAM refinement) is simple and potentially useful. The main concern is evidentiary: the reported improvements are not yet backed by variance analysis, and the δ-selection procedure on the same datasets raises the risk of benchmark overfitting. If the authors can provide multi-seed results, a sensitivity analysis for δ, and proper descriptions of the fully supervised baselines, the manuscript could become acceptable. The ill-formed Eq. (6) must also be fixed for reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: this is an incremental but legitimate CAM paper. The new piece is filtering-guided backpropagation (FGBP), which thresholds gradient magnitudes at a percentile before computing CAM weights, plus a region-aware weighting that uses the thresholded gradients. That is a small step beyond guided backprop and similar thresholding tricks, and the authors do not claim otherwise. The pseudo-label training is standard. What the paper does well: the plug-in experiment in Table IV is the most useful result. Adding FGBP to LayerCAM and FullGrad lifts IoU on both datasets, which suggests the thresholding heuristic has some general value. The qualitative figures are consistent with that.\n\nThe soft spots are real, and they sit exactly on the load-bearing claim. The 7.24-point gain over Ablation-CAM on KSDD and 3.53-point gain on KSDD2 come from a single training run per method. KSDD has only 52 defective images; IoU is a ratio of pixels that can move several points with a couple of boundary pixels, and the classifier training is stochastic. Without seeds and error bars, the superiority claim is not established. The paper's own ablation shows δ is selected on the training sets of the same datasets used for the final numbers, so part of the reported performance is tuned to the benchmark. That is not fatal, but it should be reported as a selection, not a default. Equation (6) is ill-formed as written; the notation is not clear. The fully supervised baselines in Table III (UNet, DeepLabV3+) are not sourced or described, so the '88.6% of DeepLabV3+' comparison is hard to audit. And there is no code.\n\nNone of this means the mechanism is wrong. The monotonic-gradient assumption is plausible as a heuristic, and the plug-in improvements are consistent with it. But the evidence as presented does not yet support the headline superiority claim. A serious referee could reasonably ask for variance estimates, a proper threshold-selection protocol (e.g., a held-out split), a corrected equation, sourced baselines, and code.\n\nWho is this for? People working on CAM-based weakly supervised segmentation for industrial defects. They will find the FGBP plug-in worth trying. It deserves peer review, but the revision request should be substantial. Read it once, focus on Section IV.C and the ablation.","headline":"A modest CAM variant plus pseudo-label training; the idea is plausible and the plug-in ablation is the best part, but the headline gains rest on single runs with no variance and a threshold tuned on the benchmarks.","tokens_in":11619,"tokens_out":2247,"would_cite":false,"duration_ms":23129,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that filtering out low-magnitude gradients during backpropagation produces higher-resolution class activation maps for defects, and that training a segmentation network on the resulting pseudo-labels approaches fully…","keywords":["weakly supervised semantic segmentation","class activation maps","surface defect detection","filtering-guided backpropagation","pseudo-label training","gradient-based visualization","industrial quality inspection"],"falsifier":"Measure, on a defect dataset with pixel-level ground truth, how often low-gradient pixels inside ground-truth defect masks are truncated by the 50th-percentile cutoff and how often high-gradient background-texture pixels survive it; if the two distributions overlap heavily for some layers or defect categories, the FGBP filter will remove genuine defect regions or retain background, and RA-CAM's IoU gain over LayerCAM or FullGrad should disappear or reverse.","tokens_in":10709,"feed_emoji":"🏭","tokens_out":6342,"duration_ms":58675,"temperature":0.7,"pith_summary":"The paper argues that a simple modification to gradient backpropagation—cutting off low-magnitude gradients before computing class activation maps—can dramatically improve weakly supervised defect segmentation, and that training a standard segmentation network on the resulting pseudo-labels closes most of the gap to fully supervised models. On KolektorSDD, RA-CAM reaches 25.20% defect IoU, beating the previous best Ablation-CAM by 7.24 percentage points; on KSDD2 it reaches 45.54%. After pseudo-label training, defect IoU reaches 37.86% on KSDD and 57.56% on KSDD2, which is 88.6% of DeepLabV3+'s mIoU on KSDD and close to fully supervised performance on KSDD2. The method requires only image-level labels, which matters for industrial settings where pixel-level annotation is expensive and defect samples are scarce. If right, it shows that a low-cost CAM refinement plus pseudo-label training is a practical route to high-precision defect segmentation.","feed_headline":"Gradient cutoff lifts defect IoU to 57.6%","feed_subtitle":"A simple CAM filter plus pseudo-labels approaches fully supervised performance on industrial defect datasets.","key_machinery":"The central mechanism is filtering-guided backpropagation (FGBP), which replaces the standard ReLU gate in the backward pass with an adaptive magnitude filter: gradients are multiplied by an indicator $R^{l+1}_i > \\delta$, where $\\delta$ is the $\\delta$-th percentile of positive gradient values in each feature map. RA-CAM then applies this filtered gradient to weight the feature maps $A^l_k$ and sums across channels as $M^c_{RA-CAM} = \\mathrm{ReLU}(\\sum_k R^c_{kl} \\cdot A^l_k)$. The load-bearing idea is that high-gradient locations correspond to defect regions, so thresholding at a percentile like 50% progressively strips background while retaining the target.","core_discovery":"The central claim is that gradient magnitude itself carries reliable spatial information about defect location, and that truncating small gradients during backpropagation removes background noise while preserving defect detail. Building on this, RA-CAM weights feature maps with the filtered gradients and aggregates them across layers to produce high-resolution heatmaps. The paper further claims that converting these heatmaps into pseudo-labels and training a standard segmentation network yields substantial gains, approaching fully supervised performance on both KSDD and KSDD2. The proposed FGBP also works as a plug-in, improving both FullGrad and LayerCAM on both datasets.","pith_inferences":["Because FGBP operates purely on gradients, it should transfer to other gradient-based explainability methods beyond the two tested, such as Grad-CAM++ or XGrad-CAM, and to other backbones such as ResNet or HRNet—an untested extension implied by the plug-in claim.","The method's success on commutator and similar structured surfaces may not carry to defects on highly textured or stochastic backgrounds, where high-frequency texture could masquerade as high-gradient signal; a stress test on a texture-rich dataset would clarify the boundary of the claim.","If gradient magnitude is indeed a reliable defect prior, the same filter could be applied inside fully supervised segmentation losses as a hard attention mask, potentially improving boundary precision where the paper observes fully supervised models under-segment.","The paper's comparison with full supervision suggests the biggest remaining gap is boundary fidelity, so combining RA-CAM pseudo-labels with boundary-aware post-processing could push weakly supervised IoU closer to the fully supervised numbers."],"forward_implications":["RA-CAM exceeds the previous best weakly supervised CAM method on KSDD by 7.24 IoU points and on KSDD2 by 3.53 points over LayerCAM, indicating that gradient-magnitude filtering extracts target semantics more cleanly than existing weighting schemes.","Pseudo-label training lifts defect IoU to 37.86% on KSDD and 57.56% on KSDD2, reaching 88.6% of DeepLabV3+'s mIoU on KSDD, so image-level labels plus pseudo-labels can substitute for most dense annotation.","FGBP is a drop-in replacement for gradient backpropagation in other methods: adding it to FullGrad raises KSDD IoU from 13.19% to 16.73%, and to LayerCAM from 16.25% to 18.65%.","The optimal $\\delta$ is around the 50th percentile on both datasets; training-set IoU curves track test-set curves, so the threshold can be chosen without validation annotations."],"supporting_citations":[{"why":"Supplies the base class activation map weighting scheme that RA-CAM refines.","marker":"[10]"},{"why":"Introduces gradient-based channel weighting, which RA-CAM extends with magnitude filtering.","marker":"[12]"},{"why":"Hierarchical gradient weighting baseline that RA-CAM compares against and FGBP improves.","marker":"[13]"},{"why":"Previous best baseline on KSDD, which RA-CAM surpasses by 7.24 IoU points.","marker":"[21]"},{"why":"Multi-layer gradient baseline used for comparison and as a plug-in target for FGBP.","marker":"[24]"},{"why":"Provides the KSDD benchmark dataset used for primary evaluation.","marker":"[39]"},{"why":"Provides the KSDD2 benchmark dataset and the mixed-supervision context.","marker":"[4]"},{"why":"Distinguishes RA-CAM from Guided Grad-CAM, which multiplies guided gradients with CAM outputs.","marker":"[38]"}],"fun_headline_variants":["Gradient cutoff lifts defect IoU to 57.6%","Filtered gradients produce high-res defect CAM without full labels","Weakly supervised defect segmentation approaches full supervision","Gradient gating improves CAM resolution for defect segmentation","Pseudo-labels from filtered CAM near fully supervised results"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method presumes that gradient magnitude is monotonically related to defect relevance—higher gradients mean higher probability of being part of the defect—and that a single percentile threshold $\\delta$ can separate defect from background in every feature map and every defect type.","fun_headline_variants_meta":{"raw":{"variants":["Gradient cutoff lifts defect IoU to 57.6%","Filtered gradients produce high-res defect CAM without full labels","Weakly supervised defect segmentation approaches full supervision","Gradient gating improves CAM resolution for defect segmentation","Pseudo-labels from filtered CAM near fully supervised results"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001011,"raw_usage":{"total_tokens":4221,"prompt_tokens":844,"completion_tokens":3377,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":460,"completion_tokens_details":{"reasoning_tokens":3298}},"tokens_in":460,"tokens_out":3377,"duration_ms":28453,"temperature":1.0,"reasoning_tokens":3298,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:55:17.247263+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure, on a defect dataset with pixel-level ground truth, how often low-gradient pixels inside ground-truth defect masks are truncated by the 50th-percentile cutoff and how often high-gradient background-texture pixels survive it; if the two distributions overlap heavily for some layers or defect categories, the FGBP filter will remove genuine defect regions or retain background, and RA-CAM's IoU gain over LayerCAM or FullGrad should disappear or reverse.","supporting_citations":[{"cited_title":"Learning deep features for discriminative localization,","cited_arxiv_id":null,"evidence_quote":"Supplies the base class activation map weighting scheme that RA-CAM refines."},{"cited_title":"Grad-cam: Visual explanations from deep networks via gradient-based localization,","cited_arxiv_id":null,"evidence_quote":"Introduces gradient-based channel weighting, which RA-CAM extends with magnitude filtering."},{"cited_title":"Layercam: Exploring hierarchical class activation maps for localization,","cited_arxiv_id":null,"evidence_quote":"Hierarchical gradient weighting baseline that RA-CAM compares against and FGBP improves."},{"cited_title":"Ablation-cam: Visual explanations for deep convolutional network via gradient-free localization,","cited_arxiv_id":null,"evidence_quote":"Previous best baseline on KSDD, which RA-CAM surpasses by 7.24 IoU points."},{"cited_title":"Full-gradient representation for neural network visualization,","cited_arxiv_id":null,"evidence_quote":"Multi-layer gradient baseline used for comparison and as a plug-in target for FGBP."},{"cited_title":"Segmentation-based deep-learning approach for surface-defect detection,","cited_arxiv_id":null,"evidence_quote":"Provides the KSDD benchmark dataset used for primary evaluation."},{"cited_title":"Mixed supervision for surface- defect detection: From weakly to fully supervised learning,","cited_arxiv_id":null,"evidence_quote":"Provides the KSDD2 benchmark dataset and the mixed-supervision context."},{"cited_title":"Grad-cam: visual explanations from deep networks via gradient-based localization,","cited_arxiv_id":null,"evidence_quote":"Distinguishes RA-CAM from Guided Grad-CAM, which multiplies guided gradients with CAM outputs."}],"review_version":1}