{"id":"464c0609-854a-442d-bd6a-0896f47ec58f","arxiv_id":"2412.15570","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"DefFiller fine-tunes GLIGEN with a mask encoder to synthesize steel defects that match given masks, and shows small but consistent S-measure gains when the synthetic pairs are added to a small training set.","lead":"DefFiller adapts a pre-trained layout-to-image diffusion model to generate steel surface defect images from mask conditions, then uses these synthetic mask-image pairs to expand training data for saliency-based defect detectors. It reports improved detection on the SD-Saliency-900 dataset, but the gains are modest and several evaluation choices need scrutiny.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Data-expansion gains in Table 8 may be inflated because the generator and mask DDPM were trained on the full SD-Saliency-900 dataset, including the test split; a leakage-free rerun is required before the augmentation claim can be trusted.","rationale":"I read the paper as a data-augmentation method whose utility claim rests on the detection gains in Table 8. For that claim to be valid, the detector must be trained on synthetic pairs generated without access to the test split. The paper's own text shows this condition is not met: §4.1.2 fine-tunes DefFiller on the full SD-Saliency-900 dataset, and §4.4.1 trains the DDPM mask producer on the ground-truth masks of the full dataset, even though §4.4.3 subsequently holds out 750 of those pairs as the detector test set. Because the test set is five times larger than the training set, the generated masks and images can closely resemble held-out test examples, making the reported S-measure improvements an artefact of distribution leakage rather than a demonstration of augmentation value. The reader's weakest assumption about transfer of the natural-image prior is plausible but secondary; it is exactly the kind of assumption that the fine-tuning ablation can partially support, whereas the leakage problem is not addressed anywhere and directly invalidates the central experiment. I would therefore move from the reader's CONDITIONAL to REJECT in the current form, while noting that the proposed leakage-free rerun could provide a sound basis for reconsideration.","tokens_in":11276,"tokens_out":8373,"duration_ms":74630,"concrete_test":"Rerun §4.4 under strict data isolation: fix the 1:5 split once; fine-tune DefFiller only on the 150 training mask-image pairs; train the DDPM mask producer only on those 150 training masks; generate 900 new masks and images; then train CSEPNet, TSERNet, and MINet on the original 150 training images plus these synthetic pairs and evaluate on the same 750-image test set. Report Sα, MAE, Emaxξ, and Fmaxβ over at least three random seeds with means and standard deviations. If the gains over 'None' disappear or fall within noise, the published Table 8 gains are leakage-driven; if they persist, the leakage concern is resolved.","verdict_should_be":"REJECT","load_bearing_attack":"The most load-bearing concern is not the transfer of the GLIGEN prior but the evaluation protocol for the central data-expansion claim. In §4.4.3 the SD-Saliency-900 dataset is split 1:5, so only about 150 of 900 pairs are available for training the detection models and the remaining 750 serve as test. Yet §4.1.2 states that DefFiller is fine-tuned using mask-image pairs from the full SD-Saliency-900 dataset, with no restriction to the training split, and §4.4.1 says the DDPM mask producer is trained on the ground truth from the dataset, again all 900 masks. Both the image generator and the mask generator therefore see the test masks (and the image side of the test pairs) before any augmentation pairs are synthesized. When these pairs are added to the 150-sample training set, the detector can learn from synthetic examples that indirectly encode test-set appearance, so the reported improvements (e.g., TSERNet Sα 0.793→0.845, CSEPNet 0.822→0.834) cannot be attributed to DefFiller's augmentation quality. The same ambiguity affects §4.3 if the generator used all ground-truth masks before the 9:1 split. This is a concrete protocol flaw that is independent of whether the natural-image prior transfers.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DefFiller, a mask-conditioned diffusion method for generating steel surface defect images paired with masks. The method fine-tunes a pre-trained GLIGEN model with an added mask encoder and a downsampling network, using a training objective that combines noise prediction with classifier-free guidance. The authors evaluate generation quality with FID and assess downstream utility by training three saliency-based defect detectors (CSEPNet, TSERNet, MINet) on datasets augmented with generated mask-image pairs. They report that DefFiller achieves lower FID than AdaBLDM and improves detection performance after data expansion, e.g., TSERNet S-measure from 0.793 to 0.845 in a 1:5 split experiment.","tokens_in":11502,"tokens_out":5204,"duration_ms":42889,"significance":"If the results hold, DefFiller would be a practical data-augmentation tool for low-data industrial inspection, contributing a mask-conditioned generation method that leverages a natural-image diffusion prior for a specialized defect domain. The paper includes useful components: an ablation of the fine-tuning strategy, a comparison with AdaBLDM, and an evaluation framework that combines FID with downstream detection performance. The code is made available, which supports reproducibility. However, the central data-expansion claim is currently undermined by a train/test leakage problem, and the FID numbers are tuned on the same evaluation data used to report them. These issues need to be resolved before the claimed improvements can be trusted.","major_comments":[{"comment":"The central data-expansion claim is undermined by a train/test leakage. Section 4.4.3 splits SD-Saliency-900 into a 1:5 train/test split for detector evaluation, but Section 4.1.2 states that DefFiller is fine-tuned on mask-image pairs from the full SD-Saliency-900 dataset, and Section 4.4.1 states that the DDPM mask producer is trained on the ground truth from the dataset (all 900 masks). Both generative models therefore see the test masks and the image content of the test pairs before the synthetic pairs are added to the 150-sample training set. The improvements in Table 8 (e.g., TSERNet Sα from 0.793 to 0.845, CSEPNet from 0.822 to 0.834) may then reflect information about the test set encoded in the synthetic pairs rather than DefFiller's augmentation quality. Please rerun the data-expansion experiment with DefFiller and the mask DDPM trained only on the 1:5 training split; the same ambiguity affects Section 4.3, where the generator is trained on the full dataset before the 9:1 substitution split.","section":"§4.4.3 (with §4.1.2, §4.4.1)"},{"comment":"The FID comparison in Tables 3 and 7 is not a fair out-of-sample estimate. Section 3.2.1 states that the guidance scale ω_cfg is iteratively adjusted for each defect category to achieve lower FID, and Table 2 reports FID values across the candidate scales; the final ω_cfg=3 is selected on the same data used to report the headline FID scores. This is selection on the evaluation metric, which inflates the apparent generation quality. Please use a held-out split for guidance-scale selection (or report all candidate FID values and the selection rule) before claiming that DefFiller achieves lower FID than AdaBLDM.","section":"§3.2.1, Table 2"},{"comment":"The abstract claims that DefFiller 'eliminates the need for pixel-level annotations,' but the method is trained on mask-image pairs from SD-Saliency-900 (Section 4.1.2) and the mask producer is trained on the ground-truth masks (Section 4.4.1). The method therefore requires a seed set of pixel-level annotations for training both components; it can produce new paired masks at inference time, but it does not eliminate the need for pixel-level annotations in the development pipeline. Please rephrase the contribution (e.g., 'generates paired masks without additional manual annotation after training') or justify the stronger claim.","section":"Abstract and Section 5"},{"comment":"The reported detection gains are based on single training runs for each network, with no error bars or significance tests. The gains are modest in some cases (CSEPNet Sα from 0.822 to 0.834; MINet from 0.787 to 0.833), and without repeated seeds it is unclear whether the differences are within training noise. Please report means and standard deviations over at least three seeds, and include standard classical augmentation baselines (e.g., random flips/crops/color jitter, Copy-Paste) at the same added-sample count to contextualize the benefit.","section":"§4.4.3, Table 8"}],"minor_comments":[{"comment":"The phrase 'a evaluation framework' should be 'an evaluation framework'.","section":"Section 3.2"},{"comment":"The word 'downsmpling' appears to be a typo for 'downsampling', and Table 1 header 'Adapation' should be 'Adaptation'.","section":"Section 4.1.2 and Fig. 1"},{"comment":"The claim of 'approximately 4%' average S-measure improvement should be stated more precisely: the table shows absolute improvements of 0.012, 0.052, and 0.046, and the percentage depends on the chosen baseline; please clarify the calculation.","section":"Section 4.4.3"},{"comment":"The DFMGAN rows are empty because it does not accept a mask condition; please add a footnote in the table itself for readability.","section":"Table 3"},{"comment":"The text refers to 'blue bars', but the figure appears to be grayscale in the preprint; please ensure that color labels are legible in the final version.","section":"Figure 7"}],"recommendation":"major_revision","confidential_remarks":"The leakage issue appears to be a methodological oversight rather than intentional, and the availability of code makes a corrected rerun feasible. However, a corrected experiment with train-only generative models could change the conclusions, so I cannot recommend acceptance without it. The FID tuning on the evaluation metric is also a fixable protocol issue. I would advise the editor to request a major revision with explicit re-analysis of the data-expansion and FID experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does one thing well: it adapts GLIGEN to mask-conditioned steel defect generation with a simple mask encoder, and the data substitution experiments (Table 5) are a sensible fidelity check. Replacing the real training images with generated ones roughly preserves detection performance, which suggests the generated images are at least in the right ballpark. The authors also ship code, which is appreciated.\n\nThe problem is the data expansion experiment, which carries the main claim. The stress-test note is correct and it is fatal for the current version. DefFiller is fine-tuned on the full SD-Saliency-900 dataset (Section 4.1.2), and the DDPM mask producer is trained on all 900 ground-truth masks (Section 4.4.1). Then in Section 4.4.3 the dataset is split 1:5, so the detection models are trained on about 150 real pairs and evaluated on the remaining 750. But the synthetic pairs added to the 150-sample training set were generated by models that have already seen the test masks and the test images. The gains in Table 8, e.g., TSERNet S-measure 0.793→0.845, could simply reflect the generator having memorized the test distribution. This is not a subtle point; the protocol needs a leakage-free rerun where the generator and mask producer are trained only on the training split.\n\nThe same concern also taints the data substitution experiment, though less directly. There, the detection models are trained entirely on generated images, and the generator has seen the test images during fine-tuning, so the \"maintains performance\" result could be inflated by the generator reproducing test-like images.\n\nOther weaknesses are secondary but real. The guidance scale is tuned per category to minimize FID on the same data used to report FID, which inflates the generation-quality numbers. No error bars or significance tests are given, and no classical augmentation baselines (e.g., copy-paste, geometric transforms) are compared. The abstract's claim that pixel-level annotations are unnecessary is misleading, since masks are pixel-level conditions used in both training and inference.\n\nThe core idea is plausible and worth investigating, but the current evidence does not support the augmentation benefit. I would not cite it as a reliable result until the leakage is fixed and the gains survive a clean split. That said, the paper deserves a serious referee: the problem is important, the method is a reasonable and clearly presented adaptation, and the flaw is fixable with more careful experimentation. I would send it to review with a strong request for revision, not desk-reject it.\n\nWho is this for? Researchers working on industrial defect data augmentation, particularly those considering diffusion-based conditioning. They should read it for the method idea and the evaluation pitfall, not for the reported numbers.","headline":"A clean GLIGEN adaptation for defect generation, but its central data-expansion result is undermined by a training/test leakage that needs a rerun before the claims can be trusted.","tokens_in":12046,"tokens_out":2368,"would_cite":false,"duration_ms":22706,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A diffusion model fine-tuned on 900 steel defect pairs generates mask-matched defects and improves saliency detection in low-data regimes.","keywords":["mask-conditioned defect generation","steel surface defect","saliency-based defect detection","diffusion model","data augmentation","layout-to-image generation","SD-Saliency-900","FID evaluation"],"falsifier":"A direct test would be to fine-tune DefFiller on a different steel defect dataset with equally scarce data and check whether generated images both match masks and improve a detector trained on augmented data; if FID fails to drop below the GAN baseline or detection S-measure does not improve, the transfer claim is refuted. A more controlled test is to train DefFiller from random initialization on the same 900 pairs: if the resulting FID is comparable to the fine-tuned version, the natural-image prior is not load-bearing, whereas if FID collapses, the transfer assumption is supported.","tokens_in":11034,"feed_emoji":"🏭","tokens_out":4531,"duration_ms":36316,"temperature":0.7,"pith_summary":"The paper tries to establish that a diffusion model pre-trained on natural images can be adapted, with a small fine-tuned mask encoder, to generate realistic steel-surface defect images that obey a given binary mask. If true, this gives defect-detection researchers a data augmentation method that needs only mask conditions, not pixel-level manual annotation, and that improves saliency-based detectors in low-data regimes. The paper reports that on SD-Saliency-900, generated images achieve an average FID of 54.40 versus 106.22 for AdaBLDM, and adding 900 generated pairs raises S-measure from 0.822 to 0.834 for CSEPNet, 0.793 to 0.845 for TSERNet, and 0.787 to 0.833 for MINet. The method is framed as the first mask-conditioned defect generation built on a layout-to-image diffusion prior.","feed_headline":"Steel defect images generated from masks lift detection scores","feed_subtitle":"A fine-tuned diffusion model turns binary masks into defect photos that boost saliency detectors in low-data settings.","key_machinery":"The load-bearing mechanism is a layout-conditioned latent diffusion model: a pre-trained GLIGEN is fine-tuned with a trainable mask encoder that converts a binary mask into 64 layout tokens, and a gated self-attention layer that injects these tokens into the U-Net; the mask is also downsampled and concatenated to the noisy latent at the U-Net input. Training minimizes the standard noise-prediction mean squared error while freezing the autoencoder and text encoder, and inference uses classifier-free guidance with the guidance scale set to 3. This design lets the model keep the natural-image diffusion prior while adding pixel-level control.","core_discovery":"The paper's central claim is that DefFiller, a fine-tuned layout-to-image diffusion model with an added mask encoder, can synthesize steel defect images whose defective regions align with user-supplied masks, and that these synthetic mask-image pairs are faithful enough to substitute for real training data and to improve saliency-based defect detectors when the real training set is small. The evidence is FID comparisons (54.40 average versus 106.22 for AdaBLDM on original masks, and 87.68 versus 297.21 on newly generated masks) and detector experiments showing that replacing the real training set with DefFiller images keeps performance close to the original, while expanding a 1:5 training split with 900 DefFiller pairs improves all three tested detectors.","pith_inferences":["If the transfer assumption holds beyond SD-Saliency-900, the same recipe could be applied to other industrial surfaces, such as fabrics or semiconductors, using only mask-image pairs and thereby reducing annotation cost.","The reported S-measure gains might partly reflect added training quantity rather than the fidelity of generated textures; a controlled experiment that adds real mask-image pairs or random image crops could disentangle quantity from quality.","Because the new masks are produced by a separate DDPM, the pipeline's upper bound depends on mask realism, so improving the mask producer could yield further detection gains.","FID uses features trained on natural images, which may not capture steel texture realism; texture-specific perceptual metrics could produce a different ranking of generation methods."],"forward_implications":["Mask-conditioned synthetic pairs can substitute for real training data with minimal performance loss; CSEPNet's S-measure stays at 0.860 against 0.887 on real data.","Expanding a scarce training set with DefFiller pairs improves saliency detection, with the average S-measure across the three tested detectors rising by roughly 4 percent.","The method requires only masks rather than pixel-level defect annotations, so annotators can draw coarse shapes instead of labeling every defective pixel.","DefFiller handles multiple defect classes in one model and generates 300 images per category per run, in contrast to AdaBLDM which generates one defect type per training session.","The evaluation framework ties generation quality, measured by FID, to downstream detection utility, giving a template for judging other mask-conditioned generative models."],"supporting_citations":[{"why":"Supplies the layout-to-image diffusion backbone (GLIGEN) that DefFiller fine-tunes.","marker":"[20]"},{"why":"Provides the SD-Saliency-900 dataset of 900 mask-image pairs used for fine-tuning and evaluation.","marker":"[52]"},{"why":"Is the main comparison baseline (AdaBLDM) for FID and detection performance.","marker":"[11]"},{"why":"Provides the DDPM used to generate new masks for the data-expansion experiments.","marker":"[12]"},{"why":"One of the saliency detectors whose before-and-after performance measures the value of augmentation.","marker":"[5]"},{"why":"A second saliency detector used to assess the generated data's impact.","marker":"[6]"},{"why":"A third saliency detector used to assess the generated data's impact.","marker":"[7]"},{"why":"The SD v1.4 initialization compared against GLIGEN in the training-strategy ablation.","marker":"[56]"}],"fun_headline_variants":["Masks to steel defects: DefFiller boosts detection","Diffusion turns masks into steel defects that train detectors","DefFiller: mask-conditioned generation lifts saliency detection","Synthetic steel defects from masks sharpen low-data detectors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the pre-trained diffusion prior, learned mostly on natural images, can be transferred to steel surface defect textures by fine-tuning on only 900 mask-image pairs for 30,000 iterations.","fun_headline_variants_meta":{"raw":{"variants":["Masks to steel defects: DefFiller boosts detection","Diffusion turns masks into steel defects that train detectors","DefFiller: mask-conditioned generation lifts saliency detection","Synthetic steel defects from masks sharpen low-data detectors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000205,"raw_usage":{"total_tokens":1342,"prompt_tokens":842,"completion_tokens":500,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":458,"completion_tokens_details":{"reasoning_tokens":434}},"tokens_in":458,"tokens_out":500,"duration_ms":4701,"temperature":1.0,"reasoning_tokens":434,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:17:42.352020+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test would be to fine-tune DefFiller on a different steel defect dataset with equally scarce data and check whether generated images both match masks and improve a detector trained on augmented data; if FID fails to drop below the GAN baseline or detection S-measure does not improve, the transfer claim is refuted. A more controlled test is to train DefFiller from random initialization on the same 900 pairs: if the resulting FID is comparable to the fine-tuned version, the natural-image prior is not load-bearing, whereas if FID collapses, the transfer assumption is supported.","supporting_citations":[{"cited_title":"CVPR (2023) 17","cited_arxiv_id":null,"evidence_quote":"Supplies the layout-to-image diffusion backbone (GLIGEN) that DefFiller fine-tunes."},{"cited_title":"IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020)","cited_arxiv_id":null,"evidence_quote":"Provides the SD-Saliency-900 dataset of 900 mask-image pairs used for fine-tuning and evaluation."},{"cited_title":"Measurement 199, 111429 (2022)","cited_arxiv_id":null,"evidence_quote":"One of the saliency detectors whose before-and-after performance measures the value of augmentation."},{"cited_title":"IEEE Transactions on Instrumentation and Measurement 71, 1–12 (2022)","cited_arxiv_id":null,"evidence_quote":"A second saliency detector used to assess the generated data's impact."},{"cited_title":"IEEE Transactions on Industrial Informatics (2024)","cited_arxiv_id":null,"evidence_quote":"A third saliency detector used to assess the generated data's impact."},{"cited_title":"https://huggingface.co/CompVis/stable-diffusion-v1-4 20","cited_arxiv_id":null,"evidence_quote":"The SD v1.4 initialization compared against GLIGEN in the training-strategy ablation."}],"review_version":1}