{"id":"8dbcb897-ff37-4316-a23b-acb3ee79c13a","arxiv_id":"2505.09263","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A few-shot embedding guides a frozen latent diffusion model to generate realistic industrial defects inside bounding boxes, and a weakly supervised loss improves DRAEM and DeSTSeg on MVTec, pending clarification of test-set overlap.","lead":"This paper generates fake industrial defect images from just a few real defect photos using a frozen diffusion model, then uses those images to improve anomaly detectors. It reports AU-PR gains on MVTec for DRAEM and DeSTSeg, but the evaluation may be contaminated because the few real defects come from the same test set.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Main risk: Section 5.1 samples support anomalies from MVTec without stating a hold-out, so the Table 1 AU-PR gains may partly reflect test-set leakage rather than genuine few-shot generation.","rationale":"I read the paper in good faith: the three-stage pipeline is coherent, the embedding-learning loss with mask guidance (Eq. 6) is a sensible way to capture local anomaly structure, and the weak-supervision loss (Eq. 9) with confident-normal filtering is a reasonable way to handle bounding-box labels. The ablations, especially the support-count and mask-guided-loss studies, add useful evidence. However, the central empirical claim rests on the integrity of the data protocol in Section 5.1. The paper never states whether the support anomalies are disjoint from the MVTec test set, and MVTec's structure makes this a live and testable risk: all real anomaly images live in the test split, so sampling from them without explicit hold-out means the learned embedding is trained on the evaluation distribution. The leakage concern is not a speculative philosophical point; it directly explains why the downstream gains could be large, and Table 2a shows the choice of support set alone shifts the main metric by 1.8 AU-PR points, larger than the smaller of the two headline improvements. The reader's CONDITIONAL verdict is appropriate: the architecture and method are plausible, but the central quantitative claim cannot be accepted until the hold-out rule is stated and verified. No other concern I found is as load-bearing. The introduction's misstatement of the DRAEM baseline and the missing variance reporting are real issues but secondary; the leakage question is the one that, if answered adversely, would overturn the headline result.","tokens_in":12327,"tokens_out":3071,"duration_ms":33836,"concrete_test":"Ask the authors to release the exact support-set image IDs and rerun the evaluation with a clean protocol: (1) for each of the 73 MVTec anomaly types, list the image IDs used as support; (2) remove all support images from the MVTec test set before computing Table 1, and ideally also hold out the same defect instances from the generation stage; (3) repeat the full pipeline with 3-5 random support draws and report mean plus standard deviation. If the DRAEM pixel AU-PR gain drops substantially below 5.8 when support images are excluded, the headline improvement is inflated by test leakage. Publishing the split is the single most informative check.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract, Table 1) is that generated anomalies improve DRAEM and DeSTSeg on both classification and segmentation, with the headline numbers DRAEM pixel AU-PR 67.4 to 73.2 and DeSTSeg 76.6 to 78.1. The mechanism is the embedding learned from a few support anomalies via Eq. 5-6. Section 5.1 states: \"we randomly select 3 anomalous images from the real anomalies to be the support anomalies,\" but it never specifies how those images relate to the MVTec test set used in Table 1. MVTec contains anomalous images only in its test split, so unless the authors deliberately held those images out of evaluation, the support set is drawn from the evaluation distribution. The embedding is then optimized with the support anomalies' ground-truth masks (Eq. 6), making the generated anomalies semantically and often visually close to the test anomalies. This gives AnoGen an advantage that DRAEM and CutPaste baselines, which synthesize anomalies from DTD textures, do not have. The absence of any hold-out rule therefore threatens the validity of the 5.8 and 1.5 point AU-PR improvements. The concern is amplified by the paper's own Table 2a: different support sets change pixel AU-PR by 1.8 points (73.2 vs 71.4), which is larger than the reported DeSTSeg improvement, and no variance over support draws is reported. If the support images are MVTec test images, the reported gains may not transfer to a realistic few-shot deployment where the support anomalies are sampled before seeing test anomalies.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AnoGen, a three-stage few-shot anomaly generation framework. In Stage 1, an embedding vector is learned by minimizing a masked diffusion loss (Eq. 6) on a few real anomaly images with their segmentation masks. In Stage 2, this embedding conditions a pre-trained latent diffusion model to inpaint anomalies into normal images at user-specified bounding-box locations. In Stage 3, the generated images and their bounding boxes are used to train DRAEM and DeSTSeg with a weakly supervised loss (Eq. 9) that suppresses loss for high-confidence normal pixels inside the box. Experiments on MVTec report consistent improvements for both models on image-level and pixel-level metrics, with the headline result being a 5.8 percentage-point improvement in pixel AU-PR for DRAEM (67.4 to 73.2) and 1.5 points for DeSTSeg (76.6 to 78.1). Ablations study support-set choice, number of support anomalies, mask-guided loss, confidence threshold tau, mixing with synthetic anomalies, and number of generated images N.","tokens_in":12657,"tokens_out":2363,"duration_ms":23779,"significance":"If the empirical claims hold, the paper would make a useful contribution: it demonstrates that a tiny learnable embedding (768 parameters) can inject a few-shot anomaly concept into a diffusion model and that the resulting generated anomalies can improve discriminative anomaly segmentation models. The method is simple, parameter-efficient, and the authors release code and generated data, which are concrete assets for reproducibility. The mask-guided embedding loss and the bounding-box weak-supervision loss are reasonable design choices that could be reused by other works. However, the significance is currently conditional on resolving the data-protocol question described in the major comments: because MVTec contains anomalous images only in its test split, the support set used for embedding learning may overlap with the evaluation set, which would make the reported gains partly an artifact of test-distribution leakage rather than genuine few-shot generation. The paper does not provide error bars, multiple-seed results, or a sensitivity analysis over support draws, so the robustness of the headline numbers is not established.","major_comments":[{"comment":"The paper does not state how the three support anomalies selected in Section 5.1 relate to the MVTec test set used in Table 1. MVTec contains anomalous images only in its test split, so sampling '3 anomalous images from the real anomalies' draws directly from the evaluation distribution unless those images are explicitly held out. Because the embedding is optimized on these support anomalies (Eq. 5-6) and then used to generate training images, the evaluation on the same test set can reward memorization of test-set appearance. This is load-bearing for the central claim that generated anomalies improve DRAEM and DeSTSeg. Please specify the exact hold-out rule (e.g., per defect type, the support images are excluded from evaluation, or the generated set is ensured not to include test images) and, ideally, re-run the main comparison under a protocol where the support set is chosen before seeing any test images.","section":"5.1, Eq. (5)-(6), Table 1"},{"comment":"The reported numbers are internally inconsistent. The abstract says DRAEM and DeSTSeg achieved a 5.8% and 1.5% improvement in AU-PR, which matches Table 1 (67.4 to 73.2 and 76.6 to 78.1, respectively). However, the introduction states that DRAEM improves 'from 67.4% to 76.6%' and DeSTSeg 'from 73.2% to 78.1%', which swaps the baseline values of the two methods. This discrepancy must be corrected, as it currently undermines trust in the reported results.","section":"Abstract, Introduction, Table 1"},{"comment":"Table 2a shows that different support sets change pixel AU-PR by 1.8 points (73.2 vs. 71.4), which is larger than the reported DeSTSeg improvement of 1.5 points. No variance over support draws is reported, and the main experiment uses a single randomly drawn support set. Please report mean and standard deviation over multiple support-set draws, and state how many draws were used for the main numbers in Table 1.","section":"Table 2a, Section 5.1"},{"comment":"The confidence threshold tau is selected on the test set: Table 3 reports results for tau = 1.0, 0.95, 0.90, 0.80 and the best value (0.90, with 73.2 AU-PR) is used for the main comparison. Since tau is a hyperparameter of the proposed weak-supervision loss, tuning it on the evaluation set can inflate the reported improvement. Please either fix tau without looking at test performance or perform selection on a validation split and report the resulting test numbers.","section":"Table 3, Section 6.2"}],"minor_comments":[{"comment":"There are multiple typos and inconsistent name spellings: 'DseTSeg', 'DesTSeg', and 'DeSTSeg' are all used; 'addnosing' should be 'adding noise' in Section 3. These should be unified and corrected.","section":"Abstract and full text"},{"comment":"The entry '70.5.' for the 1-shot pixel AU-PR contains a stray period; this is presumably '70.5' and should be cleaned.","section":"Table 2b"},{"comment":"The x-axis of Figure 5 is labeled 'N' but the caption does not clarify whether N is the number of generated images per object per anomaly type or another quantity. Please make the axis label and caption explicit.","section":"Figure 5"},{"comment":"The inpainting update in Eq. (7) is described as replacing the area outside the box with a noisy version of the input, but the formula conditions on a bounding-box mask Mbox that is not explicitly defined as binary. Please define the mask convention (0/1 for inside/outside the box) so that the equation is unambiguous.","section":"Section 4.2, Eq. (7)"},{"comment":"Reference [42] is cited for the pre-trained LDM but points to a CVPR 2018 paper on conditional GANs ('High-resolution image synthesis and semantic manipulation with conditional GANs'). This appears to be the wrong reference; please cite the actual latent diffusion model paper.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core idea is reasonable and the paper is clearly written, but the data-protocol issue is serious enough that the main empirical claim cannot be accepted as is. If the authors can demonstrate a clean hold-out rule and provide variance estimates, the paper could become a solid contribution. The internal inconsistency in the reported numbers and the tau selection also need to be addressed. I would not reject outright, because the concern is fixable within the manuscript's scope, but the current version does not support the strong claims in the abstract."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this for the evaluation protocol, not for the diffusion novelty. The three-stage pipeline — textual inversion against a frozen LDM, mask-guided inpainting, then confidence-filtered weak supervision — is sensible and the engineering is clean. The mask-guided embedding loss (Eq. 6) and the weak-supervision loss (Eq. 9) are genuinely useful additions over AnomalyDiffusion, which already did few-shot generation. Credit is also due for honest ablations, open code, and a stated limitation of box-level supervision.\n\nThe load-bearing problem is the data protocol in Section 5.1. MVTec has anomalous images only in its test split, and the paper says \"we randomly select 3 anomalous images from the real anomalies\" without ever saying those images were held out of the Table 1 evaluation. So the support set is drawn from the evaluation distribution, and the embedding is optimized on it (Eq. 5-6). That makes the 5.8 and 1.5 point AU-PR gains hard to interpret — they may partly reward memorization of test appearances rather than genuine few-shot generation. Table 2a shows different support sets shift pixel AU-PR by 1.8 points, which is larger than the DeSTSeg gain, and no error bars or seed variation are reported. That is not a fatal flaw, but it is a result-level uncertainty that the paper needs to resolve.\n\nThere are also smaller credibility hits. The introduction says DRAEM goes from 67.4 to 76.6 P-AUPR, but Table 1 says DRAEM goes to 73.2; 76.6 is the DeSTSeg baseline. That internal inconsistency suggests a copy-paste error, but in a results-driven paper it matters. The pre-trained LDM citation [42] points to Wang et al. (pix2pixHD), not Rombach et al.; the wrong reference undermines trust in the reproducibility chain.\n\nIs the central idea sound? Yes. The weak-supervision trick is a reasonable way to use box-level annotations, and the combination with mixed synthetic/real anomalies is practical. But the missing hold-out rule and the lack of variance reporting mean the headline numbers should not be taken at face value yet. If the authors fix those two issues, the method is a credible incremental contribution to industrial anomaly detection.\n\nRecommendation: send this to peer review. It deserves serious refereeing, not a desk reject. The concerns are fixable, and the method is useful enough to warrant the referee time.","headline":"Read this for the evaluation protocol, not for the diffusion novelty: the pipeline is clean, but the headline AU-PR gains rest on a support-set selection that likely leaks MVTec test data into training.","tokens_in":13236,"tokens_out":2015,"would_cite":false,"duration_ms":19492,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AnoGen learns a 768-parameter anomaly embedding from three real defects; generated images lift DRAEM segmentation AU-PR from 67.4 to 73.2.","keywords":["anomaly detection","anomaly generation","diffusion model","few-shot learning","weakly supervised learning","anomaly segmentation","MVTec","embedding learning"],"falsifier":"Re-run the Table 1 experiment with support anomalies drawn only from a held-out partition of MVTec test anomalies, so that the evaluated images cannot appear in the embedding-learning set, and check whether DRAEM's pixel AU-PR still rises from 67.4 to 73.2; if the gain shrinks toward noise, the effect depends on test leakage.","tokens_in":12099,"feed_emoji":"🏭","tokens_out":8165,"duration_ms":75925,"temperature":0.7,"pith_summary":"AnoGen tries to close the gap between synthetic and real defects in industrial anomaly detection. The paper's claim is that with only three real anomaly images per defect type, a pre-trained latent diffusion model can be steered by a learned 768-dimensional embedding to generate realistic and diverse anomalies on normal MVTec images, with the anomaly region controlled by a bounding box. Those generated images, combined with standard synthetic anomalies and a weakly-supervised loss that ignores high-confidence normal pixels inside the box, improve both image-level and pixel-level anomaly detection for the discriminative models DRAEM and DeSTSeg. The headline numbers are a pixel-level AU-PR rise for DRAEM from 67.4 to 73.2 and for DeSTSeg from 76.6 to 78.1, alongside smaller gains in classification. If true, this would make scarce defect collections usable as training signal for segmentation-grade anomaly detectors.","feed_headline":"Three defects lift DRAEM's AU-PR from 67.4 to 73.2","feed_subtitle":"AnoGen guides a frozen diffusion model with a 768-parameter embedding and box-level labels, improving DRAEM and DeSTSeg on MVTec.","key_machinery":"The load-bearing object is a 768-dimensional embedding $v$, initialized from the CLIP text embedding of the word 'defect' and optimized with a mask-weighted latent-diffusion noise-prediction loss. It carries the anomaly concept into the frozen network via cross-attention, and the bounding-box inpainting (Eq. 7) gives spatial control and a free weak label. The weak-supervision loss (Eq. 9) with confidence filter $\\delta$ prevents normal pixels inside the box from being trained as anomalies. The pre-trained latent diffusion model itself is a denoising network whose weights are never updated, so the only learnable parameters in the generation stage are those of the embedding.","core_discovery":"On MVTec, the paper shows that the anomaly distribution of a defect type can be compressed into a single embedding $v$ by optimizing the latent diffusion noise-prediction loss over a few real anomaly images $I_a^T$ and their segmentation masks $M_a^T$: $v^* = \\arg\\min_v \\mathbb{E}[\\|(\\epsilon - \\epsilon_\\theta(\\mathcal{E}(I_a^T),t,v)) \\odot M_a^T\\|^2]$. The learned $v$ is then injected through cross-attention into a frozen pre-trained latent diffusion model; inpainting with Eq. (7) places the anomaly inside an arbitrary bounding box on a normal image. For the downstream model, the box is treated as a weak label, and the weakly-supervised loss $L'_{\\rm seg} = M_{\\rm box} \\odot (1-\\delta) \\odot L_{\\rm seg} + (1-M_{\\rm box}) \\odot L_{\\rm seg}$ zeros out high-confidence normal predictions (predicted normal probability $\\hat p \\ge \\tau=0.9$) inside the box. The paper's central discovery is that this combination—generated anomalies plus box supervision—simultaneously improves DRAEM's and DeSTSeg's image-level AU-ROC/AU-PR and pixel-level AU-ROC/AU-PR, with the largest absolute gain in pixel-level AU-PR: 67.4 to 73.2 for DRAEM and 76.6 to 78.1 for DeSTSeg.","pith_inferences":["Because the embedding is only 768 parameters and the diffusion backbone is frozen, the per-defect-type cost is small; one could generate counterfactual placements of the same defect on different normal objects to probe detector robustness without collecting new data.","The weak-supervision loss (Eq. 9) is not specific to DRAEM or DeSTSeg; any pixel-level detector trained on box-annotated pseudo-anomalies could adopt the confident-normal filtering trick, so the recipe may generalize beyond the two architectures tested.","The authors' stated limitation—bounding boxes rather than pixel masks—suggests a next step: use the generated images to bootstrap a refinement network that predicts dense masks from the box, removing the extra hyperparameter $\\tau$."],"forward_implications":["Generated anomalies improve both classification and segmentation of DRAEM and DeSTSeg on MVTec, with segmentation benefiting more (5.8pp vs 1.5pp pixel AU-PR).","Using generated anomalies together with DRAEM's original DTD-based synthetic anomalies gives the best result (73.2 pixel AU-PR), rather than replacing the old synthetic data.","Three support anomalies are nearly as good as five; one support image degrades diversity and downstream performance.","Mask-guided embedding learning is necessary; without the mask the embedding captures the whole object and fails to generate the defect.","The confidence threshold $\\tau$ matters; $\\tau=0.9$ balances ignoring normal pixels and learning anomalous pixels inside the box."],"supporting_citations":[{"why":"Supplies the MVTec AD dataset used for all generation and evaluation.","marker":"[6]"},{"why":"DRAEM is the first base anomaly detection model whose training is augmented with generated anomalies.","marker":"[47]"},{"why":"DeSTSeg is the second base model; its improvements validate the generated anomalies with a different architecture.","marker":"[49]"},{"why":"Provides the idea of learning an embedding with a frozen diffusion model, adapted here with a mask-guided loss.","marker":"[15]"},{"why":"Cited as the source of the pre-trained latent diffusion model kept frozen in stages 1 and 2.","marker":"[42]"},{"why":"Supplies the blended latent diffusion inpainting procedure used to place anomalies inside bounding boxes.","marker":"[3]"},{"why":"The CLIP text encoder provides the 'defect' word embedding that initializes $v$.","marker":"[30]"},{"why":"GrabCut is used to constrain bounding box placement so it overlaps the foreground region.","marker":"[34]"},{"why":"Justifies using AU-PR as the preferred metric for pixel-level evaluation under class imbalance.","marker":"[10]"}],"fun_headline_variants":["Few-shot AnoGen lifts DRAEM AU-PR to 73.2","AnoGen: few defects yield big segmentation gains on MVTec","Embedding injection boosts DRAEM and DeSTSeg anomaly detection","Few-shot diffusion improves DRAEM AU-PR by 5.8 points"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central assumption is that the few support anomalies used to learn the embedding are not themselves evaluation images: MVTec's anomalous images live only in its test split, and the paper never states a rule for holding those images out, so the reported AU-PR gains could be inflated if test defects leak into the generation stage.","fun_headline_variants_meta":{"raw":{"variants":["Few-shot AnoGen lifts DRAEM AU-PR to 73.2","AnoGen: few defects yield big segmentation gains on MVTec","Embedding injection boosts DRAEM and DeSTSeg anomaly detection","Few-shot diffusion improves DRAEM AU-PR by 5.8 points"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000608,"raw_usage":{"total_tokens":2942,"prompt_tokens":1168,"completion_tokens":1774,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":784,"completion_tokens_details":{"reasoning_tokens":1692}},"tokens_in":784,"tokens_out":1774,"duration_ms":13729,"temperature":1.0,"reasoning_tokens":1692,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:36:18.390922+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the Table 1 experiment with support anomalies drawn only from a held-out partition of MVTec test anomalies, so that the evaluated images cannot appear in the embedding-learning set, and check whether DRAEM's pixel AU-PR still rises from 67.4 to 73.2; if the gain shrinks toward noise, the effect depends on test leakage.","supporting_citations":[{"cited_title":"In: ICCV (2021)","cited_arxiv_id":null,"evidence_quote":"DRAEM is the first base anomaly detection model whose training is augmented with generated anomalies."},{"cited_title":"In: CVPR (2018) AnoGen 17","cited_arxiv_id":null,"evidence_quote":"Cited as the source of the pre-trained latent diffusion model kept frozen in stages 1 and 2."},{"cited_title":"In: CVPR (2022)","cited_arxiv_id":null,"evidence_quote":"Supplies the blended latent diffusion inpainting procedure used to place anomalies inside bounding boxes."},{"cited_title":"TOG23(3) (2004)","cited_arxiv_id":null,"evidence_quote":"GrabCut is used to constrain bounding box placement so it overlaps the foreground region."}],"review_version":1}