{"id":"33a3986a-fae1-42c0-b5f9-5707c4639daf","arxiv_id":"2411.19220","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A training-free pipeline with LLM-generated prompts, grounding detection, and CLIP matching raises zero-shot anomaly detection AUROC on MVTec-AD and VisA above the WinCLIP baseline.","lead":"This paper combines three off-the-shelf AI models, GPT-3, Grounding DINO, and CLIP, to detect defects in industrial product images without any training. The method beats the WinCLIP zero-shot baseline on two benchmarks, but the paper omits key implementation details and comparisons.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section III-A's prompt-generation step is the unvalidated black box: x_normal/x_anomaly, the generated prompt sets, and GPT-3 sampling parameters are unreported, so every downstream score in Eq. (4) is conditioned on an unseen stochastic LLM draw.","rationale":"The reader's weakest assumption pinpoints exactly the load-bearing issue: the pipeline's success is gated by GPT-3 generating useful prompts from unspecified inputs, and the paper neither reports those inputs nor the outputs nor any measure of prompt quality. I agree with that assessment. The empirical numbers could be perfectly reproducible, but as written the central claim is not testable because the one component that differentiates this method from WinCLIP is a black box. This is not evidence of misconduct; it is an unverified link in the chain. The ablation studies in Tables IV and V show that removing prompt generation hurts performance, but they do not control for the stochasticity or content of the generated prompts, so they do not close the gap. I also note secondary issues: the ablation text in Section IV-C mislabels which component is being discussed, the conclusion describes a different pipeline than the methodology, recent zero-shot baselines are missing, and no code is released. Those reinforce the conditional verdict but do not change it. A prompt-disclosure and sensitivity experiment would settle whether the concern actually lands: if the method is robust across GPT-3 sampling runs and the inputs are truly minimal, the central claim survives; if not, the headline overstates what is demonstrated.","tokens_in":7408,"tokens_out":4449,"duration_ms":44487,"concrete_test":"Ask the authors to release (i) the exact x_normal and x_anomaly inputs for every MVTec-AD and VisA category, (ii) the full generated P_normal and P_anomaly sets, and (iii) GPT-3 version, temperature, seed, and prompt-count settings. Then independently re-run the pipeline with GPT-3 sampling temperatures in {0.2, 0.7, 1.0}, five seeds each, and report per-class AUROC spread on MVTec-AD. If the overall AUROC varies by more than about 1 point across prompt draws, or if the released prompts turn out to contain hand-added defect terms, the automatic-prompt-generation claim and the claimed margin over WinCLIP are not supported as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central novelty is automatic prompt generation, but Section III-A defines P_normal = GPT3(x_normal) and P_anomaly = GPT3(x_anomaly) without ever specifying x_normal or x_anomaly, showing the generated prompt sets, or reporting temperature, seed, or number of samples. Because the anomaly score in Eq. (4) is a softmax over similarities to exactly these prompt embeddings, the reported 0.932 AUROC and 0.966 AUPR on MVTec-AD depend on one unobserved, stochastic output of a black-box language model. If x_normal and x_anomaly are hand-authored per class with defect names, the 'automatic' claim is overstated; if they are only class names, the method's success hinges on GPT-3 spontaneously producing useful defect concepts, and different generations could change the result. The paper also never explains how the two sets of prompts are collapsed into the single tanomaly and tnormal vectors used in Eq. (4). Without these details, the headline comparison to WinCLIP is not independently reproducible and may reflect one favorable prompt draw rather than a robust method.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a zero-shot, training-free anomaly detection pipeline for industrial images that combines three foundation models: GPT-3 generates normal and anomaly text prompts, Grounding DINO localizes the product in the image, and CLIP scores the similarity between the cropped product image and the generated prompts. The method is evaluated on MVTec-AD and VisA, with reported AUROC/AUPR of 0.932/0.966 and 0.829/0.857, respectively, claimed to outperform the previous zero-shot baseline WinCLIP. The paper also reports ablations that remove the prompt generation and object detection components.","tokens_in":7655,"tokens_out":4325,"duration_ms":34558,"significance":"If the reported results are reproducible, the paper makes a useful practical contribution: it identifies two real limitations of prior CLIP-based zero-shot anomaly detection (fixed template prompts that are not class-specific, and whole-image or sliding-window features that ignore object scale) and proposes a concrete multimodal pipeline to address them. The idea of using LLM-generated class-specific prompts together with grounding-based cropping is sensible, and the improvements over WinCLIP on VisA in particular are encouraging. The paper's strengths are its clear problem framing, the use of two standard benchmarks, and straightforward ablations. However, the central prompt-generation step is described only abstractly, with no disclosure of the LLM inputs, output prompts, or sampling parameters, and the ablation discussion contains inconsistent attributions; these issues currently prevent the results from being independently verified.","major_comments":[{"comment":"The inputs x_normal and x_anomaly are never defined, the generated prompt sets P_normal and P_anomaly are never reported, and no GPT-3 sampling parameters (temperature, number of samples, seed) are given. Because the anomaly score in Eq. (4) is a softmax over CLIP similarities to exactly these generated prompts, the reported results in Tables II and III are conditioned on an unreported stochastic draw from a black-box language model. This makes the comparison to WinCLIP irreproducible and leaves open the possibility that the gains reflect one favorable prompt set rather than a robust property of the method.","section":"III-A, Eqs. (1)-(2)"},{"comment":"The ablation discussion attributes the wrong components. On MVTec-AD, removing object detection (0.932→0.913) is a 1.9pp drop, yet the text credits this to 'the language model-based prompt generation'; removing prompt generation (0.932→0.905) is a 2.7pp drop, yet the text credits it to 'the importance of object-level features'. On VisA the ordering of the two drops is reversed (3.7pp for object detection vs 1.4pp for prompt generation), so the textual attributions are inconsistent with the numbers in both datasets. These sentences must be rewritten so the explanation matches Tables IV and V.","section":"IV-C, Tables IV-V"},{"comment":"Experimental details are insufficient for reproduction. The CLIP backbone variant (e.g., RN50 vs ViT-B/32), the Grounding DINO version and detection threshold, the handling of multiple detected boxes, and the number of prompts per class are all unspecified. The absence of error bars or per-class AUROC/AUPR also makes the claimed margin over WinCLIP statistically unverifiable, since zero-shot prompt-based pipelines are sensitive to prompt selection and random sampling.","section":"IV-A"},{"comment":"The zero-shot comparison is incomplete. Among the baselines in Tables II and III, only WinCLIP is truly zero-shot; SPADE, PaDiM, and PatchCore are 1-shot methods. More recent zero-shot anomaly detection methods (e.g., AnomalyCLIP, WinCLIP+, APRIL-GAN) are not compared, so the claim of state-of-the-art zero-shot performance is not established by the presented evidence.","section":"IV-B"},{"comment":"The sentence 'outperforming the previous best zero-shot method, WinCLIP, by a significant margin of over 1 percentage point in both metrics' is contradicted by Table II: on MVTec-AD the AUPR gain over WinCLIP is 0.001 (0.966 vs 0.965), far below 1 percentage point. The claim should be stated separately per dataset and per metric.","section":"IV-B"}],"minor_comments":[{"comment":"The abstract contains the fragment 'GPT-3. generate text prompts'; the missing 'to' should be inserted.","section":"Abstract"},{"comment":"The phrase 'efficient, accurate, and salable quality control' uses 'salable' where 'scalable' is intended.","section":"Introduction"},{"comment":"The word 'anootated' in the description of MVTec-AD should be 'annotated'.","section":"IV-A"},{"comment":"The references section header appears twice ('REFERENCES REFERENCES'), which should be corrected.","section":"References"},{"comment":"The conclusion misdescribes the pipeline: it says the method 'first employs a deep learning-based anomaly detection model', then an LLM, then object detection, which contradicts the order in Section III (prompt generation, object localization, then CLIP anomaly scoring).","section":"V"},{"comment":"The anomaly score is defined as a ratio of raw dot products. CLIP embeddings are typically normalized to unit vectors, but the paper does not state this, and the ratio is undefined if the denominator approaches zero; please specify the exact similarity computation.","section":"III-C, Eq. (4)"},{"comment":"The related-work text says language models have been used for 'question answering, and sentiment analysis [5, 6]', but references [5] (WinCLIP) and [6] (GPT-3) are not sentiment-analysis papers; the citation is misplaced.","section":"II-B"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a short conference paper with a promising empirical idea, but the central prompt-generation component is described only as a black box. The authors should be given the opportunity to provide the missing prompts, sampling details, and corrected ablations. The scope fit for APSIPA is reasonable, and the topic is appropriate for the venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a plausible incremental paper that combines off-the-shelf GPT-3, Grounding DINO, and CLIP for zero-shot industrial anomaly detection, and it reports a consistent gain over WinCLIP on two benchmarks. The combination is genuinely new relative to the cited prior work, and the ablation shows both new pieces contribute. But the paper leaves the most important detail—the actual prompts generated by GPT-3 and the inputs that produce them—entirely unreported, and its own conclusion describes a different pipeline. That is fixable, but right now the headline numbers are not independently checkable.\n\nWhat it does well: the idea is clean and directly targets WinCLIP's known weaknesses (fixed templates, sliding windows that ignore object scale). The two ablations on MVTec-AD and VisA show that removing either the prompt generation or the object detector drops AUROC by 1–3 points, which is consistent with the claims. The gains over WinCLIP are modest but positive on both datasets.\n\nThe soft spots: the biggest is the prompt generation step. Section III-A defines P_normal = GPT3(x_normal) and P_anomaly = GPT3(x_anomaly) but never says what x_normal and x_anomaly are, never shows the generated prompt sets, and never reports temperature, seed, or number of samples. Since Eq. (4) computes the anomaly score as a softmax over similarities to exactly these prompt embeddings, every reported number is conditioned on an unseen stochastic LLM draw. If the inputs are hand-crafted defect names, the 'automatic' claim is overstated; if they are just class names, the method depends on GPT-3 spontaneously generating useful concepts. Either way, the paper doesn't let a reader replicate it.\n\nAlso, the ablation discussion in Section IV-C swaps the explanations: removing object detection is said to show the importance of prompt generation, and removing prompt generation is said to highlight object-level features. The conclusion in Section V describes a completely different pipeline (first a learned anomaly detector, then LLM-generated prompts for detected anomalies). These are textual errors, but they make me wonder how carefully the writing was checked.\n\nThe paper does not report a CLIP backbone, detection thresholds, or error bars. Those are important for judging the robustness of the 1–5 point gains.\n\nOverall, the core idea is sound and the results are plausible, but the missing details keep it from being reproducible. This is a revise-and-resubmit situation, not a reject.\n\nWho it's for: anyone working on zero-shot anomaly detection or CLIP-based inspection. The idea of using LLM-generated prompts plus grounding detection is worth knowing about.\n\nRecommendation: send it to serious peer review. A good referee will ask for the prompt details, the model variants, and a corrected conclusion and ablation discussion. If the authors supply those, the paper would be a useful incremental contribution.","headline":"Plausible incremental win on zero-shot anomaly detection, but the key prompt-generation step is a black box and the paper's own conclusion describes a different method; needs reporting fixes before I trust the numbers.","tokens_in":8164,"tokens_out":3802,"would_cite":false,"duration_ms":32198,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A zero-shot pipeline combining GPT-3 prompts, Grounding DINO crops, and CLIP scoring beats WinCLIP on industrial anomaly detection.","keywords":["zero-shot anomaly detection","industrial inspection","prompt generation","grounding object detection","CLIP","GPT-3","MVTec-AD","VisA"],"falsifier":"Replace the GPT-3 prompt sets in the published method with WinCLIP's fixed template prompts while keeping the Grounding DINO crop and CLIP scoring; if the AUROC on MVTec-AD does not fall below the reported 93.2%, the prompt-generation component is not the source of the gain and the claim would be falsified. Symmetrically, feed the same GPT-3 prompts to CLIP on whole images without the object crop; if AUROC stays near 93.2%, the grounding component is not load-bearing.","tokens_in":7223,"feed_emoji":"🏭","tokens_out":10402,"duration_ms":75820,"temperature":0.7,"pith_summary":"This paper sets out to show that industrial defect detection can be done without any training data or fine-tuning, by chaining three pretrained models: GPT-3 writes text prompts for what a normal and an anomalous product look like, Grounding DINO crops the product out of the image, and CLIP scores the crop against those prompts. The authors report that this training-free pipeline reaches an AUROC of 93.2% and an AUPR of 96.6% on the MVTec-AD benchmark, and 82.9% and 85.7% on VisA, both above the zero-shot WinCLIP baseline. The point of the approach is that a factory could set up inspection for a new product just by naming its category, without collecting labeled defect examples or training a model. If the reported results hold, it would make quality control cheaper, faster, and easier to deploy across many product lines.","feed_headline":"GPT-3 prompts and object cropping lift anomaly detection to 93.2%","feed_subtitle":"On MVTec-AD and VisA, the three-model pipeline beats WinCLIP without any training.","key_machinery":"The load-bearing mechanism is a three-stage zero-shot pipeline. GPT-3, a large language model, produces the text prompts; Grounding DINO, a text-guided open-set object detector, produces a bounding box and crop of the product; and CLIP, a contrastive image-text model, embeds both text and image and computes the anomaly score via the softmax ratio in Equation (4). The averaging of the global image embedding with the local object-crop embedding is what fuses context with product-specific detail, and the anomaly score formula converts the two prompt similarities into a single probability-like number.","core_discovery":"The central claim is that automatic prompt generation and object grounding together improve zero-shot image anomaly detection over the existing CLIP-based approaches. Instead of relying on fixed human-designed template prompts, the method lets GPT-3 generate two sets of product-specific prompts, $P_{\\text{normal}}$ and $P_{\\text{anomaly}}$, from inputs $x_{\\text{normal}}$ and $x_{\\text{anomaly}}$. Grounding DINO then detects the product in the image and crops it to $I_{\\text{object}}$, and CLIP computes an anomaly score as the ratio of the fused image embedding's similarity to the anomaly prompt over its similarities to both prompts: $s = (e_{\\text{fused}} \\cdot t_{\\text{anomaly}}) / (e_{\\text{fused}} \\cdot t_{\\text{anomaly}} + e_{\\text{fused}} \\cdot t_{\\text{normal}})$, where $e_{\\text{fused}}$ averages the whole-image and object-crop embeddings. The paper reports state-of-the-art zero-shot results on MVTec-AD and VisA, and ablation studies show that removing either the prompt generation or the object detection component degrades performance, supporting the claim that both modules contribute to the gain.","pith_inferences":["The paper never reports the exact inputs $x_{\\text{normal}}$, $x_{\\text{anomaly}}$ or the generated prompt sets, so the method as published is not fully reproducible; releasing these would let others verify that the gain comes from the LLM prompts rather than from the object detector alone.","A natural test of the mechanism's source of gain would be to compare the reported improvements across MVTec-AD categories of different object sizes; if most of the improvement concentrates in categories where the product occupies a small part of the image, the crop step is doing the work.","The same prompt-and-ground architecture could transfer to other inspection domains, such as detecting anomalies in a specific organ in medical scans or in a known structure in satellite imagery, whenever a class name is available for grounding."],"forward_implications":["A factory could set up defect screening for a new product by supplying only the product category name, with no labeled defect images and no model training.","The reported margin over WinCLIP on both MVTec-AD and VisA suggests that class-specific LLM prompts and object-cropped features carry information that fixed templates and sliding windows miss.","The ablation results, showing AUROC drops of roughly 1 to 3 points when either module is removed, indicate that both prompt generation and object localization are needed for the full gain.","Because the pipeline is built from three separate pretrained models, each component can be upgraded independently, so future improvements in LLM prompt quality or open-set detection could translate directly into better anomaly detection."],"supporting_citations":[{"why":"Provides the CLIP image-text encoder used for the final anomaly scoring and the shared embedding space.","marker":"[4]"},{"why":"WinCLIP is the zero-shot baseline the paper compares against and improves on.","marker":"[5]"},{"why":"GPT-3 is the large language model used to generate the normal and anomaly prompts.","marker":"[6]"},{"why":"Grounding DINO provides the open-set object detection that crops the product region.","marker":"[7]"},{"why":"MVTec-AD is the primary benchmark dataset with 15 product categories used for evaluation.","marker":"[21]"},{"why":"VisA is the second benchmark dataset with 20 product categories used to show generalization.","marker":"[22]"}],"fun_headline_variants":["GPT-3 prompts and object cropping beat WinCLIP for zero-shot anomaly detection","Zero-shot anomaly detection without training, just auto prompts and object crops","No training: GPT-3 prompts and object grounding set new zero-shot anomaly detection SOTA","Auto prompts and object crops make zero-shot anomaly detection state-of-the-art"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline depends on the unstated inputs $x_{\\text{normal}}$ and $x_{\\text{anomaly}}$ and on GPT-3 turning them into sensible, object-specific prompt sets, yet the paper never reports these inputs, the prompts, or any quality check, so the anomaly score's numerator and denominator rest on an unvalidated black-box language model.","fun_headline_variants_meta":{"raw":{"variants":["GPT-3 prompts and object cropping beat WinCLIP for zero-shot anomaly detection","Zero-shot anomaly detection without training, just auto prompts and object crops","No training: GPT-3 prompts and object grounding set new zero-shot anomaly detection SOTA","Auto prompts and object crops make zero-shot anomaly detection state-of-the-art"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001524,"raw_usage":{"total_tokens":6125,"prompt_tokens":985,"completion_tokens":5140,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":601,"completion_tokens_details":{"reasoning_tokens":5056}},"tokens_in":601,"tokens_out":5140,"duration_ms":31159,"temperature":1.0,"reasoning_tokens":5056,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:23:35.344516+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the GPT-3 prompt sets in the published method with WinCLIP's fixed template prompts while keeping the Grounding DINO crop and CLIP scoring; if the AUROC on MVTec-AD does not fall below the reported 93.2%, the prompt-generation component is not the source of the gain and the claim would be falsified. Symmetrically, feed the same GPT-3 prompts to CLIP on whole images without the object crop; if AUROC stays near 93.2%, the grounding component is not load-bearing.","supporting_citations":[{"cited_title":"Learn- ing transferable visual models from natural language supervision,","cited_arxiv_id":null,"evidence_quote":"Provides the CLIP image-text encoder used for the final anomaly scoring and the shared embedding space."},{"cited_title":"Language models are few-shot learners,","cited_arxiv_id":null,"evidence_quote":"GPT-3 is the large language model used to generate the normal and anomaly prompts."},{"cited_title":"Spot-the-difference self-supervised pre-training for anomaly detection and segmentation,","cited_arxiv_id":null,"evidence_quote":"VisA is the second benchmark dataset with 20 product categories used to show generalization."}],"review_version":1}