Pith. sign in

REVIEW 4 major objections 5 minor 27 references

Zero-Shot Industrial Anomaly Segmentation with Image-Aware Prompt Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read IAP-AS claims that per-image, LLM-generated prompts improve zero-shot anomaly segmentation by up to 10% in F1-max across seven industrial datasets.

desk verdict The prompt-generation idea is real and the gains are plausible, but the size threshold is computed from the test image itself, which confounds the zero-shot claim and needs clarification before the numbers can be trusted. read the letter →

arxiv 2504.13560 v1 pith:RFNM5Q3J submitted 2025-04-18 cs.CV cs.AI

SoYoung Park , Hyewon Lee , Mingyu Choi , Seunghoon Han , Jong-Ryul Lee , Sungsu Lim , Tae-Ho Kim This is my paper · ORCID
classification cs.CVcs.AI
keywords zero-shotanomalysegmentationimage-awarepromptgenerationlargelanguagemodelimagetaggingindustrialinspectiontext-guideddefectlocalization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that zero-shot industrial anomaly segmentation can be improved by replacing fixed text prompts such as "defect" or "abnormal" with prompts generated from the content of each individual image. The proposed IAP-AS pipeline tags an image with object words, sends those tags plus a fixed instruction to a large language model, and uses the resulting context-aware phrase list to drive a detection-and-segmentation model. Across seven industrial datasets the method reports F1-max gains of up to 10 percentage points over fixed-prompt and few-shot baselines, and first or second place on most benchmarks. If true, this means the content of the prompt, not the model architecture, may be the main remaining lever for zero-shot defect localization.

What carries the argument

The central object is the Image-Aware Prompt (IAP), a text prompt generated per image by merging object tags from an image tagging model with an LLM instruction that asks for anomalous nouns; the final prompt combines these LLM-generated adjective clauses, the object tag set, and fixed words like "abnormal" and "defect". This prompt carries the argument: it is fed to an open-set detector, whose boxes are filtered by a size threshold, and the surviving boxes are given to a promptable segmentation model whose masks are weighted by confidence to produce the anomaly score.

What would settle it

Run the full IAP-AS pipeline on any dataset while computing the size threshold from a separate set of normal images instead of from the test image; if the F1-max gain over fixed prompts shrinks toward zero, the reported improvement comes from test-image information in the threshold, not from the prompt generation itself.

Watch

Extended reading notes

Core claim

The central claim is that a prompt built from the image's own tags—for example, leather, brown, fabric, material—expanded by an LLM into a list like cut, hole, split, blemish, tear, mark, lets an open-vocabulary detector find defects that a generic "abnormal" prompt misses. The paper attributes the gain to three cooperating components: image tagging, LLM-based prompt expansion, and a size filter that removes detections larger than a per-image threshold. On the MVTec-AD dataset the fully integrated system reaches 40.13 F1-max versus 24.73 for the same pipeline without these components, and it reports improved or competitive results on six other industrial datasets that include both objects and textures.

Load-bearing premise

The method assumes that a size threshold computed from the very image being tested, using only normal-object tags, is a legitimate zero-shot filter rather than a way of leaking information from the test image into the prediction.

Editorial extensions

If this is right

  • Replacing fixed prompts with image-tag-derived prompts is a drop-in change for text-guided zero-shot anomaly segmentation pipelines, since it does not require retraining.
  • Because prompts are generated per image, the method can adapt to products whose surface material or object type changes across a production line, where a single fixed prompt would fail.
  • Image tagging, LLM prompting, and size filtering are complementary: the ablation shows each component contributes, and the full combination performs best.
  • The gains hold across both object and texture industrial datasets, suggesting the mechanism is not specific to one defect type.
  • The method's output masks are weighted by detector confidence, giving a per-pixel anomaly score suitable for downstream quality-control decision rules.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A fair comparison would need to verify whether the per-image size threshold is computed from the test image itself; if so, part of the reported gain may be test-time adaptation, and a clean zero-shot evaluation should derive the threshold from normal images only.
  • Because the tags come from a fixed-vocabulary image tagging model, images containing objects outside that vocabulary may produce poor prompts; extending the approach with a tagging model that outputs free-form attributes could be tested.
  • The same prompt-generation mechanism could in principle be applied to other text-guided dense prediction tasks, such as referring segmentation or open-vocabulary detection, by swapping the anomaly-specific instruction in the LLM query.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes IAP-AS, a zero-shot anomaly segmentation method that generates image-aware text prompts by combining an image tagging model (RAM) and an LLM (LLaMA-3-8B), then feeds these prompts to a Grounding DINO + SAM pipeline. The claimed contribution is that dynamic, context-aware prompts replace fixed prompts and improve F1-max by up to about 10% over existing zero-shot baselines across seven industrial datasets. The method comprises a preprocessing stage (object tagging, size-threshold extraction, LLM prompting) and a segmentation stage (anomaly region generation, size filtering, SAM masking, score aggregation). The paper reports quantitative results in Table 1, an ablation in Table 3, and qualitative examples.

Significance. If the claims hold, the work would demonstrate a simple and potentially impactful way to improve zero-shot anomaly segmentation: replacing manually fixed prompts with automatic, image-conditioned prompt generation. The paper releases code, uses several standard industrial anomaly datasets, and includes an ablation study. These are positive features. The central question is whether the reported gains are actually attributable to the image-aware prompting mechanism or to a test-image-derived size filter that the comparison methods do not receive. Because that confound is not resolved, the significance of the contribution is currently uncertain but the underlying idea is worth exploring.

major comments (4)
  1. [Algorithm 1 lines 7 and 14] The size threshold Sthreshold is computed on line 7 as GroundingDINO(Ia, Tn), where Ia denotes the anomalous (test) images, and is then used on line 14 to filter detections from those same images. This means the method uses information from the test image itself to determine which detection boxes to keep, which is a form of test-time adaptation rather than a pure zero-shot procedure. The comparison methods in Table 1 do not have access to such a per-image adaptive filter. The abstract and Section 5.2 attribute the F1-max gains to the image-aware prompts, but the ablation in Table 3 shows that the largest jump occurs only when filtering is combined with tagging (O O O: 40.13 vs. O O X: 33.08), and that LLM prompting without tagging (X O X: 33.66) outperforms tagging with LLM but without filtering (O O X: 33.08). This interaction pattern is consistent with the test-derived threshold, not the LLM prompt, being the decisive component. The authors should recompute Sthreshold from normal images only (or fix it a priori) and re-run the experiments and ablation to verify whether the prompt-based gains persist without this test-image leak.
  2. [§5.1] The implementation details in Section 5.1 state that 'a size threshold of 0.8 filtered out unusually sized bounding boxes,' which suggests a fixed constant. This directly contradicts Algorithm 1, line 7, where Sthreshold is computed per test image from GroundingDINO(Ia, Tn), and also contradicts the Step 1-2 text, which says the threshold is the maximum size of detected regions in the input image. Additionally, the Step 1-2 text says the threshold 'filters out smaller regions,' whereas Eq. (3) defines J = {i | bi < Sthreshold}, which retains only boxes smaller than the threshold. These inconsistencies make the reported configuration ambiguous and the method impossible to reproduce as described. Please clarify which threshold definition is actually used and provide the exact filtering rule.
  3. [Table 1] No error bars, standard deviations, or significance tests are reported. Since the method involves a random selection of normal images (random value r in Algorithm 1), the results may vary across runs, and the claimed improvements (e.g., up to 10.99% F1-max on BTAD) may lie within noise. The authors should report mean and standard deviation over multiple random seeds and, ideally, a significance test for the headline comparisons against SAA+ and the CLIP-based baselines.
  4. [§5.1] The comparison is not fully controlled: WinCLIP, SAA+, and SDP are described as using 'dataset-specific Few-Shot prompts combined with object names,' while IAP-AS additionally uses a per-image test-derived size threshold. To isolate the contribution of the image-aware prompt generation, the authors should add an ablation variant that uses the IAP-AS pipeline with a fixed size threshold (or no size filtering) and compares against a variant that uses the test-derived threshold without the LLM-generated prompts. This would directly address whether the reported gains come from the prompts or from the adaptive filter.
minor comments (5)
  1. [Keywords] The keyword 'Large language model (LMM)' appears to be a typo; it should read 'Large language model (LLM)' for consistency with the rest of the text.
  2. [Table 1] The caption contains a typo: 'trainig-free' should be 'training-free'.
  3. [§4.1] The description of the size threshold is unclear: it says 'the threshold is calculated as the maximum size of detected regions in the input image Ia' but does not specify how 'size' is measured (area, width, height, or diagonal). Please define the size metric explicitly.
  4. [§4.1] The notation Pad_j is introduced but the index j is not defined, and the relationship between Pad_j and the final prompt P_final is described only in words. Please define the composition of P_final formally.
  5. [§3] The problem definition states that segmentation masks are generated with SAM and anomaly scores are computed as a weighted sum, but it does not define how the confidence scores si are obtained when multiple masks overlap or how the final pixel-level score is thresholded for AP/F1-max computation. Please clarify the evaluation protocol.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation; the test-image-derived size threshold is a leakage confound, not a circular step.

full rationale

The central derivation chain is self-contained: normal-image tags T_n are extracted by RAM, combined with an LLM to form P_final, GroundingDINO produces candidate boxes from P_final, and SAM converts filtered boxes to masks. The paper compares against external fixed-prompt baselines and includes a fixed-prompt ablation, so the image-aware prompt claim has independent empirical content. The only data-dependent parameter that weakens the zero-shot label is S_threshold, computed in Algorithm 1 line 7 as GroundingDINO(I_a, T_n) from the anomalous/test image itself and then applied in Eq. (3) to filter detections from the same image. The ablation (Table 3) shows the filtering step produces the largest jump (33.08 to 40.13 on MVTec-AD), so part of the reported gain may come from per-image test-time adaptation rather than from the prompt-generation mechanism. This is a correctness and attribution concern, not a circularity in the derivational sense: the final masks are not equivalent by construction to the prompt or to the threshold, and the LLM-prompting component alone also improves over the fixed-prompt baseline (24.73 to 33.66). There is no self-citation chain, no imported uniqueness theorem, no ansatz smuggled via citation, and no renaming of a known result as a new derivation. F1-max is an oracle threshold metric, but it is applied equally to all compared methods. The score reflects the mild test-time confound rather than a genuine circular derivation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on off-the-shelf foundation models and a set of hand-chosen thresholds. The most fragile piece is the size threshold, whose definition is inconsistent and which appears to be derived from the test image itself. No new physical or mathematical entities are introduced.

free parameters (4)
  • Box and text thresholds = 0.2 for object datasets, 0.1 for texture datasets
    Manually chosen per dataset category; these thresholds directly control which detection boxes survive and can inflate F1-max on the evaluated benchmarks.
  • Size threshold Sthreshold = Maximum detected region size from the test image (or 0.8 as a fixed ratio in Section 5.1)
    Algorithm 1 computes this from the anomalous test image Ia, and the paper gives inconsistent definitions across Section 4.1, Equation 3, and Section 5.1. This is a data-dependent filtering choice.
  • IoU threshold = 0.5
    Removes overlapping detection boxes; no sensitivity analysis is reported.
  • Random seed r = 111
    Selects the normal image used for tagging; no multiple-seed or variance analysis is reported.
assumptions (5)
  • domain assumption Grounding DINO can localize arbitrary anomaly nouns generated by LLaMA on industrial images.
    This is the core of Step 2-1; the paper does not calibrate or analyze failures of the open-vocabulary detector on the generated nouns.
  • domain assumption RAM image tagging produces correct, useful object tags on normal industrial images.
    Step 1-1 feeds tags into both threshold computation and LLM prompting; incorrect tags propagate through the whole pipeline.
  • domain assumption LLaMA-3-8B outputs non-redundant anomaly nouns that match the visual appearance of defects in the target domain.
    Step 1-3 relies on the LLM to invent useful prompt words, but the paper does not evaluate prompt quality or coverage of possible defect types.
  • domain assumption SAM produces accurate pixel masks from Grounding DINO bounding boxes.
    Step 2-3 assumes that box-prompted SAM segments the defective region faithfully enough for the reported metrics.
  • domain assumption F1-max and AP computed on test ground truth are appropriate summary measures of zero-shot performance.
    F1-max selects the best threshold over the test set, which is an oracle metric and may not reflect deployment conditions with a fixed threshold.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Zero-Shot Industrial Anomaly Segmentation with Image-Aware Prompt Generation." pith.science (2026). https://pith.science/paper/RFNM5Q3J

@misc{pith2026250413560,
  author       = {Pith},
  title        = {Pith review of: Zero-Shot Industrial Anomaly Segmentation with Image-Aware Prompt Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RFNM5Q3J}},
  note         = {Machine review of arXiv:2504.13560}
}
read the original abstract

Anomaly segmentation is essential for industrial quality, maintenance, and stability. Existing text-guided zero-shot anomaly segmentation models are effective but rely on fixed prompts, limiting adaptability in diverse industrial scenarios. This highlights the need for flexible, context-aware prompting strategies. We propose Image-Aware Prompt Anomaly Segmentation (IAP-AS), which enhances anomaly segmentation by generating dynamic, context-aware prompts using an image tagging model and a large language model (LLM). IAP-AS extracts object attributes from images to generate context-aware prompts, improving adaptability and generalization in dynamic and unstructured industrial environments. In our experiments, IAP-AS improves the F1-max metric by up to 10%, demonstrating superior adaptability and generalization. It provides a scalable solution for anomaly segmentation across industries

Figures

Figures reproduced from arXiv: 2504.13560 by the authors.

Figure 1
Figure 1. Example of the text prompt generation process for an industrial image, where object tags extracted from the image are combined with Image-Aware Prompt (IAP) and processed by an LLM to create context-aware prompts for anomaly segmentation. Model (SAM) [9] and Contrastive Language-Image Pre-training (CLIP) [18] to recognize object attributes in images and align image features with semantic information using text promp… view at source ↗
Figure 2
Figure 2. Overview of the proposed IAP-AS framework, which operates in two stages: Preprocessing and Anomaly Segmentation. The Preprocessing stage includes image tagging, size threshold extraction, and LLM-based prompt generation. The Anomaly Segmentation stage involves anomaly region detection, filtering, segmentation, and anomaly score computation. an anomaly score Scorea for each anomalous image. The segmentation masks Ma … view at source ↗
Figure 3
Figure 3. Visual comparison of IAP-AS and other models across four datasets, highlight￾ing differences in anomaly segmentation results. 5.3 Qualitative Results A qualitative evaluation was conducted to validate the anomaly segmentation performance of IAP-AS using industrial datasets. Two object datasets (MVTec￾AD [3] and MPDD [8]) and two texture datasets (KSDD1 [19] and MTD [6]) were selected for the assessment [PITH_FULL_I… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 21 canonical work pages

  1. [1]

    In: WACV

    Aota, T., Tong, L.T.T., Okatani, T.: Zero-shot versus many-shot: Unsupervised texture anomaly detection. In: WACV. pp. 5564–5572 (2023), https://drive. google.com/drive/folders/10OyPzvI3H6llCZBxKxFlKWt1Pw1tkMK1/

    work page 2023

  2. [2]

    In: ICCV

    Bao, T., Chen, J., Li, W., Wang, X., Fei, J., Wu, L., Zhao, R., Zheng, Y.: Miad: A maintenance inspection dataset for unsupervised anomaly detection. In: ICCV. pp. 993–1002 (2023)

    work page 2023

  3. [3]

    In: CVPR

    Bergmann, P., Fauser, M., Sattlegger, D., Steger, C.: Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection. In: CVPR. pp. 9592–9600 (2019), https://www.mvtec.com/downloads/

    work page 2019

  4. [4]

    arXiv preprint arXiv:2305.10724 (2023)

    Cao, Y., Xu, X., Sun, C., Cheng, Y., Du, Z., Gao, L., Shen, W.: Segment any anomaly without training via hybrid prompt regularization. arXiv preprint arXiv:2305.10724 (2023)

    arXiv 2023

  5. [5]

    In: IJCAI

    Chen, X., Zhang, J., Tian, G., He, H., Zhang, W., Wang, Y., Wang, C., Liu, Y.: Clip-ad: A language-guided staged dual-path model for zero-shot anomaly detec- tion. In: IJCAI. pp. 17–33 (2024)

    work page 2024

  6. [6]

    The Visual Computer 36(1), 85–96 (2020), https://github.com/abin24/ Magnetic-tile-defect-datasets./

    Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36(1), 85–96 (2020), https://github.com/abin24/ Magnetic-tile-defect-datasets./

    work page 2020

  7. [7]

    In: CVPR

    Jeong, J., Zou, Y., Kim, T., Zhang, D., Ravichandran, A., Dabeer, O.: Winclip: Zero-/few-shot anomaly classification and segmentation. In: CVPR. pp. 19606– 19616 (2023)

    2023

  8. [8]

    In: ICUMT

    Jezek, S., Jonak, M., Burget, R., Dvorak, P., Skotak, M.: Deep learning-based defect detection of metal parts: evaluating current methods in complex conditions. In: ICUMT. pp. 66–71 (2021),https://github.com/stepanje/MPDD/

    work page 2021

Show all 27 references
  1. [9]

    In: ICCV

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., et al.: Segment anything. In: ICCV. pp. 4015–4026 (2023) 12 S.Y. Park et al

    2023

  2. [10]

    In: CVPR

    Lee, M., Choi, J.: Text-guided variational image generation for industrial anomaly detection and segmentation. In: CVPR. pp. 26519–26528 (2024)

    2024

  3. [11]

    In: AAAI

    Li, J., Wang, X., Zhao, H., Wang, S., Zhong, Y.: Anomaly segmentation for high- resolution remote sensing images based on pixel descriptors. In: AAAI. pp. 4426– 4434 (2023)

    2023

  4. [12]

    Neurocomputing618, 129122 (2025)

    Li, S., Cao, J., Ye, P., Ding, Y., Tu, C., Chen, T.: Clipsam: Clip and sam collabo- ration for zero-shot anomaly segmentation. Neurocomputing618, 129122 (2025)

    2025

  5. [13]

    In: WACV

    Li, Y., Goodge, A., Liu, F., Foo, C.S.: Promptad: Zero-shot anomaly detection using text prompts. In: WACV. pp. 1093–1102 (2024)

    2024

  6. [14]

    In: ECCV

    Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Jiang, Q., Li, C., Yang, J., Su, H., et al.: Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In: ECCV. pp. 38–55 (2025)

    2025

  7. [15]

    In: CVPR

    Lüddecke, T., Ecker, A.: Image segmentation using text and image prompts. In: CVPR. pp. 7086–7096 (2022)

    2022

  8. [16]

    In: NeurIPS

    Ma, H., Zhang, C., Bian, Y., Liu, L., Zhang, Z., Zhao, P., Zhang, S., Fu, H., Hu, Q., Wu, B.: Fairness-guided few-shot prompting for large language models. In: NeurIPS. pp. 43136–43155 (2023)

    2023

  9. [17]

    In: ISIE

    Mishra, P., Verk, R., Fornasier, D., Piciarelli, C., Foresti, G.L.: Vt-adl: A vision transformer network for image anomaly detection and localization. In: ISIE. pp. 01–06 (2021), http://avires.dimi.uniud.it/papers/btad/btad.zip

    2021

  10. [18]

    In: ICML

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: ICML. pp. 8748–8763 (2021)

    2021

  11. [19]

    Journal of Intelligent Manufacturing31(3), 759–776 (2020), https://www.vicos.si/resources/kolektorsdd/

    Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing31(3), 759–776 (2020), https://www.vicos.si/resources/kolektorsdd/

    2020

  12. [20]

    arXiv preprint arXiv:2302.13971 (2023)

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al.: Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  13. [21]

    In: CVPR

    Wang, Z., Lu, Y., Li, Q., Tao, X., Guo, Y., Gong, M., Liu, T.: Cris: Clip-driven referring image segmentation. In: CVPR. pp. 11686–11695 (2022)

    2022

  14. [22]

    Wieler, M., Hahn, T., Hamprecht, F.A.: Weakly supervised learning for in- dustrial optical inspection (2007),https://conferences.mpi-inf.mpg.de/dagm/ 2007/prizes.html

    2007

  15. [23]

    Computer-Aided Civil and Infrastructure Engineering38(11), 1536–1554 (2023)

    Yong, G., Jeon, K., Gil, D., Lee, G.: Prompt engineering for zero-shot and few- shot defect detection and classification using a visual-language pretrained model. Computer-Aided Civil and Infrastructure Engineering38(11), 1536–1554 (2023)

    2023

  16. [24]

    In: GeoAI@SIGSPATIAL

    Zhang, J., Zhou, Z., Mai, G., Hu, M., Guan, Z., Li, S., Mu, L.: Text2seg: Zero- shotremotesensingimagesemanticsegmentationviatext-guidedvisualfoundation models. In: GeoAI@SIGSPATIAL. pp. 63–66 (2024)

    2024

  17. [25]

    In: CVPR

    Zhang, Y., Huang, X., Ma, J., Li, Z., Luo, Z., Xie, Y., Qin, Y., Luo, T., Li, Y., Liu, S., et al.: Recognize anything: A strong image tagging model. In: CVPR. pp. 1724–1732 (2024)

    2024

  18. [26]

    arXiv preprint arXiv:2406.20076 (2024)

    Zhang, Y., Cheng, T., Hu, R., Liu, L., Liu, H., Ran, L., Chen, X., Liu, W., Wang, X.: Evf-sam: Early vision-language fusion for text-prompted segment anything model. arXiv preprint arXiv:2406.20076 (2024)

    2024 arXiv

  19. [27]

    Computers & Industrial Engineering 177, 109045 (2023)

    Zipfel,J.,Verworner,F.,Fischer,M.,Wieland,U.,Kraus,M.,Zschech,P.:Anomaly detection for industrial quality assurance: A comparative evaluation of unsuper- vised deep learning models. Computers & Industrial Engineering 177, 109045 (2023)

    2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.