REVIEW 3 major objections 7 minor 20 references
Evaluating Modern Visual Anomaly Detection Approaches in Semiconductor Manufacturing: A Comparative Study
T0 review · 3 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Modern unsupervised visual anomaly detection transfers to semiconductor SEM images: pretrained feature extractors reach 99.0% image-level ROC-AUC (CFA) and 77.3% pixel-level F1 (STFPM) on the MIIC benchmark.
desk verdict Useful first benchmark of feature-based VAD on semiconductor SEM images, but a load-bearing validation-protocol flaw may inflate the headline numbers. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the MIIC benchmark: 25,276 grayscale SEM images of integrated-circuit metal layers, with 25,160 normal images, 116 defective images, and pixel-level ground-truth masks. Against this benchmark the paper runs feature-embedding VAD methods that use neural feature extractors pretrained on natural photographs—teacher–student methods such as STFPM, memory-bank methods such as PatchCore and CFA (coupled-hypersphere feature adaptation), and normalizing-flow methods such as FastFlow—and scores them with ROC, F1, PR, and the per-region-overlap (PRO) metric. The mechanism carrying the argument is domain transfer: features learned from natural images are applied to SEM patches, and the anomaly maps produced by each method are compared against human-annotated defect regions.
What would settle it
Run CFA and STFPM on MIIC and break down pixel-level F1 by defect type across the 116 defective images; if any defect category is consistently missed while aggregate F1 stays high, the claim that pretrained feature extractors are a robust foundation for localization would be false.
Extended reading notes
Core claim
The paper's central claim is that modern feature-based VAD methods work on semiconductor SEM images despite the domain gap from natural photographs. On MIIC, CFA reaches 99.0% image-level ROC-AUC and 87.96% image-level F1, while STFPM reaches 77.30% pixel-level F1, and the authors conclude that pretrained feature extractors remain a robust foundation for defect detection in this field. The paper also reports that the older inpainting-based reconstruction method IAD+Inpainting remains competitive at the image level (99.27% ROC-AUC, 91.23% F1), which the authors read as evidence that reconstruction-based methods deserve renewed attention when large normal-only training sets are available. A notable domain-specific finding is that method rankings differ from natural-image benchmarks: PaDiM underperforms on MIIC, while STFPM outperforms RD4AD, the reverse of their usual ordering.
Load-bearing premise
The benchmark assumes that the hyperparameters and feature-extraction layers taken from the methods' original natural-image implementations are suitable for MIIC without re-tuning.
Editorial extensions
If this is right
- A semiconductor fab could deploy feature-based VAD without any anomalous training images: the 25,160 normal MIIC images are enough to train detectors that flag the 116 defective images at high ROC-AUC.
- Image-level and pixel-level winners differ—CFA leads detection while STFPM leads localization—so deployment choices depend on whether the priority is flagging bad images or telling an operator exactly where the defect is.
- The high PRO scores for all feature-based methods imply that anomaly maps align closely with ground-truth defect regions, supporting human-in-the-loop review in an Industry 5.0 setting.
- The competitive image-level result of the inpainting baseline suggests reconstruction-based approaches should be re-examined when the normal-only training set is large, rather than being dismissed as obsolete.
- Domain-specific evaluation matters: method rankings on SEM data can invert rankings seen on natural-image benchmarks, so benchmark conclusions should not be assumed to transfer across imaging modalities.
Reading between the lines
- A direct extension of the paper's reasoning is that self-supervised fine-tuning of the feature extractor on unlabeled SEM images should improve feature-based VAD; CFA's feature-adaptation step already points in that direction.
- The inherited hyperparameters and feature-extraction layers from natural-image benchmarks mean the reported rankings are likely conservative for feature-based methods; a validation-set search over backbone layers could change which method leads.
- Because the reconstruction baselines' numbers are taken from the original publication rather than re-run in the same codebase, a head-to-head re-implementation on the same train-test split would be needed to know whether reconstruction truly matches feature-based methods at scale.
- The benchmark covers a single defect class and one metal-layer imaging modality, leaving open whether the conclusions extend to other layers, other process steps, or different defect morphologies in semiconductor fabrication.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript presents a comparative benchmark of visual anomaly detection (VAD) methods on the MIIC dataset of SEM images of integrated circuits. The authors evaluate three reconstruction-based methods (f-AnoGAN, GANomaly, IAD+Inpainting) and seven feature-based methods (PaDiM, STFPM, CFA, PatchCore, RD4AD, FastFlow, SuperSimpleNet), reporting image-level ROC/F1/PR and pixel-level ROC/F1/PR/PRO. The headline finding is that feature-based methods, especially CFA and STFPM, perform strongly on SEM data despite the domain gap from ImageNet, while a reconstruction-based method (IAD+Inpainting) remains competitive. The paper argues that benchmarks should diversify beyond natural-image datasets and that reconstruction-based methods deserve renewed attention.
Significance. If the reported numbers are valid, this is a useful and timely benchmark. MIIC is the largest public SEM VAD dataset, and the paper is among the first to test modern feature-based methods on it. The inclusion of both image-level and pixel-level metrics, and the explicit comparison with reconstruction-based baselines, addresses a real gap in the literature. The authors also honestly disclose the use of original hyperparameters and the PatchCore subsampling. However, the headline comparisons rest on an unsupervised-protocol question and on the mixing of two evaluation pipelines (Section 3.4 and Table 3), so the significance can only be assessed after those issues are resolved.
major comments (3)
- [Section 3.4] The manuscript states that "Half of the test set was considered as a validation set for saving the best model during training." Since the test set contains 116 anomalous images, the validation subset necessarily includes anomalous images. If the best checkpoint is selected using an anomaly-detection metric computed on this subset, then anomalous labels are used during model selection, which contradicts the unsupervised VAD premise and can inflate the feature-based results in Table 3. The paper does not report which validation metric was used. Please disclose the selection criterion; if it is label-dependent, rerun the experiments with a protocol that uses only normal validation data (or last-epoch checkpoints) and report both sets of numbers.
- [Table 3 and Section 4.2] The three reconstruction-based results marked with an asterisk are copied from the original MIIC paper, while the feature-based results were produced with the authors' own pipeline (224x224 resizing, ImageNet normalization, PatchCore training on a 50% subset, validation-based checkpointing). A head-to-head comparison of numbers generated under different preprocessing, metric definitions, and model-selection protocols is not a controlled benchmark. The discussion in Section 4.2 explicitly compares IAD+Inpainting (99.3% ROC, 91.2% F1) with CFA, so this conflation is load-bearing for the central claim. Please re-evaluate the reconstruction methods under the same pipeline or substantially weaken the comparative conclusions.
- [Section 3.4] The use of all hyperparameters and feature-extraction layers from the original MVTec implementations is disclosed, but no sensitivity analysis is provided. Since the benchmark's ranking and the conclusion that pretrained feature extractors are robust on SEM data are point estimates under these inherited choices, it is possible that some rankings (e.g., PaDiM's low performance) reflect hyperparameter transfer rather than intrinsic method behavior. Please add a sensitivity discussion or justify why the MVTec settings are expected to transfer to MIIC.
minor comments (7)
- [Section 3.2] There is a typo in "Approaches basedo on generative models" that should read "Approaches based on generative models."
- [Table 3] The table lists "R4AD" while the text and Table 2 use "RD4AD"; please make the acronym consistent.
- [Sections 1 and 5] The verb "prove" is too strong for an empirical benchmark; "demonstrate" or "provide evidence" would be more accurate.
- [Section 3.3] The PRO metric is referenced to [20], which appears to be a multi-scale memory autoencoder paper; the PRO metric itself was introduced in earlier work by Bergmann et al. Please cite the original source.
- [Section 4.1] The sentence "the F1 score for anomaly detection tasks is better because it takes into consideration the intrinsic problem of unbalanced datasets" is vague; F1 is a harmonic mean of precision and recall and is appropriate for imbalanced data, but the current phrasing is imprecise.
- [Section 3.1] The claim that MIIC is "the only large-scale dataset with SEM images" should be qualified, since NanoTwice also contains SEM images; the distinguishing point is the scale and the integrated-circuit domain.
- [General] No code or evaluation scripts are made available; for a benchmark paper, releasing the evaluation code would substantially improve reproducibility.
Circularity Check
No circularity found: the paper is an empirical benchmark with no derived quantities, no fitted parameters, and no load-bearing self-citation; its conclusions rest on direct evaluation against a public dataset.
full rationale
This paper does not contain a derivation chain in which a prediction is recovered from its own inputs. It evaluates nine existing VAD methods on the public MIIC dataset using standard metrics (ROC, F1, PR, PRO) and reports the measured numbers directly in Table 3. No model is derived from first principles, no parameter is fitted to the test data, and no quantity is defined in terms of another quantity it is supposed to predict. The only self-citation, reference [12] to the authors' PASTE work, appears as a general pointer on edge-device efficiency and is not load-bearing for the central claim that pretrained feature extractors work on SEM images. The reconstruction-based results marked with an asterisk are quoted from the original MIIC paper as comparison baselines, not presented as new predictions, so this is a reproducibility/consistency choice rather than circular reasoning. The validation-split practice described in Section 3.4, where half of the test set is used to select the best model, raises a possible experimental-leakage concern about model selection, but even if true this would be a methodological flaw, not a circularity in the logical sense: the reported scores are not algebraic consequences of the method definitions or of the paper's own assumptions. No uniqueness theorem, ansatz, or renamed empirical pattern is imported from the authors' prior work. Accordingly, the appropriate circularity score is 0.
Assumptions & free parameters
assumptions (3)
- domain assumption ImageNet-pretrained feature extractors produce meaningful representations of SEM microscopy images.
- ad hoc to paper Hyperparameters and feature extraction layers from the original MVTec-based papers transfer to MIIC without tuning.
- domain assumption The MIIC ground-truth anomaly masks and the original train/test split are accurate.
Cite this review
Pith. "Pith review of Evaluating Modern Visual Anomaly Detection Approaches in Semiconductor Manufacturing: A Comparative Study." pith.science (2026). https://pith.science/paper/JVBXDKFL
@misc{pith2026250507576,
author = {Pith},
title = {Pith review of: Evaluating Modern Visual Anomaly Detection Approaches in Semiconductor Manufacturing: A Comparative Study},
year = {2026},
howpublished = {\url{https://pith.science/paper/JVBXDKFL}},
note = {Machine review of arXiv:2505.07576}
}
read the original abstract
Semiconductor manufacturing is a complex, multistage process. Automated visual inspection of Scanning Electron Microscope (SEM) images is indispensable for minimizing equipment downtime and containing costs. Most previous research considers supervised approaches, assuming a sufficient number of anomalously labeled samples. On the contrary, Visual Anomaly Detection (VAD), an emerging research domain, focuses on unsupervised learning, avoiding the costly defect collection phase while providing explanations of the predictions. We introduce a benchmark for VAD in the semiconductor domain by leveraging the MIIC dataset. Our results demonstrate the efficacy of modern VAD approaches in this field.
Figures
Reference graph
Works this paper leans on
-
[1]
Yibo Qiao, Yanning Chen, Fang Liu, Zhouzhouzhou Mei, Yuening Luo, Yining Chen, Yiyi Liao, Bo Wu, and Yongfeng Deng. Ra-unet: A new deep learning segmentation method for semiconductor wafer defect analysis on fine-grained scanning electron microscope (sem) images. IEEE Transactions on Semiconductor Manufacturing , 2025
work page 2025
-
[2]
Yuening Luo, Zhouzhouzhou Mei, Yibo Qiao, and Yining Chen. Scsnet: a novel transformer-cnn fusion architec- ture for enhanced segmentation and classification on high-resolution semiconductor micro-scale defects.Applied Intelligence, 55(6):439, 2025
work page 2025
-
[3]
Yibo Qiao, Zhouzhouzhou Mei, Yuening Luo, and Yining Chen. Deepsem-net: Enhancing sem defect analy- sis in semiconductor manufacturing with a dual-branch cnn-transformer architecture. Computers & Industrial Engineering, 193:110301, 2024. 7
work page 2024
-
[4]
Towards improved semiconductor defect inspection for high-na euvl based on semi-superyolo-nas
Ying-Lin Chen, Jacob Deforce, Vic De Ridder, Bappaditya Dey, Victor Blanco, Sandip Halder, and Philippe Leray. Towards improved semiconductor defect inspection for high-na euvl based on semi-superyolo-nas. In Metrology, Inspection, and Process Control XXXVIII, volume 12955, pages 197–215. SPIE, 2024
work page 2024
-
[5]
Joint anomaly detection and inpainting for microscopy images via deep self-supervised learning
Ling Huang, Deruo Cheng, Yang Xulei, Lin Tong, Shi Yiqiong, Kaiyi Yang, Gwee Bah-Hwee, and Wen Bi- han. Joint anomaly detection and inpainting for microscopy images via deep self-supervised learning. In IEEE International Conference on Image processing (ICIP) , 2021
work page 2021
-
[6]
Mvtec ad — a comprehensive real-world dataset for unsupervised anomaly detection
Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. Mvtec ad — a comprehensive real-world dataset for unsupervised anomaly detection. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9584–9592, 2019
work page 2019
-
[7]
Defect detection in sem images of nanofibrous materials
Diego Carrera, Fabio Manganini, Giacomo Boracchi, and Ettore Lanzarone. Defect detection in sem images of nanofibrous materials. IEEE Transactions on Industrial Informatics , 13(2):551–561, 2016
work page 2016
-
[8]
Mixed supervision for surface-defect detection: from weakly to fully supervised learning
Jakob Bo ˇziˇc, Domen Tabernik, and Danijel Skoˇcaj. Mixed supervision for surface-defect detection: from weakly to fully supervised learning. Computers in Industry, 2021
work page 2021
Show all 20 references
-
[9]
Vt-adl: A vision transformer network for image anomaly detection and localization
Pankaj Mishra, Riccardo Verk, Daniele Fornasier, Claudio Piciarelli, and Gian Luca Foresti. Vt-adl: A vision transformer network for image anomaly detection and localization. In 2021 IEEE 30th International Symposium on Industrial Electronics (ISIE), pages 01–06. IEEE, 2021
2021
-
[10]
Spot-the-difference self- supervised pre-training for anomaly detection and segmentation
Yang Zou, Jongheon Jeong, Latha Pemula, Dongqing Zhang, and Onkar Dabeer. Spot-the-difference self- supervised pre-training for anomaly detection and segmentation. In European Conference on Computer Vision , pages 392–408. Springer, 2022
2022
-
[11]
Reconstruction by inpainting for visual anomaly detection
Vitjan Zavrtanik, Matej Kristan, and Danijel Sko ˇcaj. Reconstruction by inpainting for visual anomaly detection. Pattern Recognition, 112:107706, 2021
2021
-
[12]
Paste: Improving the efficiency of visual anomaly detection at the edge
Manuel Barusco, Francesco Borsatti, Davide Dalle Pezze, Francesco Paissan, Elisabetta Farella, and Gian An- tonio Susto. Paste: Improving the efficiency of visual anomaly detection at the edge. arXiv preprint arXiv:2410.11591, 2024
-
[13]
Student-teacher feature pyramid matching for anomaly detection
Guodong Wang, Shumin Han, Errui Ding, and Di Huang. Student-teacher feature pyramid matching for anomaly detection. arXiv:2103.04257, 2021
2021 arXiv
-
[14]
Anomaly detection via reverse distillation from one-class embedding, 2022
Hanqiu Deng and Xingyu Li. Anomaly detection via reverse distillation from one-class embedding, 2022
2022
-
[15]
PaDiM: A patch distribution mod- eling framework for anomaly detection and localization
Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier. PaDiM: A patch distribution mod- eling framework for anomaly detection and localization. In Pattern Recognition. ICPR International Workshops and Challenges, pages 475–489. Springer International Publishing, 2021
2021
-
[16]
Towards total recall in industrial anomaly detection
Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Sch ¨olkopf, Thomas Brox, and Peter Gehler. Towards total recall in industrial anomaly detection. arXiv:2106.08265, 2022
2022 arXiv
-
[17]
Cfa: Coupled-hypersphere-based feature adaptation for target-oriented anomaly localization
Sungwook Lee, Seunghyun Lee, and Byung Cheol Song. Cfa: Coupled-hypersphere-based feature adaptation for target-oriented anomaly localization. IEEE Access, 10:78446–78454, 2022
2022
-
[18]
Supersimplenet: Unifying unsupervised and supervised learning for fast and reliable surface defect detection
Bla ˇz Rolih, Matic Fu ˇcka, and Danijel Sko ˇcaj. Supersimplenet: Unifying unsupervised and supervised learning for fast and reliable surface defect detection. In International Conference on Pattern Recognition , pages 47–65. Springer, 2025
2025
-
[19]
Fastflow: Unsupervised anomaly detection and localization via 2d normalizing flows, 2021
Jiawei Yu, Ye Zheng, Xiang Wang, Wei Li, Yushuang Wu, Rui Zhao, and Liwei Wu. Fastflow: Unsupervised anomaly detection and localization via 2d normalizing flows, 2021
2021
-
[20]
Improving unsupervised anomaly localization by applying multi-scale memories to autoencoders
Yifei Yang, Shibing Xiang, and Ruixiang Zhang. Improving unsupervised anomaly localization by applying multi-scale memories to autoencoders. arXiv preprint arXiv:2012.11113, 2020. 8
2012 arXiv
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.