REVIEW 3 major objections 4 minor 10 references
Semiconductor SEM Image Defect Classification Using Supervised and Semi-Supervised Learning with Vision Transformers
T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Fine-tuning a pretrained vision transformer can classify semiconductor SEM wafer defects with over 90% accuracy using as few as 5 labeled images per defect class.
desk verdict Useful, honest empirical demonstration that DinoV2 transfer learning works for SEM defect ADC with tiny labeled sets; needs repeated-seed/split runs before the >90% claim is robust. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is DinoV2, a self-supervised vision transformer pretrained on large collections of unlabeled images. The model splits an image into patches, embeds each patch, adds positional information, passes the sequence through a transformer encoder, and outputs an image representation built from the class token and flattened patch tokens. That representation is well suited to the paper's workflow: weighted k-nearest-neighbors on the embeddings gives a strong zero-training baseline, fine-tuning adapts the feature space with a few labeled SEM images per class, and pseudo-labeling then reuses unlabeled fab images to refine the classifier. The t-SNE plots serve as visual evidence that the learned embedding space separates defect types more cleanly after fine-tuning than before.
What would settle it
Run the same fine-tuning protocol on a held-out third inspection layer using 15 labeled images per defect, across at least five random wafer-disjoint train/test splits; if the average accuracy falls below 90% or the spread across splits is large, the few-shot claim as stated is not supported.
Extended reading notes
Core claim
The paper's central claim is that DinoV2, a self-supervised vision transformer pretrained on generic image data, provides features that transfer to semiconductor SEM defect classification with very little labeled data. In experiments on more than 7,400 images of 11 defect types from two inspection layers, out-of-the-box embeddings already separated classes well enough for weighted k-nearest-neighbors to exceed 90% accuracy at 50 (layer 1) or 100 (layer 2) labeled images per class. Fine-tuning on just 5 or 15 labeled images per defect class pushed accuracy above 90% for the two layers, and adding pseudo-labels generated by the fine-tuned model improved accuracy by a further 2-6 percentage points. The authors interpret these results as evidence that a pretrained vision transformer can serve as the basis for an in-house automatic defect classification tool with reduced labeling cost and faster turnaround.
Load-bearing premise
The reported accuracies come from one train/test split per layer without repeated runs or confidence intervals, so the claim that 5-15 labeled images per class reliably gives over 90% accuracy rests on the assumption that this single split is representative.
Editorial extensions
If this is right
- A fab can build a defect classifier with over 90% accuracy using only 5-15 labeled images per defect type, greatly reducing manual labeling effort.
- Unlabeled SEM images are not wasted: adding pseudo-labels gives 2-6 percentage points of accuracy on top of fine-tuning with no additional human annotation.
- The same DinoV2-based pipeline works on two different inspection layers with different defect types, suggesting the method generalizes across layers and wafer levels.
- The k-nearest-neighbors results show that even without any fine-tuning, pretrained embeddings give useful class separation, so a fast baseline classifier can be deployed while labeled data accumulate.
Reading between the lines
- An extension the authors do not test is whether iterative pseudo-labeling, where the model retrains on its own pseudo-labels and re-predicts, compounds the reported 2-6% gain.
- Because only one train/test split per layer is reported, the practical robustness of the 90% figure is not yet established; a natural follow-up is repeated wafer-disjoint splits with confidence intervals.
- The same recipe could be probed on an unseen third inspection layer to test whether the required number of labeled images stays in the 5-15 range or grows with defect complexity.
- Per-class accuracy is not reported; if rare defect types drive the residual errors, class-balanced sampling or targeted augmentation would be a direct next step.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper investigates automatic defect classification (ADC) of semiconductor SEM images using vision transformers, specifically DinoV2. Using real 300mm wafer data from two inspection layers (11 defect types, over 7400 images), the authors evaluate three approaches: k-nearest neighbors on frozen DinoV2 embeddings, fine-tuning DinoV2 with small labeled sets, and semi-supervised learning via pseudo-labels. They report test accuracies over 90% with 5 labeled images per defect class on layer 1 and 15 on layer 2, and claim pseudo-labeling improves accuracy by 2-6% over fine-tuning alone. The paper argues this demonstrates that a pretrained ViT can serve as an effective few-shot classifier for semiconductor defect images, reducing manual labeling effort.
Significance. If the reported results are robust, this work has clear practical value for semiconductor manufacturing: it suggests that a self-supervised pretrained ViT can classify SEM defects with very few labeled examples, potentially cutting labeling cost and turnaround time. The study uses real fab data from two distinct inspection layers, and the wafer-disjoint train/test split is a strong design choice that mitigates data leakage. The held-out test evaluation means the accuracy claims are not circular. The use of DinoV2, which was not pretrained on semiconductor data, also gives the result transferability interest. However, the significance is conditional: the headline few-shot accuracies rest on single-split evaluations without variance estimates, so the robustness of the central quantitative claim is not yet established.
major comments (3)
- [Section IV, Figure 6 and Table II] The central claim of over 90% accuracy with 5 labeled images per class on layer 1 and 15 on layer 2 appears to be based on a single train/test split per layer. With 5-15 training images per class, accuracy can vary considerably with random seed, data split, and initialization. The paper reports no error bars, confidence intervals, or repeated runs. Please provide results across multiple random seeds and/or cross-validation folds, reporting mean and standard deviation, so the robustness of the >90% claim can be assessed.
- [Section IV and Table I] The paper acknowledges that some defect classes occur more frequently than others (Table I) but reports only overall accuracy. In an imbalanced test set, high overall accuracy can mask poor performance on rare defect types. To support the claim that the model classifies all 11 defect types effectively, please include per-class precision, recall, or a confusion matrix, and state explicitly whether the >90% figure holds for the rarer classes as well.
- [Section IV, Figure 6 (pseudo-labels)] The claimed 2-6% improvement from pseudo-labeling is based on the difference between solid and dashed curves in Figure 6. Given the small training set sizes and the absence of any variability measure, this difference could be within run-to-run noise. Please provide error bars or a statistical comparison (e.g., paired significance test over repeated runs) to support the pseudo-labeling improvement claim.
minor comments (4)
- [Abstract and Section IV] The abstract states 'less than 15 images per defect class,' but Section IV reports 15 images per defect for layer 2. This is internally inconsistent; please rephrase to 'up to 15' or 'no more than 15' to match the results.
- [Section III-D] The fine-tuning procedure is described only as using AdamW, cosine scheduler with warmup, and 'varied learning rates.' For reproducibility, please specify the learning rate values, batch size, number of epochs, and warmup steps used for the reported results.
- [Figure 6 caption] The caption does not identify which line style or color corresponds to which layer and which method. Please add a legend or explicitly state that solid lines are fine-tuning and dashed lines are pseudo-labeling for each layer.
- [Section II] The statement 'ViT models, when pretrained on larger datasets, outperform CNNs' is too broad without a specific reference or scope. Please add a citation or qualify the claim to the relevant comparison setting.
Circularity Check
No circularity found: the accuracy claims are empirical evaluations on held-out, wafer-disjoint test sets rather than derivations from fitted inputs.
full rationale
The paper makes empirical performance claims: fine-tuning DinoV2 and adding pseudo-labeling are evaluated on test images that were manually labeled and held out from training, with wafer-level separation between train and test (Section III.A). The k-NN baseline, fine-tuning curve, and pseudo-label results in Figures 5 and 6 and Table II are all measured against this external test set. Pseudo-labels are generated by the model on unlabeled data, but the final reported accuracies are computed against human labels, so the evaluation target is not defined by the model's own outputs. There are no fitted parameters that are renamed as predictions, no load-bearing self-citations, and no uniqueness theorem imported from the authors' prior work. The identified weakness—a single train/test split with no repeated runs, cross-validation, or confidence intervals despite only 5–15 training images per class—is a robustness/statistical limitation, not a circularity. It affects how strongly the >90% claim generalizes, but it does not make the claim true by construction. The derivation chain is therefore self-contained with respect to circularity.
Assumptions & free parameters
free parameters (2)
- Finetuning hyperparameters (learning rate, warmup, epochs)
- Pseudo-label training configuration
assumptions (3)
- domain assumption The labeled training images are representative of the test distribution for each inspection layer.
- domain assumption Features learned by DinoV2 on natural images transfer to SEM images.
- domain assumption Unlabeled images used for pseudo-labeling come from the same distribution as the labeled data.
Cite this review
Pith. "Pith review of Semiconductor SEM Image Defect Classification Using Supervised and Semi-Supervised Learning with Vision Transformers." pith.science (2026). https://pith.science/paper/7N7RWUPF
@misc{pith2026250603345,
author = {Pith},
title = {Pith review of: Semiconductor SEM Image Defect Classification Using Supervised and Semi-Supervised Learning with Vision Transformers},
year = {2026},
howpublished = {\url{https://pith.science/paper/7N7RWUPF}},
note = {Machine review of arXiv:2506.03345}
}
read the original abstract
Controlling defects in semiconductor processes is important for maintaining yield, improving production cost, and preventing time-dependent critical component failures. Electron beam-based imaging has been used as a tool to survey wafers in the line and inspect for defects. However, manual classification of images for these nano-scale defects is limited by time, labor constraints, and human biases. In recent years, deep learning computer vision algorithms have shown to be effective solutions for image-based inspection applications in industry. This work proposes application of vision transformer (ViT) neural networks for automatic defect classification (ADC) of scanning electron microscope (SEM) images of wafer defects. We evaluated our proposed methods on 300mm wafer semiconductor defect data from our fab in IBM Albany. We studied 11 defect types from over 7400 total images and investigated the potential of transfer learning of DinoV2 and semi-supervised learning for improved classification accuracy and efficient computation. We were able to achieve classification accuracies of over 90% with less than 15 images per defect class. Our work demonstrates the potential to apply the proposed framework for a platform agnostic in-house classification tool with faster turnaround time and flexibility.
Reference graph
Works this paper leans on
-
[1]
S. Cheon, H. Lee, C. O. Kim and S. H. Lee, "Convolutional Neural Network for Wafer Surface Defect Classification and the Detection of Unknown Defect Class," in IEEE Transactions on Semiconductor Manufacturing, vol. 32, no. 2, pp. 163 -170, May 2019, doi: 10.1109/TSM.2019.2902657
arXiv 2019
-
[2]
An image is worth 16x16 words: Transformers for image recognition at scale
A. Dosovitskiy, "An image is worth 16x16 words: Transformers for image recognition at scale." arXiv preprint arXiv:2010.11929 (2020)
arXiv 2020
-
[3]
Dinov2: Learning robust visual features without supervision
M. Oquab, "Dinov2: Learning robust visual features without supervision." arXiv preprint arXiv:2304.07193 (2023)
arXiv 2023
-
[4]
Patel, Dhruv V., Ravi Bonam, and Assad A. Oberai. "Deep learning-based detection, classification, and localization of defects in semiconductor processes." Journal of Micro/nanolithography, MEMS, and MOEMS 19.2 (2020): 024801-024801
work page 2020
-
[5]
A CNN -based Transfer Learning Method for Defect Classification in Semiconductor Manufacturing,
K. Imoto, T. Nakai, T. Ike, K. Haruki and Y. Sato, "A CNN -based Transfer Learning Method for Defect Classification in Semiconductor Manufacturing," 2018 International Symposium on Semiconductor Manufacturing (ISSM) , Tokyo, Japan, 2018, pp. 1 -3, doi: 10.1109/ISSM.2018.8651174. Fig. 5. Classification accuracy of k -NN algorithm using vectorized represent...
-
[6]
Semiconductor Wafer Surface: Automatic Defect Classification with Deep CNN,
C. Phua and L. B. Theng, "Semiconductor Wafer Surface: Automatic Defect Classification with Deep CNN," 2020 IEEE REGION 10 CONFERENCE (TENCON) , Osaka, Japan, 2020, pp. 714 -719, doi: 10.1109/TENCON50793.2020.9293715
arXiv 2020
-
[7]
Qiao, Yibo, et al. "DeepSEM -Net: Enhancing SEM defect analysis in semiconductor manufacturing with a dual -branch CNN -Transformer architecture." Computers & Industrial Engineering (2024): 110301
work page 2024
-
[8]
Zuo, W., Zhang, D., & Wang, K. (2008). On kernel difference -weighted k-nearest neighbor classification. Pattern Analysis and Applications , 11, 247-257
work page 2008
Show all 10 references
-
[9]
Visualizing Data using t-SNE
van der Maaten, Laurens, and Geoffrey Hinton. "Visualizing Data using t-SNE." J. Machine Learning Research 9, 2008, pp. 2579–2605
2008
-
[10]
Pseudo -label: The simple and efficient semi - supervised learning method for deep neural networks
Lee, Dong -Hyun. "Pseudo -label: The simple and efficient semi - supervised learning method for deep neural networks." Workshop on challenges in representation learning, ICML. Vol. 3. No. 2. 2013
2013
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.