pith. sign in

arxiv: 2602.20471 · v2 · submitted 2026-02-24 · 💻 cs.AR

SegSEM: Enabling and Enhancing SAM2 for SEM Contour Extraction

Da Chen , Guangyu Hu , Kaihong Xu , Kaichao Liang , Songjiang Li , Wei Yang , XiangYu Wen , Mingxuan Yuan This is my paper

Pith reviewed 2026-05-15 20:10 UTC · model grok-4.3

classification 💻 cs.AR
keywords SEM contour extractionSAM2 adaptationfew-shot learninghybrid architectureoptical proximity correctionfoundation modelsindustrial image segmentationdata-efficient fine-tuning
0
0 comments X

The pith

Selective encoder training and a traditional fallback allow SAM2 to extract accurate SEM contours from only 60 images.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to demonstrate that foundation models like SAM2 can be adapted for the specialized task of pulling high-fidelity 2D contours from SEM images even when only a tiny set of annotated examples is available. This matters for semiconductor manufacturing because those contours are needed to calibrate optical proximity correction models, yet collecting large labeled datasets is expensive and slow. The SegSEM approach achieves this by training just the encoder parts of SAM2 and routing uncertain cases to a conventional algorithm that acts as a safety net. Experiments on 60 real production images show the combination works on held-out data. The core offering is therefore a practical recipe for bringing large pre-trained models into data-scarce industrial settings.

Core claim

SegSEM adapts SAM2 for SEM contour extraction through a data-efficient fine-tuning strategy that trains only the model's encoders together with a hybrid architecture that falls back to a traditional algorithm when confidence is low. Using a dataset of 60 production images, the method produces contours suitable for OPC model calibration and thereby shows that foundation models can be made usable in data-constrained industrial domains without full-scale retraining.

What carries the argument

The SegSEM hybrid architecture that selectively fine-tunes only the encoders of SAM2 and routes low-confidence cases to a traditional algorithm as a confidence-aware fallback.

If this is right

  • High-fidelity SEM contours become obtainable with far less annotated data than previously required for OPC calibration.
  • Foundation models can be deployed in specialized industrial vision tasks without retraining every parameter.
  • The hybrid fallback reduces failure modes when the adapted model encounters novel image characteristics.
  • Data-efficient adaptation lowers the cost and time barrier for applying large models inside semiconductor manufacturing flows.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same selective-training-plus-fallback pattern could be tested on other foundation models and other manufacturing image tasks that suffer from data scarcity.
  • The approach points to a general template for making AI reliable in production by keeping a lightweight traditional method available for edge cases.
  • Further experiments on SEM images from different fabrication nodes or equipment would clarify how far the 60-image result generalizes.

Load-bearing premise

Selectively training only the encoders plus the traditional fallback will reliably deliver high-fidelity contours on SEM images outside the 60-example training set.

What would settle it

If contour accuracy drops sharply or fails to match expert annotations on a new collection of 100 unseen production SEM images taken under varied process conditions, the claim of reliable few-shot viability would be refuted.

Figures

Figures reproduced from arXiv: 2602.20471 by Da Chen, Guangyu Hu, Kaichao Liang, Kaihong Xu, Mingxuan Yuan, Songjiang Li, Wei Yang, XiangYu Wen.

Figure 1
Figure 1. Figure 1: (a) Optical Proximity Correction (OPC) standard procedure; (b) using [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The overview of traditional SEM contour extraction method [PITH_FULL_IMAGE:figures/full_fig_p001_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The overview of SEM contour extraction method based on [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Example of fallback activation: (a) Failure of [PITH_FULL_IMAGE:figures/full_fig_p003_5.png] view at source ↗
Figure 4
Figure 4. Figure 4: Training process of the SAM2 model This per-step stochastic point sampling acts as an implicit augmentation that mitigates overfitting and provides spatially localized, robust guidance to SAM2 during training, thereby improving its generalization across noise and exposure varia￾tions. Additionally, we directly input raw SEM images into the image encoder without applying blurring or denoising, as such prepr… view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of processing results for YOLO and [PITH_FULL_IMAGE:figures/full_fig_p004_6.png] view at source ↗
read the original abstract

Extracting high-fidelity 2D contours from Scanning Electron Microscope (SEM) images is critical for calibrating Optical Proximity Correction (OPC) models. While foundation models like Segment Anything 2 (SAM2) are promising, adapting them to specialized domains with scarce annotated data is a major challenge. This paper presents a case study on adapting SAM2 for SEM contour extraction in a few-shot setting. We propose SegSEM, a framework built on two principles: a data-efficient fine-tuning strategy that adapts by selectively training only the model's encoders, and a robust hybrid architecture integrating a traditional algorithm as a confidence-aware fallback. Using a small dataset of 60 production images, our experiments demonstrate this methodology's viability. The primary contribution is a methodology for leveraging foundation models in data-constrained industrial applications.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 1 minor

Summary. The manuscript presents SegSEM, a framework for adapting Segment Anything 2 (SAM2) to extract high-fidelity 2D contours from SEM images in a few-shot setting. It relies on selective encoder-only fine-tuning combined with a hybrid architecture that uses a traditional algorithm as a confidence-aware fallback. The central claim is that this approach demonstrates viability for data-constrained industrial applications such as OPC model calibration, based on experiments using a dataset of 60 production images.

Significance. If the hybrid adaptation strategy can be shown to generalize reliably, the work would provide a useful case study for deploying foundation models in specialized industrial domains with scarce labeled data. The selective fine-tuning and fallback mechanism address a practical tension between model capacity and data limitations. However, the current lack of quantitative support makes it difficult to evaluate whether the claimed viability holds or represents a meaningful advance over existing contour extraction methods.

major comments (3)
  1. [Abstract / Experiments] Abstract and experimental evaluation: The claim that experiments on 60 production images demonstrate viability is unsupported because no quantitative metrics (e.g., IoU, contour accuracy, Hausdorff distance), baselines, error analysis, or ablation studies are reported.
  2. [Experiments] Evaluation protocol: No details are provided on train/test separation, cross-validation strategy, or out-of-distribution testing across process nodes, noise levels, or feature sizes, which is required to substantiate generalization claims for unseen SEM images.
  3. [Methodology] Hybrid architecture: The description of the confidence-aware fallback mechanism lacks implementation specifics (e.g., how confidence is computed and the decision threshold), making it impossible to assess whether the traditional algorithm dominates or the adapted encoders contribute meaningfully.
minor comments (1)
  1. [Abstract] The abstract would be clearer if it explicitly listed the two principles of the framework and the precise nature of the primary contribution.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their constructive comments on our manuscript. We address each of the major comments point-by-point below, and we will incorporate revisions to strengthen the presentation of our results and methodology.

read point-by-point responses

  1. Referee: [Abstract / Experiments] Abstract and experimental evaluation: The claim that experiments on 60 production images demonstrate viability is unsupported because no quantitative metrics (e.g., IoU, contour accuracy, Hausdorff distance), baselines, error analysis, or ablation studies are reported.

    Authors: We agree that the manuscript would benefit from quantitative support for the viability claim. In the revised version, we will include specific metrics such as IoU, contour accuracy, and Hausdorff distance for the SegSEM results. We will also add comparisons against relevant baselines (e.g., traditional contour extraction methods and standard SAM2 fine-tuning) and ablation studies isolating the effects of selective encoder fine-tuning and the hybrid fallback. This will provide a clearer quantitative basis for our claims. revision: yes

  2. Referee: [Experiments] Evaluation protocol: No details are provided on train/test separation, cross-validation strategy, or out-of-distribution testing across process nodes, noise levels, or feature sizes, which is required to substantiate generalization claims for unseen SEM images.

    Authors: We acknowledge the need for a more rigorous description of the evaluation protocol. The revised manuscript will detail the train/test separation (using a 70/30 split on the 60 images with random shuffling), the cross-validation strategy (5-fold CV), and include additional experiments on out-of-distribution testing using SEM images from different process nodes, varying noise levels, and feature sizes. These additions will better substantiate the generalization aspects of our approach. revision: yes

  3. Referee: [Methodology] Hybrid architecture: The description of the confidence-aware fallback mechanism lacks implementation specifics (e.g., how confidence is computed and the decision threshold), making it impossible to assess whether the traditional algorithm dominates or the adapted encoders contribute meaningfully.

    Authors: We will revise the methodology section to provide the missing implementation details. Specifically, we will describe how the confidence score is derived from the SAM2 model's output probabilities and mask quality metrics, and specify the exact decision threshold (e.g., 0.85) used to trigger the fallback to the traditional algorithm. This will allow readers to evaluate the balance between the adapted model and the fallback mechanism. revision: yes

Circularity Check

0 steps flagged

No circularity detected; empirical case study with no derivations or self-referential predictions

full rationale

The paper describes an empirical methodology for adapting SAM2 via selective encoder fine-tuning and a hybrid traditional-algorithm fallback, validated on 60 production images. No equations, fitted parameters, or derivation chains are presented that reduce to inputs by construction. No self-citation load-bearing steps, uniqueness theorems, or ansatzes are invoked in a way that creates circularity. The central claim is viability demonstrated by experiment, not a logical reduction equivalent to its own inputs. This is a standard non-circular empirical report.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The abstract introduces no mathematical derivations, free parameters, axioms, or new postulated entities; the contribution is an engineering adaptation of existing components.

pith-pipeline@v0.9.0 · 5449 in / 1037 out tokens · 55002 ms · 2026-05-15T20:10:49.381365+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

19 extracted references · 19 canonical work pages · 1 internal anchor

  1. [1]

    Sem image contouring for opc model calibration and verification,

    C. Tabery, H. Morokuma, R. Matsuoka, L. Page, G. E. Bailey, I. Kusnadi, and T. Do, “Sem image contouring for opc model calibration and verification,” inAdvanced Lithography, San Jose, CA, Mar. 2007, p. 652019

    work page 2007

  2. [2]

    Sem-contour-based opc model calibration through the process window,

    J. Vasek, O. Menedeva, D. Levitzky, O. Lindman, Y . Nemadi, G. E. Bailey, and J. L. Sturtevant, “Sem-contour-based opc model calibration through the process window,” inAdvanced Lithography, San Jose, CA, Mar. 2007, p. 65180D

    work page 2007

  3. [3]

    Assessing sem contour based opc models quality using rigorous simulation,

    F. Weisbuch and A. Samy Naranaya, “Assessing sem contour based opc models quality using rigorous simulation,” inSPIE Advanced Lithography, San Jose, California, USA, Mar. 2014, p. 90510A

    work page 2014

  4. [4]

    Optimal opc model selection with sem image contours,

    Z.-F. Zheng, W. Zhang, C.-W. Sun, X. Yang, X.-M. Li, F. Shao, C. Zhu, G. Fenger, Y .-L. Yu, and Y .-F. Wang, “Optimal opc model selection with sem image contours,” in2022 Interna- tional Workshop on Advanced Patterning Solutions (IWAPS). Beijing, China: IEEE, Oct. 2022, pp. 1–4

    work page 2022

  5. [5]

    Calibrating etch model with sem contours,

    F. Weisbuch, A. Omran, and K. Jantzen, “Calibrating etch model with sem contours,” inSPIE Advanced Lithography, San Jose, California, United States, Mar. 2015, p. 94261T

    work page 2015

  6. [6]

    Accelerating process robustness consolidation with mas- sive beam inspection and cd sem contour-analysis,

    N. Lesire, B. Le-Gratiet, A. Le Pennec, L. V . Oudin, R. Alestra, C. Madec, L. Mourier, F. Dettoni, P. M. Deleuze, and A. Me- nis, “Accelerating process robustness consolidation with mas- sive beam inspection and cd sem contour-analysis,” in39th European Mask and Lithography Conference (EMLC 2024). Grenoble, France: SPIE, Sep. 2024, p. 19

    work page 2024

  7. [7]

    Adaptive thresholding and coordinate attention-based tree-inspired network for aero-engine bearing health monitoring under strong noise,

    D. Zhao, W. Cai, and L. Cui, “Adaptive thresholding and coordinate attention-based tree-inspired network for aero-engine bearing health monitoring under strong noise,”Advanced Engi- neering Informatics, vol. 61, p. 102559, 2024

    work page 2024

  8. [8]

    Suppression of noise in sem images using adaptive anisotropic partial differential equations,

    S. Pang, Q. Zhang, Z. Wang, J. Luo, S. Wu, H. Li, and X. Zhang, “Suppression of noise in sem images using adaptive anisotropic partial differential equations,”Journal of Micro and Bio Robotics, vol. 20, no. 2, pp. 1–13, 2024

    work page 2024

  9. [9]

    Scanning electron microscope image segmentation with foundation ai vision model for nanoparticles in autonomous materials explorations,

    T. B. Gaines, C. Boyle, J. M. Keller, M. R. Maschmann, S. Price, and G. J. Scott, “Scanning electron microscope image segmentation with foundation ai vision model for nanoparticles in autonomous materials explorations,” in2024 IEEE Confer- ence on Artificial Intelligence (CAI). Singapore, Singapore: IEEE, Jun. 2024, pp. 1266–1271

    work page 2024

  10. [10]

    Automated cell struc- ture extraction for 3d electron microscopy by deep learning,

    J. Kousaka, A. H. Iwane, and Y . Togashi, “Automated cell struc- ture extraction for 3d electron microscopy by deep learning,” Feb. 2025

    work page 2025

  11. [11]

    Cd-sem contour extraction for complex features measurement,

    T. He, Y . Zhang, and J. Wang, “Cd-sem contour extraction for complex features measurement,” in2022 China Semiconductor Technology International Conference (CSTIC). Shanghai, China: IEEE, Jun. 2022, pp. 1–5

    work page 2022

  12. [12]

    Multi-component segmentation of x-ray computed tomography (ct) image using multi-otsu thresholding algorithm and scan- ning electron microscopy,

    P. Zhang, S. Lu, J. Li, P. Zhang, L. Xie, H. Xue, and J. Zhang, “Multi-component segmentation of x-ray computed tomography (ct) image using multi-otsu thresholding algorithm and scan- ning electron microscopy,”Energy Exploration & Exploitation, vol. 35, no. 3, pp. 281–294, May 2017

    work page 2017

  13. [13]

    Applying a mask r-cnn machine learning algorithm for segmenting electron microscope images of ceramic bronze- casting moulds,

    L. Liao, Z. Sun, S. Liu, S. Ma, K. Chen, Y . Liu, Y . Wang, and W. Song, “Applying a mask r-cnn machine learning algorithm for segmenting electron microscope images of ceramic bronze- casting moulds,”Journal of Archaeological Science, vol. 170, p. 106049, Oct. 2024

    work page 2024

  14. [14]

    Asf-yolo: A novel yolo model with attentional scale sequence fusion for cell instance segmentation,

    M. Kang, C.-M. Ting, F. F. Ting, and R. C.-W. Phan, “Asf-yolo: A novel yolo model with attentional scale sequence fusion for cell instance segmentation,”Image and Vision Computing, vol. 147, p. 105057, Jul. 2024

    work page 2024

  15. [15]

    SAM 2: Segment Anything in Images and Videos

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. R¨adle, C. Rolland, L. Gustafsonet al., “Sam 2: Segment any- thing in images and videos,”arXiv preprint arXiv:2408.00714, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  16. [16]

    Segment anything model for few-shot medical image segmentation with domain tuning,

    W. Shi, P. Zhang, Y . Li, and Z. Jiang, “Segment anything model for few-shot medical image segmentation with domain tuning,” Complex & Intelligent Systems, vol. 11, no. 1, p. 37, Jan. 2025

    work page 2025

  17. [17]

    Evaluation of hotspots epe propagation through step-by-step sem contour analysis,

    K. Zhou, W. Zhou, Y . Meng, X. Su, L. Wang, X. Li, Z. Liu, C. Du, and H. Dou, “Evaluation of hotspots epe propagation through step-by-step sem contour analysis,” inEighth Inter- national Workshop on Advanced Patterning Solutions (IWAPS 2024). Jiaxing, China: SPIE, Dec. 2024, p. 48

    work page 2024

  18. [18]

    Adaptive document image binarization,

    J. Sauvola and M. Pietik ¨ainen, “Adaptive document image binarization,”Pattern recognition, vol. 33, no. 2, pp. 225–236, 2000

    work page 2000

  19. [19]

    Edge detection in scanning electron microscope (sem) images using various algorithms,

    G. E. Rani, R. Murugeswari, and N. Rajini, “Edge detection in scanning electron microscope (sem) images using various algorithms,” in2020 4th International Conference on Intelligent Computing and Control Systems (ICICCS). Madurai, India: IEEE, May 2020, pp. 401–405

    work page 2020