{"id":"f46ba417-497e-4aa1-9227-06ba1aadecd2","arxiv_id":"2507.17359","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"An active learning pipeline with contrastive pretraining and rareness-aware sampling reaches 98% of fully supervised mIoU on semiconductor XRM scans with only about 4.9% of labels.","lead":"This paper tests active learning for labeling defects in semiconductor X-ray scans, combining contrastive pretraining on unlabeled data with a rareness-aware selection rule. It reports reaching 98% of fully supervised segmentation quality using under 5% of the labels.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Rareness term degenerates under max aggregation: any image with a predicted void pixel receives nearly the same r(I), so it cannot rank images by void content; the reported 0.19% gain over Entropy+Feature is within noise and the SOTA claim is not yet tied to the rareness mechanism.","rationale":"The Reader's weakest assumption correctly flags pseudo-label dependence, but the more structural problem is the combination of Eq. (2) with max aggregation: under severe pixel imbalance, r(I) collapses to an approximately constant score for all images with a predicted void pixel. This makes the rareness term a binary presence filter rather than a continuous rarity ranking, and the paper offers no diagnostic showing that the term is doing nontrivial work. The 0.19% mIoU gain over Entropy+Feature in Table 1a is within run-to-run variation, so the empirical claim is fragile even if the mechanism were active. I do not contest the contrastive-pretraining result or the 98%-of-fully-supervised headline, which are supported by Figure 3a and do not depend on the rareness term. Therefore the correct disposition remains CONDITIONAL: the central claim is plausible, but the main novelty needs a decisive ablation and the dataset/code should be released for independent checking. The Reader's verdict is unchanged.","tokens_in":6830,"tokens_out":8853,"duration_ms":95900,"concrete_test":"At budget=200, rerun the Table 1a ablation with the rareness score r(I) replaced by the binary indicator 1[exists x in I with yhat(x)=void], keeping u and d unchanged, and compare both the selected sets and mIoU. If the binary variant selects the same images and matches 78.19% mIoU, the continuous rareness term is not the active ingredient. Additionally, report the fraction of unlabeled images with predicted void per cycle; if this fraction is near 0 or near 1, the term is either inactive or constant.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The rareness-aware acquisition function is the paper's main novelty, but its mechanism is not established. In Eq. (2), r(x)=exp(-p(yhat(x))), where p(c) is a pixel-frequency estimated from pseudo-labels. Because the void class is severely pixel-imbalanced, p(void) is near zero whenever the model predicts any void pixel. Since Eq. (3) uses max aggregation, r(I) then equals approximately exp(-p(void)) for every image containing at least one predicted void pixel; the rareness term becomes a near-binary presence indicator and cannot rank such images by void area or count. Selection among them is driven entirely by u(I)+d(I,L). If the model does not predict void at all, the void class contributes nothing to scores and the rareness term cannot recover the true rare class. The paper does not report the fraction of unlabeled images with predicted void per cycle, does not compare against a binary void-presence selector, and the ablation gain of the rareness term over Entropy+Feature is only 0.19% with overlapping standard deviations (Table 1a). Thus the attribution of the reported state-of-the-art result to the rareness-aware component is not yet supported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript addresses active learning for semantic segmentation of semiconductor XRM images, motivated by annotation cost and by two challenges: domain shift from natural images and severe class imbalance. The authors propose (i) SimCLR-style contrastive pretraining on unlabeled scans to initialize a U-Net at each active-learning cycle, and (ii) a rareness-aware acquisition score that combines a pseudo-label-based pixel-rareness term with entropy uncertainty and feature-space diversity. Experiments on a 4,086-image training set split at scan level compare the method with Random, Entropy, CoreSet, CoreGCN, VAAL, and BADGE; with 200 labeled images (about 4.9% of training data) the method reaches 78.18% mIoU, about 98% of the fully supervised model. The paper reports that contrastive pretraining markedly outperforms ImageNet initialization, and that the rareness term gives a small additional gain over entropy plus diversity.","tokens_in":7071,"tokens_out":6362,"duration_ms":63376,"significance":"If fully supported, the paper would be practically valuable: it applies active learning to a real industrial inspection task, evaluates on a scan-level split that avoids leakage, uses five-run means, and compares against several standard AL baselines. The contrastive-pretraining result is particularly encouraging, with large and likely robust gains in the low-budget regime. However, the novelty attributable to the rareness-aware term is not yet established because the reported gains are small relative to run-to-run variability and because the max-aggregated rareness score degenerates under the rare-void geometry. The paper therefore needs additional analysis before the state-of-the-art claim can be accepted.","major_comments":[{"comment":"The max aggregation makes the rareness term a near-binary presence indicator rather than a graded rarity signal. With p(void) close to zero for the rare void class, any image in which the current model predicts at least one void pixel receives r(I) approximately exp(-0) = 1, whereas the same term for a void-free image is bounded by exp(-p(common)), typically well below 1. Consequently, r(I) cannot order images by void count or area, and selection among void-containing images is driven only by u(I) + d(I,L). This means the mechanism claimed in the abstract and Section 3.2 is not what Eq. (4) actually implements. Please report the fraction of unlabeled images with at least one predicted void pixel per cycle, compare the acquisition function against a variant with a binary void-presence indicator, and consider a sum or class-balanced aggregation if a graded rareness signal is intended.","section":"Sec. 3.2, Eq. (3)"},{"comment":"The ablation table does not unambiguously support the sentence \"rareness term improves mIoU by 1.06% over Entropy, and 0.19% over Entropy+Feature.\" The row \"✓ ✓ 78.00\" is not labeled, and the reported differences are smaller than the standard deviations (e.g., 78.19 ± 0.40 versus 77.92 ± 0.79). Without paired significance tests (for example, paired permutation or bootstrap across the five runs, or additional runs), the incremental contribution of the rareness term over Entropy+Feature is within noise and cannot be claimed. Please label each row explicitly and provide significance tests for all budgets displayed in Fig. 3b.","section":"Table 1a and text after Sec. 4.2"},{"comment":"The paper does not report numeric mIoU means and standard deviations for the baseline methods at each budget, nor significance tests for the pairwise comparisons. Since some differences at budget=200 are likely within one standard deviation, the statement that the method \"consistently outperforms\" other methods is not quantitatively supported. Please include a table with all methods and budgets, report means and standard deviations, and perform paired tests between the proposed method and each baseline.","section":"Fig. 3b and Sec. 4.2"},{"comment":"The rareness score depends on pseudo-labels from the model trained in the previous cycle. In early cycles with very few labels, the model may never predict the rare void class, so p(void)=0 and the rareness term cannot guide selection toward true void-containing images. The paper does not analyze this failure mode or ablate the effect of cycle number. Please report pseudo-label void recall at each cycle and, if possible, the performance of an oracle variant that uses ground-truth void frequency.","section":"Sec. 3.2, pseudo-label reliability"}],"minor_comments":[{"comment":"There is a typo: \"constrastive loss\" should be \"contrastive loss.\"","section":"Sec. 3.1"},{"comment":"The row labels are ambiguous; please use explicit labels such as \"Entropy\", \"Entropy+Feature\", \"Entropy+Rareness\", and \"Entropy+Feature+Rareness\" in the table or caption.","section":"Table 1a"},{"comment":"Please clarify whether p(c) is computed over the unlabeled pool, the labeled set, or all pixels in the training split; the text says \"training set\" but pseudo-labels are used for querying unlabeled images.","section":"Sec. 3.2, Eq. (2)"},{"comment":"The description \"other augmentation techniques and hyper-parameters are kept the same as used in [21]\" is vague; please list the augmentation operations and hyper-parameter values explicitly.","section":"Sec. 4.1"},{"comment":"There is a typo: \"state-of-art-performance\" should be \"state-of-the-art performance.\"","section":"Conclusion"},{"comment":"The error bars are difficult to read at the printed figure size; consider enlarging them or providing a numeric table of all plotted values.","section":"Fig. 3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a compact conference submission and the reported gains are plausible but thin. The main risk is that the rareness term may simply be adding a binary void-presence prior that is hard to distinguish from random variation; the additional analyses requested in the report should settle this. If the authors cannot show the rareness term helps beyond a binary presence signal, I would accept a revised paper that honestly reframes the contribution as a contrastive-pretraining-plus-void-presence active-learning pipeline."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on arXiv:2507.17359. The paper is a competent applied active learning study for semiconductor XRM defect segmentation. The strongest result is not the rareness term; it's the contrastive pretraining: swapping ImageNet initialization for SimCLR pretraining on the unlabeled training data gives a large, consistent boost in early AL cycles. That is a useful and believable finding, and the scan-level train/test split with 5-run means is a solid experimental design.\n\nThe rareness-aware acquisition function is the claimed novelty, but the evidence for it is weak. The ablation shows the rareness term adds 0.19% mIoU over an entropy-plus-feature baseline at budget 200, with overlapping standard deviations. That is noise-level. The stress-test concern about Eqs. (2)-(3) is correct: with max aggregation, any image with a predicted void pixel gets r(I) ≈ 1, so the term is a binary void-presence indicator and cannot rank images by void content. That may be acceptable if the goal is just to ensure rare-class images get selected, but the paper does not check how often the model predicts void on unlabeled images, nor does it compare against a simple void-presence selector. If the model never predicts void in a cycle, the rareness term is blind. So the mechanism is plausible but unverified.\n\nOther soft spots: one private dataset, no code release, no significance tests, and the choice of max over mean aggregation is justified post hoc. These limit how much weight the SOTA claim can carry.\n\nWho is this for? Industrial practitioners working on semiconductor inspection and AL researchers who want a reference point for contrastive pretraining in segmentation AL. It deserves a serious referee: the experimental work is honest, the domain is real, and the contrastive-pretraining result is worth checking. But the rareness-aware component needs stronger validation, and the paper would benefit from releasing code and data (or at least a public benchmark) and from significance testing.\n\nRecommendation: send to peer review, but expect the reviewers to push for better statistical support and a direct test of the rareness mechanism.","headline":"Solid applied AL study; the contrastive pretraining result is real, but the rareness-aware acquisition term is not yet supported by the evidence.","tokens_in":7657,"tokens_out":2422,"would_cite":false,"duration_ms":26416,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a combination of contrastive pretraining on unlabelled X-ray scans and a rareness-aware acquisition function lets active learning reach 98% of fully supervised defect-segmentation performance using only about 4.9%…","keywords":["active learning","semantic segmentation","semiconductor defect detection","X-ray microscopy","contrastive pretraining","class imbalance","rareness-aware acquisition","high bandwidth memory"],"falsifier":"Take the proposed pipeline but construct the first annotated batch so that it contains no void pixels. If, over several active-learning cycles, the rareness term does not pull void-containing images into the labelled set and void mIoU stays near zero, the central claim that rareness-aware selection drives rare-class performance would be refuted.","tokens_in":6592,"feed_emoji":"🔬","tokens_out":6347,"duration_ms":57277,"temperature":0.7,"pith_summary":"This paper asks whether active learning can cut the annotation cost of training deep segmentation models for semiconductor inspection. It identifies two obstacles specific to X-ray microscopy scans: a large domain shift from natural-image pretraining, and severe class imbalance in which the defect class (void) occupies only a tiny area. To counter both, the authors pretrain a U-Net with SimCLR-style contrastive learning on the unlabelled scans and re-use those weights to initialize the model each active-learning cycle, and they add a rareness term to the acquisition function that favors images whose pseudo-labels contain rare classes. On a dataset of X-ray scans of high bandwidth memory structures, the method reaches 78.18% mIoU with 200 annotated images, about 4.9% of the training set, which is 98% of the fully supervised model's score. The paper's claim is that this combination is the current state of the art for this data.","feed_headline":"Active learning hits 98% of full supervision on 5% of labels","feed_subtitle":"With just 200 annotated XRM images, the method matches 98 percent of fully supervised defect segmentation.","key_machinery":"The load-bearing machinery is the rareness-aware acquisition function $s(I)=r(I)+u(I)+d(I,L)$, where $r(I)$ aggregates pixel-wise rareness scores $r(x)=e^{-p(\\hat{y}(x))}$ computed from pseudo-labels of the current model, $u(I)$ is predictive entropy, and $d(I,L)$ is the distance to already selected images, with max pooling used for aggregation. The other half is contrastive pretraining: the segmentation model's decoder output is globally pooled and passed through a two-layer projection head, and an InfoNCE loss is minimized on the unlabelled training set; the resulting encoder-decoder weights initialize every active-learning cycle.","core_discovery":"The central discovery is that both components---contrastive pretraining on the target unlabelled data and rareness-aware selection---are needed to make active learning work under large domain shift and class imbalance. With SimCLR weights instead of ImageNet weights, early-cycle segmentation improves substantially; with the rareness term added to entropy and feature-distance scores, mIoU improves by about 1.06 percentage points over entropy alone at a budget of 200 images. The authors attribute part of the gain to max aggregation of pixel rareness scores, which lets a single rare void pixel drive image selection. At the final tested budget the full pipeline reaches 78.18% mIoU, about 98% of the fully supervised performance, while using only about 4.9% of the training labels.","pith_inferences":["A stress test outside the paper's experiments would start a cycle from a first batch containing no void images; because pseudo-labels would assign zero probability to the void class, the rareness term could not rank void-containing images, so any remaining gain would come from uncertainty and diversity alone.","The mechanism should transfer to other tiny-defect inspection domains, such as medical or materials imaging, but only if the segmentation model's pseudo-labels are confident enough that the estimated class frequencies $p(c)$ are informative.","The contrastive pretraining step costs about 3.5 hours on one V100 GPU; a deployment decision would weigh that fixed cost against the annotation savings, a trade-off the paper does not quantify."],"forward_implications":["With the proposed pipeline, annotating roughly 4.9% of the training images (200 of 4,086) yields 78.18% mIoU, about 98% of the fully supervised upper bound.","Contrastive pretraining on unlabelled domain data matters most when the labelling budget is small, exactly the regime active learning targets.","Adding rareness to uncertainty-plus-diversity selection improves rare-class segmentation on imbalanced semiconductor data without sacrificing overall mIoU.","The method outperforms Random, Entropy, CoreSet, CoreGCN, VAAL, and BADGE at all tested labelling budgets on this dataset."],"supporting_citations":[{"why":"Supplies the SimCLR contrastive learning framework that the paper adapts for segmentation-model pretraining.","marker":"[10]"},{"why":"Defines the InfoNCE loss used as the contrastive objective in the pretraining step.","marker":"[15]"},{"why":"Provides the two-layer MLP projection head design used to compute contrastive feature vectors.","marker":"[16]"},{"why":"Motivates the rareness term by showing that labelling more samples from rare classes improves imbalanced segmentation performance.","marker":"[17]"},{"why":"Establishes the idea of unsupervised pretraining on all data to initialize active-learning cycles, which the paper extends to segmentation.","marker":"[13]"},{"why":"Provides a diversity-based active-learning baseline (CoreSet) that the rareness-aware function is compared against.","marker":"[6]"},{"why":"Provides a hybrid uncertainty-and-diversity baseline (BADGE) that the proposed acquisition function is compared against.","marker":"[9]"},{"why":"Supplies the U-Net architecture used as the segmentation model.","marker":"[18]"},{"why":"Supplies the ResNet-18 backbone used in the segmentation model.","marker":"[19]"}],"fun_headline_variants":["Active learning matches 98% of full supervision on just 5% of labels","Rare-aware active learning hits 98% of full supervision with 5% labels","Contrastive pretraining unlocks 98% AL performance on 5% labels","Semiconductor defect AL matches 98% full supervision at 5% labeling","Contrastive pretraining + rare-aware AL: 98% with 5% labels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The rareness score is built from pseudo-labels produced by the current model, so if that model never predicts the rare void class, the rareness term cannot tell which images actually contain voids and selection falls back to uncertainty and diversity.","fun_headline_variants_meta":{"raw":{"variants":["Active learning matches 98% of full supervision on just 5% of labels","Rare-aware active learning hits 98% of full supervision with 5% labels","Contrastive pretraining unlocks 98% AL performance on 5% labels","Semiconductor defect AL matches 98% full supervision at 5% labeling","Contrastive pretraining + rare-aware AL: 98% with 5% labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000806,"raw_usage":{"total_tokens":3507,"prompt_tokens":879,"completion_tokens":2628,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":495,"completion_tokens_details":{"reasoning_tokens":2520}},"tokens_in":495,"tokens_out":2628,"duration_ms":18946,"temperature":1.0,"reasoning_tokens":2520,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:49:59.677299+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the proposed pipeline but construct the first annotated batch so that it contains no void pixels. If, over several active-learning cycles, the rareness term does not pull void-containing images into the labelled set and void mIoU stays near zero, the central claim that rareness-aware selection drives rare-class performance would be refuted.","supporting_citations":[{"cited_title":"The power of ensembles for active learning in image classification,","cited_arxiv_id":null,"evidence_quote":"Supplies the SimCLR contrastive learning framework that the paper adapts for segmentation-model pretraining."},{"cited_title":"A simple framework for contrastive learning of visual representations,","cited_arxiv_id":null,"evidence_quote":"Provides the two-layer MLP projection head design used to compute contrastive feature vectors."},{"cited_title":"Momentum contrast for unsupervised visual representation learning,","cited_arxiv_id":null,"evidence_quote":"Motivates the rareness term by showing that labelling more samples from rare classes improves imbalanced segmentation performance."},{"cited_title":"Sequential graph convolutional network for ac- tive learning,","cited_arxiv_id":null,"evidence_quote":"Establishes the idea of unsupervised pretraining on all data to initialize active-learning cycles, which the paper extends to segmentation."},{"cited_title":"Automated Detection and Segmentation of HBMs in 3D X-ray Images using Semi-Supervised Deep Learning,","cited_arxiv_id":null,"evidence_quote":"Provides a hybrid uncertainty-and-diversity baseline (BADGE) that the proposed acquisition function is compared against."},{"cited_title":"During each AL cycle, the model is trained with RMSprop optimizer with weighted cross entropy loss","cited_arxiv_id":null,"evidence_quote":"Supplies the ResNet-18 backbone used in the segmentation model."}],"review_version":1}