{"id":"8c41b1cc-8847-4e43-a01d-dc8846aa0350","arxiv_id":"2506.04717","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"An enhanced SegGPT model with scribble-based prompting auto-labels display defects, reaching about 60 percent coverage with downstream performance comparable to human labels.","lead":"This paper describes an AI system that automatically labels defects in display panel images after a human draws a rough scribble on one example. The authors report that the auto-labels are good enough to train downstream defect detectors that match human-annotation performance.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1 may compare only the ~60% of images that passed the coverage threshold, so the claimed match with human labels does not yet establish usability on the full data distribution.","rationale":"I read this as an industrial application paper whose central scientific claim is empirical: auto-labels are as usable as human labels for training a downstream defect segmenter. The strongest support is Table 1, and the most load-bearing condition for that comparison is the composition of the auto-labeled training set. The paper is transparent that one-shot auto-labeling only passes its own quality gate on about 60% of images (Section 3, Fig. 5), but it does not spell out whether the Figure 6 comparison inherits that filter. If it does, the equivalence result is consistent with a much weaker claim: auto-labels are usable only on the subset where the auto-labeler is already confident. The reader's weakest_assumption identifies exactly this issue, and I agree. I do not see a more central threat: the absence of ablations weakens attribution of which proposed component helps but not the headline equivalence itself, and the reliance on proprietary data is a reporting limitation rather than a logical flaw. The check I propose would settle the selection concern in one experiment. Since this is an addressable ambiguity and not evidence that the result is false, the CONDITIONAL verdict should stand, hence UNCHANGED.","tokens_in":4390,"tokens_out":5519,"duration_ms":67359,"concrete_test":"Report the exact composition of the auto-labeled training set used in Figure 6: number of images, fraction that passed the coverage threshold, and how rejected images were handled. Then retrain the downstream segmentation model on the full single-pass auto-labeled training set, without filtering by IoU or HD, and evaluate on the same test set used for Table 1 over multiple seeds. Compare IoU and recall against both the human-labeled baseline and the filtered auto-label model; if the unfiltered model drops materially, the reported equivalence is an artifact of coverage-based selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the Table 1 equivalence between downstream models trained on human labels and on auto-labels. The paper defines coverage as IoU>0.60 and HD<10 and reports ~60% average coverage (Fig. 5), and Section 2 says rejected labels are returned to the unlabeled pool. The Figure 6 workflow caption says 'Pick corresponding human annotated images' but never states whether the 'Auto Labelled Images' branch contains all images, only the covered 60%, or the result of multiple human-in-the-loop re-prompting rounds. If the downstream model was trained only on images whose auto-labels already passed the quality threshold, then Table 1 is an upper bound on an easy subset and says nothing about the 40% of images that would still require manual labeling or further processing. The absence of error bars or repeated runs makes the 0.01 IoU and 0.33% recall gaps hard to interpret as well. This selection ambiguity is the load-bearing issue because the practical claim that auto-labeling can replace manual annotation depends on the composition of the auto-labeled training set.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an AI-assisted auto-labeling system for display panel defect segmentation based on an enhanced SegGPT model. The authors propose a two-stage training pipeline with adaptive cropping/spatial ensembling, multiclass training with dynamic color assignment, selective label suppression, and a scribble-based prompting mechanism using a new learnable token. They report that the enhanced model improves IoU by 0.22 and recall by 14% over the pretrained SegGPT baseline across three LCD product types, with about 60% average auto-labeling coverage (IoU>0.60, HD<10). A downstream experiment compares models trained on human-labeled versus auto-labeled data, claiming equivalent performance (IoU 0.85 vs 0.84, recall 98.83% vs 99.16%). The paper concludes that the system can reduce manual annotation effort while maintaining accuracy in industrial inspection.","tokens_in":4576,"tokens_out":3784,"duration_ms":42340,"significance":"If the claimed equivalence holds, this is a practically valuable contribution to reducing annotation cost in display manufacturing, an industrial domain where labeled data are scarce and expensive. The paper's strengths include a real industrial dataset, a two-stage training recipe that is plausible and reproducible with modest hardware, a comparison against the publicly available SegGPT baseline, and a downstream-task evaluation that goes beyond pixel-level metrics. The scribble-based prompting idea is interesting and could ease annotation further. However, the empirical support is currently under-developed: all metrics are point estimates without variance, the proposed components are not ablated, the scribble mechanism is not quantitatively evaluated, and the selection bias in the downstream comparison is unresolved. As presented, the central claim is plausible but not yet established to the standard expected for a journal publication.","major_comments":[{"comment":"The paper does not state whether the 'Auto Labelled Images' branch in Figure 6 consists of all images or only the subset that passed the coverage threshold (IoU>0.60, HD<10). Since Section 2 states that rejected labels are returned to the unlabeled pool and Figure 5 reports about 60% average coverage, it is likely that the auto-labeled training set used for Table 1 contains only the accepted easy subset. If so, Table 1 demonstrates equivalence only on that subset and does not establish that auto-labels are usable on the full data distribution, including the roughly 40% of images that would require re-prompting or manual labeling. Please clarify the composition of the auto-labeled training set and, if only the accepted subset was used, evaluate the downstream model on the full distribution or at least on the rejected images to quantify the gap.","section":"Section 3 (Experiments and Evaluation), Figure 6, Table 1"},{"comment":"All reported metrics are point estimates with no error bars, confidence intervals, or repeated-run variance. The downstream differences in Table 1 (IoU 0.85 vs 0.84, recall 98.83% vs 99.16%) are small and could easily arise from training stochasticity; without variance information the claimed equivalence is not statistically supported. Please report per-run results, standard deviations or confidence intervals, and per-product-type breakdowns for Table 1, since the abstract and text claim consistency across multiple product types.","section":"Section 3 (Experiments and Evaluation), Figures 4–5, Table 1"},{"comment":"The scribble token is a central novelty, but no experiment evaluates the scribble-to-mask path. The paper claims that scribble-based input 'substantially reduces the annotation burden while maintaining segmentation accuracy,' yet it never compares segmentation quality with scribble prompts versus full-mask prompts, nor does it report annotation time or user effort. An ablation comparing the same model with scribble and full-mask prompts, including IoU/recall differences and manual effort measurements, is needed to support this claim.","section":"Section 2 (Scribble-Based Annotation), Figure 3"},{"comment":"The proposed contributions—adaptive cropping/spatial ensembling, selective label suppression, and the scribble token—are only evaluated as a combined system. There is no ablation study that isolates any of these components. In particular, the claim that selective label suppression improves prompt faithfulness is qualitative and unsupported. Please provide an ablation that removes or toggles each proposed component and reports the resulting IoU, recall, and coverage on the same evaluation sets.","section":"Section 2 (Enhancing Defect Representation, Selective Label Suppression, Scribble-Based Annotation)"},{"comment":"The stage-2 curation 'filtered out extremely small defects that fell below our minimum pixel threshold,' but the threshold value is not reported, and the impact of this filtering on the evaluation is not analyzed. If the test set also contains such small defects, the reported coverage and IoU may be optimistic; if the test set excludes them, the claims do not transfer to raw inspection data. Please report the threshold, the fraction of defects removed, and the sensitivity of the results to this curation choice.","section":"Section 3 (Model Training)"}],"minor_comments":[{"comment":"The Hausdorff distance formula is typeset with garbled subscripts and superscripts; please rewrite it cleanly and define h(A,B) and H(A,B) precisely.","section":"Equation (1)"},{"comment":"The Introduction states that conclusions and future work appear in Section 5, but the Conclusion is Section 4 in the provided manuscript; fix the cross-reference.","section":"Introduction, Section 4"},{"comment":"The downstream 'SegModel' used for the Table 1 comparison is not specified. Please provide the architecture, training hyperparameters, and the number of training images used for both the human-labeled and auto-labeled branches.","section":"Section 3 (Experiments and Evaluation), Figure 6"},{"comment":"The reported average IoU increase of 0.22 and recall increase of 14% are stated in the abstract but not tied to a specific table or figure value; ensure the text clearly explains how these numbers are derived from Figure 4.","section":"Section 3 (Experiments and Evaluation), Figure 4"},{"comment":"The coverage rate definition (IoU>0.60 and HD<10) is introduced with no sensitivity analysis or justification for these thresholds. Please report how coverage varies with threshold choices, and note any resulting impact on the practical workflow.","section":"Section 3 (Experiments and Evaluation), Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is an industry case study with potential practical value, but the experimental reporting is not yet at journal standard. The selection-bias ambiguity in Table 1 is the most load-bearing issue: if the auto-labeled branch contains only the covered 60%, the central equivalence claim is not established on the full distribution. A clarification and a full-distribution or rejected-subset evaluation are needed. The lack of ablations and statistical variance also needs to be addressed. I would be willing to reconsider after revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a competent industry paper with real engineering value, but the central claim—auto-labels match human labels for downstream training—is under-supported as written. The most likely issue is that the comparison in Table 1 may only use the ~60% of images that passed the coverage threshold, not the full set. That needs to be clarified before the headline claim holds.\n\nWhat's new: the authors adapt SegGPT to display panel defect labeling with a few sensible domain tricks: composite multi-defect augmentation with grid layouts, selective label suppression to improve prompt adherence, and a learnable scribble token for scribble-based prompting. Two-stage fine-tuning (200k images then 10k curated) on proprietary LCD data is a reasonable recipe. The reported gains over the pre-trained SegGPT baseline—0.22 average IoU and 14% recall—are plausible and consistent with the approach.\n\nWhat's solid: the workflow is practical and the problem is real. The paper is honest about the 60% coverage rate and the need for human review. Using four RTX 3090s is also realistic for industrial settings.\n\nSoft spots: (1) The Table 1 equivalence is the key result, and the paper never says whether the auto-labeled training set includes all images or only the ones that passed the coverage threshold. Figure 1's workflow explicitly returns rejected labels to the unlabeled pool, so the natural reading is that the auto-labeled stream uses only the ~60% that passed. If so, Table 1 just says the model is good on the easy subset. The stress-test note is right to flag this. (2) No error bars, no repeated runs, no per-product breakdown. A 0.01 IoU gap and 0.33% recall gap are within noise, but we can't tell. (3) The scribble token is never quantitatively evaluated. We don't learn how much accuracy drops when prompts are scribbles rather than full masks. (4) No ablation isolates the proposed components (composite augmentation, label suppression, scribble token). The baseline comparison to SegGPT is fine, but the contributions are bundled.\n\nCitation pattern looks fine; SegGPT and Painter are properly credited, and the display-manufacturing references are appropriate.\n\nBottom line: this is a paper worth reviewing seriously, but it needs a revision that addresses the training-set composition question, adds error bars or repeated trials, and reports scribble and ablation results. If the Table 1 ambiguity resolves in the authors' favor, the practical claim holds. For now, I'd treat the equivalence result as unproven.","headline":"Useful applied paper on SegGPT for display defect auto-labeling, but the headline equivalence result may only hold on the easy 60% of images.","tokens_in":5083,"tokens_out":1814,"would_cite":false,"duration_ms":21556,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Auto-labelled defect masks from an in-context learning model train a display-inspection detector as well as human-annotated masks do.","keywords":["display panel inspection","defect detection","auto-labelling","deep learning","in-context learning","segmentation","scribble annotation","SegGPT"],"falsifier":"Inspect the Figure 6 workflow and re-run the downstream comparison with all auto-labelled images, including those with IoU at or below 0.60 or Hausdorff distance at or above 10 pixels; if including rejected images degrades downstream performance, the claim that auto-labelled data match human labels holds only for the accepted subset.","tokens_in":4172,"feed_emoji":"🏭","tokens_out":9279,"duration_ms":95011,"temperature":0.7,"pith_summary":"This paper tries to show that an in-context-learning segmentation model can take over a substantial share of manual labelling in display-panel defect inspection. The authors adapt the SegGPT architecture to panel-defect images, add a scribble-based prompt that lets an annotator mark a defect with a rough stroke instead of a precise mask, and train in two stages on 200,000 then 10,000 labelled circuit images. On three product types, the auto-labelling model beats the pre-trained baseline by an average of 0.22 IoU and 14% recall, and it produces acceptable masks for about 60% of unseen images. In a downstream comparison, a model trained on auto-generated labels performs about as well as one trained on human labels (IoU 0.84 vs 0.85; recall 99.16% vs 98.83%). If the result holds, manufacturers can cut annotation effort while keeping defect-detection accuracy.","feed_headline":"Auto-labels match human labels for display defect detection","feed_subtitle":"A scribble-based in-context model auto-labels about 60% of panel images, and trained detectors perform on par with human-annotated data.","key_machinery":"The load-bearing mechanism is an in-context-learning segmentation model built on SegGPT, which turns a reference image plus its mask and a query image into a mask-prediction task. Three modifications carry the argument: adaptive cropping and spatial ensembling place multiple zoomed defects into composite grids up to 4x4 so small defects remain learnable; selective label suppression randomly deactivates some defect-class labels during training so the model learns to segment only the defect class named in the prompt; and a learned scribble token lets a user prompt with a rough stroke, with paired scribble-to-mask training expanding the stroke into a full mask. The model is trained in two stages, first on 200,000 labelled LCD circuit images and then on 10,000 curated high-quality images during which the suppression and scribble mechanisms are introduced.","core_discovery":"The central claim is that in-context learning, in the form of an enhanced SegGPT model, can label industrial display-panel defects accurately enough to train downstream detectors as well as human annotation does. The model takes a reference image with its mask, or a scribble, as a prompt and produces a segmentation mask for a new query image. The paper reports that this auto-labelling pipeline reaches about 60% coverage—defined as predicted masks with IoU above 0.60 and Hausdorff distance below 10 pixels—across three LCD panel types, and that a detector trained on the auto-labelled data matches one trained on human labels (IoU 0.84 versus 0.85, recall 99.16% versus 98.83%). The authors present this as evidence that AI-assisted labelling is a practical way to reduce manual annotation in industrial inspection rather than a full replacement for human review.","pith_inferences":["A testable extension would be to measure end-to-end annotator time: if reviewing an AI-generated mask is faster than drawing a full mask, the roughly 40% of images below the coverage threshold still yield savings even before re-prompting.","The quality threshold (IoU above 0.60 and Hausdorff distance below 10 pixels) could be used as an explicit acquisition rule: accept auto-labels above threshold, re-prompt below it, and grow the training set iteratively.","The same scribble-token and label-suppression mechanisms could transfer to other defect-imaging domains with small, elongated, or low-contrast flaws, such as semiconductor wafers or metal surfaces, where full-mask annotation is equally costly.","A useful stress test would be to measure coverage on a product type excluded from training; the paper reports results across three product categories but does not state whether any of them were held out during fine-tuning."],"forward_implications":["With an average coverage rate near 60%, the auto-labelling model can pre-label more than half of unseen display-panel images, leaving human annotators to review masks rather than draw them from scratch.","A detector trained on auto-labelled data reaches IoU 0.84 and recall 99.16%, essentially matching one trained on human labels (IoU 0.85, recall 98.83%).","The reported gains of +0.22 IoU and +14% recall over the pre-trained baseline indicate that domain-specific training techniques, not the base model alone, make in-context labelling usable on industrial defect data.","The scribble-based prompting mechanism lowers the skill and time needed per annotation because a rough stroke is expanded into a complete segmentation mask.","The two-stage training pipeline runs on four consumer-grade RTX 3090 GPUs, so the approach does not depend on specialized industrial hardware."],"supporting_citations":[{"why":"Provides the SegGPT architecture that the paper adopts and enhances for in-context defect segmentation.","marker":"[7]"},{"why":"Establishes the in-context learning formulation by treating visual tasks as image inpainting, the basis for the prompting approach.","marker":"[6]"},{"why":"Supplies the masked auto-encoding objective used to train the model to reconstruct occluded segmentation regions.","marker":"[8]"},{"why":"Inspires the scribble-based prompting mechanism, which the paper turns into a specialized token with paired scribble-to-mask training.","marker":"[9]"},{"why":"Provides the Hausdorff distance metric used in the coverage quality criterion for accepted auto-labels.","marker":"[10]"}],"fun_headline_variants":["Scribble-based AI labeling matches human annotations for panel defects","In-context learning auto-labels display defects as well as humans","60% auto-label coverage with human-level defect detection","AI auto-labeling for display defects meets human-level accuracy","In-context model labels 60% of panel images, matches manual labels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The downstream equivalence result assumes that the auto-labelled training data were not limited to the roughly 60% of images whose masks passed the quality threshold; if only accepted images were used, the match with human labels does not show that rejected images can be labelled automatically.","fun_headline_variants_meta":{"raw":{"variants":["Scribble-based AI labeling matches human annotations for panel defects","In-context learning auto-labels display defects as well as humans","60% auto-label coverage with human-level defect detection","AI auto-labeling for display defects meets human-level accuracy","In-context model labels 60% of panel images, matches manual labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000165,"raw_usage":{"total_tokens":1201,"prompt_tokens":850,"completion_tokens":351,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":466,"completion_tokens_details":{"reasoning_tokens":264}},"tokens_in":466,"tokens_out":351,"duration_ms":4987,"temperature":1.0,"reasoning_tokens":264,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:34:52.779688+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the Figure 6 workflow and re-run the downstream comparison with all auto-labelled images, including those with IoU at or below 0.60 or Hausdorff distance at or above 10 pixels; if including rejected images degrades downstream performance, the claim that auto-labelled data match human labels holds only for the accepted subset.","supporting_citations":[{"cited_title":"Seggpt: Towards segmenting everything in context","cited_arxiv_id":null,"evidence_quote":"Provides the SegGPT architecture that the paper adopts and enhances for in-context defect segmentation."},{"cited_title":"Images speak in images: A generalist painter for in-context visual learning","cited_arxiv_id":null,"evidence_quote":"Establishes the in-context learning formulation by treating visual tasks as image inpainting, the basis for the prompting approach."},{"cited_title":"Masked autoencoders are scalable vision learners","cited_arxiv_id":null,"evidence_quote":"Supplies the masked auto-encoding objective used to train the model to reconstruct occluded segmentation regions."},{"cited_title":"Segment everything everywhere all at once","cited_arxiv_id":null,"evidence_quote":"Inspires the scribble-based prompting mechanism, which the paper turns into a specialized token with paired scribble-to-mask training."},{"cited_title":"Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool","cited_arxiv_id":null,"evidence_quote":"Provides the Hausdorff distance metric used in the coverage quality criterion for accepted auto-labels."}],"review_version":1}