{"id":"91c01036-5f22-49ec-8e75-e1c7651784aa","arxiv_id":"2411.17217","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A self-drafting and relation-aware fine-tuning scheme lifts SAM's anomaly segmentation mIoU by 1.5 to 2.0 points over PEFT baselines across six industrial datasets.","lead":"The paper introduces a fine-tuning method for SAM, a general image segmentation model, so that it can outline defects in industrial product photos. The method, Self-Perception Tuning, first has the model generate a rough anomaly mask and then uses that mask as an extra prompt to refine the final segmentation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Decisive unverified premise: a private, unreleased ~15k-image training set is both representative of and disjoint from the six public test benchmarks; if false, the reported 15% and 2% gains do not establish SPT's effectiveness.","rationale":"I read the paper as a method contribution whose evidence is internally consistent: the method is described clearly, ablations support both SDT and VRA-Adapter, gains over PEFT baselines are consistent across ViT-B/L/H backbones (Tables 9-11), and code is released. Those are real strengths. However, the central claim is empirical and the training data is the single unverifiable input. Without the private dataset, no independent group can reproduce the exact numbers, check for train/test distribution overlap, or test sensitivity to imaging conditions. The concern is not that the authors are dishonest; it is that the paper's main evidence cannot be audited. The proposed leave-one-dataset-out transfer experiment is a feasible way to test whether the SPT mechanism itself transfers, independent of the proprietary data. If the advantage persists in public-to-public transfer, the private-data concern is substantially mitigated; if not, the reported improvements are attributable to the private training distribution rather than to SPT. This is consistent with the reader's conditional verdict: the paper should be accepted only if this dependence is resolved or clearly documented, for example by releasing the data or providing a public-data transfer evaluation.","tokens_in":68,"tokens_out":9417,"duration_ms":104097,"concrete_test":"Using the released code, run a leave-one-dataset-out transfer check: train SPTLoRA and LoRA (with the same hyperparameters as Sec. 4.1) on pixel-level anomaly masks from five public datasets and evaluate on the held-out sixth dataset, repeating for all six choices. If SPT's average margin over LoRA/DoRA is not reproduced, or reverses, compared with the paper's private-data result, the reported gains depend on the proprietary training distribution; if the margin persists across held-out datasets, the concern is mitigated. Report per-dataset means and standard deviations over at least three seeds so the word 'significantly' can be checked.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim—'over 15% improvement' and 'SPTDoRA surpasses DoRA by 2.0/2.2' (Sec. 4.2)—is established only on a private training set of ~15,000 real-factory images (Sec. 4.1, Appx. A.1) that is not released and whose images and annotations cannot be audited. The paper asserts that its categories are 'distinct from those in the test datasets,' but this is not demonstrated; the listed private categories (3C, display screens, photovoltaics, lithium batteries) plausibly overlap in appearance with public test categories (e.g., PCB/transistor/capsules in MVTec and VisA, metal parts in BTAD/MPDD). If the private set contains defects visually similar to the test sets, the gap over DoRA could reflect train/test distribution overlap rather than the SPT draft-refine and relation-aware mechanisms. Moreover, Table 1 reports only dataset-averaged numbers, so a reader cannot see whether the margin is positive on each of the six datasets, further weakening the 'across six benchmark datasets' wording. The representativeness and disjointness of the private data is therefore the load-bearing assumption for the paper's headline result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Self-Perception Tuning (SPT), a PEFT-based adaptation of SAM for promptable anomaly segmentation. SPT adds a draft decoder that first produces a coarse anomaly mask; this draft is encoded and fed as dense embedding into a refine decoder. A Visual-Relation-Aware Adapter (VRA-Adapter) enhances the decoder image features using a relation matrix computed from cosine similarities. The method is evaluated on six industrial benchmarks with box and point prompts, claiming over 15% average improvement in mIoU/mBIoU over zero-shot SAM and 2.0/2.2 point gains over DoRA, with small trainable-parameter overhead. Code is released, but the training set is private.","tokens_in":16834,"tokens_out":4804,"duration_ms":42928,"significance":"If the claims hold, SPT provides a simple, parameter-efficient recipe for adapting a segmentation foundation model to industrial defect segmentation, with the draft-then-refine self-prompting idea being reusable beyond this specific task. The paper's strengths include clean ablations isolating the SDT and VRA components, cost/efficiency analysis, experiments across three SAM backbones, and public code. The significance is moderate: the reported gains are consistent and the method is practical, but the lack of a public training set and the aggregate-only reporting substantially limit independent verification.","major_comments":[{"comment":"The load-bearing premise is that the private, unreleased ~15,000-image training set is representative of and disjoint from the six public test benchmarks. The text only asserts that the training categories are 'distinct' without demonstrating appearance-level disjointness; the listed private categories (3C, photovoltaics, display screens, lithium batteries) plausibly share visual structure with test categories such as PCBs, transistors, capsules, and metal parts. Because no images or annotations are released, independent readers cannot verify whether the reported 15% and 2% gains reflect the SPT mechanism or train/test overlap. Please release the data or a representative subset, or provide a per-category/per-dataset transfer analysis quantifying the train-test distribution gap.","section":"§4.1, Appx. A.1"},{"comment":"The headline claim that SPT improves anomaly segmentation 'across six benchmark datasets' is supported only by dataset-averaged scores. With six datasets and four prompt modes, a reader cannot tell whether SPT wins on each dataset or whether the average is driven by a few favorable cases. Report per-dataset mIoU/mBIoU for all methods (at least for the main Table 1 configurations), and include run-to-run variance or significance tests; the 2.0% mIoU gain over DoRA is small enough that it could be within noise.","section":"§4.2, Table 1"},{"comment":"There is an internal data inconsistency: the 'Ours' row in Table 8 reports Point=5 70.1/64.5 and Point=10 64.6/59.8, which are the reverse of the Point=5/Point=10 values for SPTLoRA in Table 1 and for 'Ours' in Table 9. This must be corrected. Additionally, it is not stated which SPT variant (SPTLoRA, SPTDoRA, SPTAdapter) is used in Tables 8–11 where the label 'Ours' appears; specify this for each table.","section":"Appx. C.4, Table 8"}],"minor_comments":[{"comment":"There is a typo in the abstract: 'Self-Perceptinon' should be 'Self-Perception', and the spacing in '( SPT)' should be fixed.","section":"Abstract"},{"comment":"The sensitivity-analysis figure appears garbled by rendering tokens; ensure the figure is legible in the camera-ready version.","section":"Fig. 7, Appx. C.3"},{"comment":"The threshold α/d is not motivated; clarify whether d is the feature dimension and report the selected α values per backbone/PEFT method in a small table, including how α was chosen (validation set vs. test benchmarks).","section":"§3.3, Eq. (9)"},{"comment":"The training description says 'a batch of 8 images' with 8 GPUs; please clarify whether this means 8 total images or 8 per GPU, and report the number of training iterations or effective batch size.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The main gate for this paper is reproducibility: the training set is private, and the headline claims are based on aggregate numbers with no per-dataset or variance analysis. I recommend asking the authors to either release the training data (or a representative subset) or provide a strong per-category and per-dataset transfer analysis, together with error bars, before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuine, well-ablated engineering contribution, and the main thing to watch is the private training set and the way results are reported (averages only, no error bars). The stress-test's worry about train/test overlap is reasonable, but it is a risk to be examined, not a demonstrated flaw.\n\nThe new idea is clean: instead of just PEFT-tuning SAM, you add a draft decoder that produces a coarse mask; that mask is then embedded by the prompt encoder and fed as the dense embedding to the refine decoder. It's a kind of self-feedback, and the authors show it helps even without any fine-tuning (the zero-shot SDT row improves mIoU by ~1 point). The VRA-Adapter, which aggregates image features with a learned relation matrix, is simpler still, but the ablations show each piece adds something; using it in both transformer layers works best. Gains are consistent across LoRA, DoRA, and Adapter backends and across ViT-B/L/H (the latter two in the appendix). Code is released, which makes the method reproducible in principle.\n\nThe soft spots are real but in proportion. First, the training set is ~15k industrial images from real factories, and it is not released. The paper says the categories are distinct from the test sets, but it does not demonstrate that. The stress-test is right that 3C/display/photovoltaic images could look a lot like some MVTec/VisA/BTAD categories. If the private set is close to the test distribution, a chunk of the reported gain over DoRA may be an artifact of that overlap. Second, the headline 'over 15% improvement' is vs zero-shot SAM; the more relevant comparison, +2.0 mIoU over DoRA, is modest, and there are no error bars, significance tests, or per-dataset numbers in the main table to show the margin is positive everywhere. The appendix has more baselines but still only averaged numbers.\n\nI wouldn't call this circular; the draft mask is a model output used as an internal feature, which is a feedback mechanism, not fitting on the test labels. The method is a reasonable adaptation of SAM, not a new capability.\n\nBottom line: this deserves a serious referee. The conditions should be (a) release the training data or at least provide per-dataset breakdowns and some distribution analysis, and (b) add variance estimates. I'd cite it, and I'd bring it to reading group as a good example of self-refinement.","headline":"Solid, honest engineering contribution; the private training set and missing variance/per-dataset numbers are the real soft spots.","tokens_in":17434,"tokens_out":2178,"would_cite":true,"duration_ms":20596,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Tuning SAM to draft and then refine its own coarse anomaly mask, with a visual-relation-aware adapter, is reported to improve promptable anomaly segmentation by more than 15 percentage points on average across six industrial datasets.","keywords":["anomaly segmentation","Segment Anything Model","promptable segmentation","parameter-efficient fine-tuning","self-draft tuning","visual-relation-aware adapter","industrial defect detection","domain shift"],"falsifier":"Train SPT on a publicly available substitute training set, such as the training images from one of the six benchmark datasets, and test on the other five; if the average mIoU gain over zero-shot SAM falls far below the reported 15-plus points, the private training distribution rather than the method itself is what carries the result.","tokens_in":16403,"feed_emoji":"🏭","tokens_out":12069,"duration_ms":99619,"temperature":0.7,"pith_summary":"Anomaly segmentation means locating defects in industrial images, and the paper asks whether SAM, a general-purpose segmenter trained on natural images, can be adapted to this out-of-domain task cheaply and robustly. The proposed Self-Perception Tuning (SPT) makes SAM first generate a coarse anomaly mask with a second decoder and then refine that draft, while a Visual-Relation-Aware Adapter adds inter-region similarity information into the decoding. Across six public industrial datasets and four prompt types, the three SPT variants are reported to beat zero-shot SAM by more than 15 percentage points in average overlap (mIoU) and boundary-overlap (mBIoU), and to beat the underlying PEFT baselines by about 2 points, using only a few tenths of a percent of trainable parameters. A reader should care because a single lightly tuned SAM would then segment defects on many production lines without per-dataset retraining and without giving up prompting by boxes or points.","feed_headline":"SAM gains 15 points by drafting and refining its own defect masks","feed_subtitle":"A self-draft loop and relation-aware adapter beat efficient fine-tuning methods on six industrial defect benchmarks.","key_machinery":"The load-bearing object is the draft-then-refine loop of Self-Draft Tuning (SDT), organized into display, draft, and refine stages: the image and prompt encoders extract features, a draft decoder initialized from the original mask decoder generates a coarse anomaly mask, and the original mask decoder refines it using the draft's embedding as additional dense knowledge. The second component is the Visual-Relation-Aware Adapter (VRA-Adapter), which builds a relation matrix from cosine similarities between image-region embeddings, suppresses weak relations by a threshold $\\alpha/d$, and adds relation-aggregated features back into the decoder with a learned scale $\\beta$. These two mechanisms are what the paper claims let a lightly tuned SAM perceive industrial anomalies rather than merely interpolating prompt points.","core_discovery":"The central claim is that SAM's poor anomaly segmentation on industrial images is a perception problem, not merely a prompt problem, and that it can be fixed by letting the model perceive anomalies through its own coarse draft and through explicit visual-relation cues. SPT augments SAM with a draft decoder that produces an initial anomaly mask; that mask is encoded and fed back to the original mask decoder as a dense embedding, and a Visual-Relation-Aware Adapter computes a cosine-similarity relation matrix between image regions, thresholds weak relations, and aggregates related features during decoding. Fine-tuned with a PEFT method while nearly all SAM weights stay frozen, the model is reported to outperform both zero-shot SAM and the PEFT baselines on MVTec, VisA, MTD, KSDD2, BTAD, and MPDD, with the largest relative gains on sparse point prompts. The paper frames SPT as a flexible tuning framework: any PEFT method can be wrapped by SPT and inherit the draft-then-refine gains.","pith_inferences":["Editorial inference: if the private 15,000-image factory set is what carries the gains, then releasing a comparable public industrial anomaly corpus could turn SPT into a general pre-training recipe; the paper itself does not release the training set.","Editorial inference: the draft-then-refine idea is not SAM-specific, since any promptable segmenter with a mask decoder could feed its own draft back as a conditioning input, so the mechanism may transfer to other out-of-domain segmentation tasks.","Editorial inference: the VRA-Adapter's threshold $\\alpha/d$ assumes anomalies have coherent regional structure, so extremely fragmented or heavily textured defects may need re-tuning; the paper does not test that regime.","Editorial inference: the point-prompt results suggest a stress test the paper does not run, namely perturbing prompt points and measuring how much mIoU drops, which would quantify how much internal perception compensates for noisy prompts."],"forward_implications":["A single SPT-tuned SAM can be prompted by boxes or points and segment defects across different product lines, including electronics, photovoltaics, magnetic tiles, and metal parts, without per-dataset retraining.","The reported point-prompt gains imply that the model does not need the prompt to trace the full defect: its internal draft and relation cues complete regions the prompt only partially covers.","Because SPT wraps existing PEFT methods, new PEFT designs can be plugged into the same self-draft loop and likely inherit the same relative lift.","The added cost is small: on the smallest backbone the full SPT model trains roughly 0.4% of the parameters and keeps throughput close to the untuned model, so the gain is not bought with a large fine-tuning budget.","Replacing the untuned SAM inside an existing automatic-prompt pipeline with SPT-tuned SAM also raises detection metrics, so the improvement survives when prompts come from a detector instead of a human."],"supporting_citations":[{"why":"Provides the SAM model and its zero-shot behavior that SPT adapts, plus the SA-1B pretraining background.","marker":"Kirillov et al. 2023"},{"why":"Supplies the LoRA low-rank adaptation baseline and the PEFT mechanism used inside one SPT variant.","marker":"Hu et al. 2021"},{"why":"Supplies the DoRA baseline that SPTDoRA is compared against and whose weight-decomposed adaptation SPT wraps.","marker":"Liu et al. 2024"},{"why":"Supplies the Adapter baseline and the adapter-module design that the SPTAdapter variant builds on.","marker":"Houlsby et al. 2019"},{"why":"Provides the MVTec AD benchmark, one of the six test sets on which the gains are measured.","marker":"Bergmann et al. 2019"},{"why":"Provides the VisA benchmark, one of the six test sets.","marker":"Zou et al. 2022"},{"why":"Provides the MTD magnetic-tile defect benchmark, one of the six test sets.","marker":"Huang, Qiu, and Yuan 2020"},{"why":"Provides the KSDD2 surface-defect benchmark, one of the six test sets.","marker":"Božič, Tabernik, and Skočaj 2021"},{"why":"Provides the BTAD benchmark, one of the six test sets.","marker":"Mishra et al. 2021"},{"why":"Provides the MPDD metal-parts benchmark, one of the six test sets.","marker":"Jezek et al. 2021"}],"fun_headline_variants":["Self-draft masks boost SAM anomaly segmentation","SAM's self-perception tuning beats fine-tuning on defects","Draft-and-refine: SAM anomaly segmentation boost","Visual-relation adapter lifts SAM defect masks","SPT: SAM perceives anomalies via self-drafts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the private, unreleased training set of about 15,000 real-factory defect images is representative enough of the six public test benchmarks that the reported cross-dataset gains transfer; if that match fails, the measured improvements would not reproduce for other industrial imaging conditions.","fun_headline_variants_meta":{"raw":{"variants":["Self-draft masks boost SAM anomaly segmentation","SAM's self-perception tuning beats fine-tuning on defects","Draft-and-refine: SAM anomaly segmentation boost","Visual-relation adapter lifts SAM defect masks","SPT: SAM perceives anomalies via self-drafts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000564,"raw_usage":{"total_tokens":2666,"prompt_tokens":929,"completion_tokens":1737,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":545,"completion_tokens_details":{"reasoning_tokens":1664}},"tokens_in":545,"tokens_out":1737,"duration_ms":12283,"temperature":1.0,"reasoning_tokens":1664,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:21:21.562024+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train SPT on a publicly available substitute training set, such as the training images from one of the six benchmark datasets, and test on the other five; if the average mIoU gain over zero-shot SAM falls far below the reported 15-plus points, the private training distribution rather than the method itself is what carries the result.","supporting_citations":[],"review_version":1}