{"id":"4da83467-c41e-4633-aa3e-53d96af4bbf4","arxiv_id":"2507.11003","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"FiSeCLIP achieves state-of-the-art zero-shot anomaly detection by using a batch of test images as mutual references and filtering noisy features with text-guided masks, without any training.","lead":"The paper introduces FiSeCLIP, a training-free method for zero-shot anomaly detection that uses a small batch of test images as mutual references to spot defects without any training data. It combines CLIP text-image alignment with batch-based feature matching and a mutual-filtering step, reporting new state-of-the-art results on industrial anomaly detection benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The method's central mutual-filtering mechanism is only evaluated on low-anomaly benchmark batches; its behavior when reference images are mostly or entirely defective is never tested, so the SOTA claim depends on an unstated prevalence assumption.","rationale":"The reader's weakest assumption is the right one. The paper's main novelty is filtering noisy features in the reference, and the failure mode where references are anomalous is exactly what the filter claims to address, yet no experiment isolates it. I do not think this warrants rejecting the paper, because the benchmark results may still be valid under the stated low-prevalence conditions, and the ablation in Tables 3-4 suggests hyperparameters are not the decisive factor. However, the claim that batch-based testing is practical and that mutual filtering enables it requires a controlled test. I agree with the reader's conditional verdict; the conditions should include the batch-composition experiment and a clear definition of how test batches are constructed (e.g., pair sampling), plus multi-run statistics, since the current protocol does not specify whether each image is evaluated with one fixed partner or over all possible pairings.","tokens_in":13203,"tokens_out":5848,"duration_ms":75721,"concrete_test":"Run the released FiSeCLIP code on a fixed set of test images (e.g., MVTec-AD category 'bottle') and construct pairs with controlled reference composition: 0% defective, 50% defective, and 100% defective references, while keeping the query set fixed. Recompute segmentation AU-ROC/F1-max and report the fraction of pairs for which Eq. 11's filtered reference F_hat_v is empty or has fewer than, say, 5% of patches. If AU-ROC drops by more than a few points as reference defect prevalence increases, or if empty-reference pairs occur, the mutual-filtering mechanism does not support the claimed robustness and the SOTA claim must be scoped to low-prevalence batches.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that FiSeCLIP achieves SOTA zero-shot AD by using unlabeled batch images as references—rests on the assumption that the text-guided mask (Eq. 9) and the mutual-filtering loop (Alg. 1) leave enough true-normal features in the reference for Eq. 10-11 to compute meaningful min-distances. This assumption is load-bearing but never stress-tested. MVTec, VisA, and BTAD test sets contain mostly normal images with small, localized defects, so even an imperfect mask will usually produce a reference with abundant normal patches. In a deployed batch with high defect prevalence—or two images of the same defect class—unfiltered anomalous features in the reference can match anomalous patches in the query, directly suppressing the anomaly score; if the thresholding in Eq. 9 removes all reference features, the min over an empty set in Eq. 11 is undefined. The paper reports no experiment varying the anomaly ratio of the reference batch, no degenerate-case handling, and no error bars over random batch compositions. Since the entire contribution is the filtering mechanism, this is not a peripheral deployment concern: the mutual-filtering claim is unverified precisely in the regime it was designed for.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FiSeCLIP, a training-free zero-shot anomaly detection (ZSAD) method that uses a batch of test images as mutual references. It combines feature matching (FiCLIP-AD) with text-image alignment (SeCLIP-AD): text-guided masks filter out anomalous features from the reference images before computing patchwise minimum distances, and the resulting anomaly scores are iteratively fed back to refine the masks via mutual voting. The method also modifies the final CLIP attention layer by substituting intermediate-layer attention and removing the feed-forward network to improve local semantic coherence. Experiments on MVTec AD, VisA, and BTAD report state-of-the-art results, e.g., 93.3 AU-ROC and 49.1 F1-max for anomaly segmentation on MVTec AD, outperforming prior ZSAD methods including MuSc-2 and AdaCLIP.","tokens_in":13473,"tokens_out":5718,"duration_ms":66401,"significance":"If the claims are reproducible and the method generalizes beyond the specific evaluation protocol, FiSeCLIP is a valuable contribution to ZSAD: it is training-free, aligns with batch-based industrial deployment, and the mutual-filtering idea is a plausible way to suppress noisy references. The paper ships code, uses standard benchmarks, and includes ablations on the filtering mechanism, hyperparameters, and attention modifications. However, the reported superiority currently rests on an incompletely specified evaluation protocol: the batch size is never stated, hyperparameters are tuned on the test set, and the behavior under high anomaly prevalence is untested. These issues must be resolved before the SOTA claim can be taken at face value.","major_comments":[{"comment":"The batch size B is never specified in the method or the implementation details. Algorithm 1 and Eq. (12) are written explicitly for two images (I_u and I_v), and the discussion of references uses v != u without explaining how multiple references are aggregated. If all experiments use B=2, this should be stated, and the comparison to MuSc-2 (which also uses two images) is then fair. If B>2 is used or intended, the mutual-filtering extension to multiple references is undefined, and the reported improvements over MuSc-2 would need to be disentangled from the added reference images. This is load-bearing for the central benchmark claim.","section":"Sec. 3.1, 4.1, Algorithm 1"},{"comment":"The hyperparameters λ in Eq. (9) and μ in Eq. (13), as well as the attention layer index (Table 5), are selected by scanning values on the MVTec AD test set, with no validation split or fixed defaults used for the main results. This constitutes test-set adaptation, which is in tension with the 'zero-shot' and 'training-free' framing. The authors should report results for the default values without test-set tuning, or perform hyperparameter selection on a separate validation split (e.g., a subset of categories) and then report test performance.","section":"Sec. 4.3, Tables 3-5"},{"comment":"The anomaly score in Eq. (11) is the minimum distance over the filtered reference features. If the text-guided mask (Eq. 9) removes all reference patches, the minimum over an empty set is undefined, and the paper provides no fallback for this degenerate case. Because the entire contribution is the mutual-filtering mechanism, the method's behavior when reference batches are mostly or entirely anomalous should be tested. The benchmark datasets have low anomaly proportions (roughly 11-27%), so this regime is never exercised. Provide experiments varying the anomaly ratio of the reference batch (e.g., 0%, 50%, 100% anomalous references) and describe how empty references are handled.","section":"Sec. 3.3.2, Eq. (11)"},{"comment":"The paper does not report how test batches are constructed in the experiments (e.g., random sampling, fixed order, or category-wise grouping), nor does it report any variance across batch compositions. Since the method's output depends on which images are in the batch, the reported single-number results may be sensitive to the specific batch composition. Please provide error bars (mean and standard deviation) over multiple random batch compositions, or at minimum describe the batch construction in detail so the experiments can be reproduced.","section":"Sec. 4.1, Evaluation"},{"comment":"The sentence 'The results indicate that the best performance is achieved when λ is and µ is 2' is incomplete: the optimal value for λ is missing, and Table 4 lists μ values from 0.47 to 0.57, not 2. This needs to be corrected to an explicit statement such as 'λ = 1.10 and μ = 0.57' if that is what the tables show.","section":"Sec. 4.3, Hyperparameter ablation"}],"minor_comments":[{"comment":"There is a typo in the introduction: 'vlaue' should be 'value'.","section":"Sec. 1"},{"comment":"The text says 'Average Prevision'; this should be 'Average Precision'.","section":"Sec. 4.1"},{"comment":"The phrase 'SOTA AdaCLIP' is ambiguous because AdaCLIP is a fine-tuned method. Specify that it is the previous state-of-the-art among ZSAD methods, or say 'fine-tuned AdaCLIP'.","section":"Abstract and Sec. 1"},{"comment":"The figure caption annotates 'FiCLIP-AD (Sec. 3.2)' and 'SeCLIP-AD (Sec. 3.1)', but in the text SeCLIP-AD is Section 3.2 and FiCLIP-AD is Section 3.3. These references should be corrected.","section":"Figure 2"},{"comment":"The input 'initial anomaly score ˆa' is not defined in the text around Eqs. (12)-(13). Clarify whether this is the score from Eq. (7) or from a previous iteration of the loop.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern raised by the reader is well-founded: the mutual-filtering loop is never tested under high anomaly prevalence, and the absence of any batch-size specification makes the comparison to MuSc-2 difficult to interpret. I also note that the hyperparameter tuning on the MVTec test set is a serious issue for a 'zero-shot' method; the authors should be asked to clarify this. The paper's core idea is worth publishing if these points are addressed, but as it stands the SOTA claim is not fully supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nRead FiSeCLIP. The core idea is real: use a text-image alignment mask to filter reference features before feature matching, then refine the mask with the matching scores, iterating within a small test batch. That mutual-filtering loop is new, and it works. On MVTec it beats the training-free MuSc-2 by 5.6 points AU-ROC on classification and 0.7 on segmentation; on VisA the segmentation gain is 1.8 points. The method is training-free and the code is out. I'd call this an honest empirical contribution.\n\nThe paper is also careful in some ways. The ablations on filtering variants, hyperparameters, and attention layers give a real sense of what matters. The attention-layer replacement (borrowing from ClearCLIP but only on the last layer) is sensible, and they show intermediate-layer attention is best.\n\nSoft spots, in order of importance.\n\nFirst, the batch size is never stated in the experiments. The method is described for pairs, but the algorithm uses B images. If they run with B=2 and compare to MuSc-2, fine. If they use larger batches, the comparison is unfair and the claims need adjusting. This must be clarified.\n\nSecond, the stress-test concern is legitimate: the mutual-filtering assumes the reference batch contains enough normal patches. On MVTec/VisA/BTAD the anomaly ratio is low, so it works. But the paper explicitly says batch-based testing 'better aligns with real industrial needs.' In a real production batch with high defect prevalence, the filter could remove all normal features or leave anomalous ones that match the query, and scores collapse. They never test this. It's not a fatal flaw for the benchmark result, but it is a caveat on the framing.\n\nThird, lambda and mu are tuned on the test set. The ablations show the response is fairly flat, so the damage is limited, but error bars over random batch compositions would help; the method is stochastic in how the batch is drawn.\n\nFourth, the abstract cherry-picks AdaCLIP for the headline gain. AdaCLIP is fine-tuned, not the training-free SOTA; MuSc-2 is, and the gain there is much smaller on segmentation. The main text gets this right, but the abstract oversells.\n\nAlso a copyedit: Section 4.3 has a garbled sentence about lambda and mu ('lambda is and mu is 2'). Minor.\n\nWho is this for? Anyone working on zero-shot anomaly detection or CLIP-based dense prediction. The mutual-filtering idea is worth citing. It deserves a serious referee. I'd accept with minor revisions: specify the batch size, add a sensitivity analysis for anomaly prevalence, move hyperparameter selection off the test set, fix the abstract overclaim.","headline":"A genuinely new mutual-filtering mechanism for training-free zero-shot AD, with solid benchmark gains, but the paper overclaims practicality without testing high-anomaly batches or reporting batch size.","tokens_in":13975,"tokens_out":3755,"would_cite":true,"duration_ms":44112,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a training-free CLIP pipeline, which uses the other images in the same test batch as filtered mutual references, achieves state-of-the-art zero-shot anomaly detection, reporting 93.3 AU-ROC for segmentation on…","keywords":["zero-shot anomaly detection","CLIP","feature matching","cross-modal alignment","mutual filtering","batch inference","anomaly segmentation","vision-language models"],"falsifier":"Batch the MVTec-AD test set into groups whose anomaly fraction is varied from 0% to 100% (for example, all-normal batches, natural batches, then batches made only of 'damaged' class images) and run the released FiSeCLIP code; if anomaly AU-ROC and F1-max fall monotonically and collapse near the all-defect end, the assumption that a batch always provides normal reference patches is falsified.","tokens_in":1805,"feed_emoji":"🔍","tokens_out":4677,"duration_ms":131377,"temperature":0.7,"pith_summary":"The paper aims to establish that zero-shot anomaly detection does not require fine-tuning, full-dataset priors, or hand-crafted prompts to be competitive: a frozen CLIP can detect and locate defects by using the other images in the same test batch as mutual references. Because those references are unlabeled and may themselves contain defects, the paper introduces a mutual-filtering loop that uses text-to-image alignment to mask out anomalous patches before feature matching, then feeds the matching scores back to refine the mask. On MVTec-AD this training-free recipe reports 93.3 AU-ROC and 49.1 F1-max for segmentation and 95.3 AU-ROC and 95.4 F1-max for classification, beating the previous training-free method MuSc-2 and the prompt-tuned AdaCLIP. If true, this matters because batch-based, training-free inference is closer to how industrial inspection actually runs, and it sidesteps the data-leakage risk of methods that adapt CLIP on auxiliary data.","feed_headline":"Batch mutual filtering lifts zero-shot defect detection to 93.3 AU-ROC","feed_subtitle":"Training-free FiSeCLIP uses other batch images as filtered references, beating tuned zero-shot baselines.","key_machinery":"The central mechanism is the mutual-filtering loop between two CLIP-based heads. SeCLIP-AD computes per-patch anomaly probabilities by cosine similarity between CLIP visual features and averaged normal/abnormal text features; to restore local semantic correlation it replaces the final layer's q–k attention with the intermediate layer's attention weights and drops the residual connection and feed-forward network. FiCLIP-AD extracts patch tokens from layers 6, 12, 18, and 24, aggregates them over neighbourhoods $r = 1, 3, 5$, filters the reference features with the mask ($\\hat{F}_v = F_v[\\bar{M}]$), and scores each query patch by the minimum $\\ell^2$ distance to the filtered reference set. The per-layer, per-scale scores are averaged into an intermediate mask that votes with the text-guided mask, and the loop repeats, so the same machinery both cleans the references and sharpens the anomaly map.","core_discovery":"FiSeCLIP's central claim is that zero-shot anomaly detection can be performed with a completely frozen CLIP by treating every test image in a batch as a reference for the others, provided the anomalous patches in those references are filtered out first. The filter is a text-guided mask: a patch is removed from the reference pool when its abnormal probability exceeds $\\lambda$ times its normal probability, where those probabilities come from CLIP's similarity between visual patch features and averaged 'normal'/'abnormal' text features. After filtering, the score for each query patch is the minimum Euclidean distance to the remaining reference patches, and this score is then fed back to refine the mask in a mutual-filtering loop. The paper reports that this recipe reaches 93.3 AU-ROC and 49.1 F1-max for segmentation and 95.3 AU-ROC and 95.4 F1-max for classification on MVTec-AD, surpassing the previous training-free method MuSc-2 and the prompt-tuned AdaCLIP.","pith_inferences":["Editorial inference: on batches that are mostly or entirely defective, the mutual-filtering reference pool would collapse, so the method's reported gains likely depend on the low anomaly density of the benchmarks; this regime is not tested in the paper.","Editorial inference: because the text mask is prompt-driven, richer or class-specific prompts beyond plain 'normal' and 'abnormal' could plausibly improve filtering, and the paper only evaluates one fixed template.","Editorial inference: the batch requirement could be turned into a feature by deliberately injecting a known-good anchor image into every batch; the paper does not explore this, but the mechanism suggests it would stabilise the reference pool."],"forward_implications":["On MVTec-AD, FiSeCLIP reports 93.3 AU-ROC and 49.1 F1-max for segmentation and 95.3 AU-ROC and 95.4 F1-max for classification, exceeding the best training-free baseline MuSc-2 by 0.7 and 5.6 AU-ROC respectively.","On VisA, the method reports 97.4 AU-ROC segmentation, surpassing fine-tuned prompt methods such as AnomalyCLIP and the one-shot PatchCore baseline.","Because the model is training-free and the test distribution is not required in advance, it avoids the posterior optimization that MuSc needs and the data-leakage risk of prompt-tuning methods.","Batch-based inference means detection can run on the same groups of images that an industrial inspection line would process, with no need to hold the full test set in memory.","The mutual-filtering loop combines multi-layer and multi-scale features, so the method's gains come from the iterative refinement rather than any single similarity computation."],"supporting_citations":[{"why":"Supplies the frozen CLIP vision and text encoders, and the image-text similarity, on which both heads of FiSeCLIP are built.","marker":"[34]"},{"why":"Introduces mutual scoring of unlabeled images for zero-shot anomaly detection; FiSeCLIP's feature-matching branch extends it to batch-wise filtering and is compared against its MuSc-2 variant.","marker":"[26]"},{"why":"Contributes the nearest-neighbour feature matching and neighborhood aggregation used to score each patch against a reference set.","marker":"[35]"},{"why":"Establishes the CLIP-based zero-shot anomaly detection protocol with normal/abnormal text prompts that FiSeCLIP's classification and segmentation heads build upon.","marker":"[19]"},{"why":"Provides the observation that residual connections and feed-forward layers weaken dense segmentation, motivating the proxy-layer attention swap in SeCLIP-AD.","marker":"[20]"},{"why":"Serves as the prompt-tuned state-of-the-art baseline AdaCLIP that FiSeCLIP claims to outperform without training.","marker":"[7]"},{"why":"Provides the learnable-prompt baseline AnomalyCLIP used as a comparison in the main experiments.","marker":"[47]"},{"why":"Supplies the MVTec-AD benchmark and the test-set characteristics on which the primary numbers and ablations are reported.","marker":"[4]"}],"fun_headline_variants":["Mutual filtering with text guidance boosts zero-shot anomaly detection","Training-free CLIP spots defects better with batch-based mutual filtering","Zero-shot anomaly detection: mutual filtering lifts MVTec scores","FiSeCLIP uses text-filtered batch references for zero-shot defect detection","Batch mutual filtering improves training-free zero-shot anomaly detection"],"cache_read_input_tokens":16256,"weakest_assumption_plain":"The load-bearing premise is that, after text-guided filtering, the other images in a test batch still contain enough normal patches to serve as clean references; the paper never tests batches that are mostly or entirely defective, so a contaminated reference pool remains an unexamined failure mode.","fun_headline_variants_meta":{"raw":{"variants":["Mutual filtering with text guidance boosts zero-shot anomaly detection","Training-free CLIP spots defects better with batch-based mutual filtering","Zero-shot anomaly detection: mutual filtering lifts MVTec scores","FiSeCLIP uses text-filtered batch references for zero-shot defect detection","Batch mutual filtering improves training-free zero-shot anomaly detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000339,"raw_usage":{"total_tokens":1905,"prompt_tokens":1014,"completion_tokens":891,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":630,"completion_tokens_details":{"reasoning_tokens":805}},"tokens_in":630,"tokens_out":891,"duration_ms":11001,"temperature":1.0,"reasoning_tokens":805,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:18:58.074929+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Batch the MVTec-AD test set into groups whose anomaly fraction is varied from 0% to 100% (for example, all-normal batches, natural batches, then batches made only of 'damaged' class images) and run the released FiSeCLIP code; if anomaly AU-ROC and F1-max fall monotonically and collapse near the all-defect end, the assumption that a batch always provides normal reference patches is falsified.","supporting_citations":[{"cited_title":"Learning transferable visual models from natural language supervi- sion","cited_arxiv_id":null,"evidence_quote":"Supplies the frozen CLIP vision and text encoders, and the image-text similarity, on which both heads of FiSeCLIP are built."},{"cited_title":"Towards total recall in industrial anomaly detection","cited_arxiv_id":null,"evidence_quote":"Contributes the nearest-neighbour feature matching and neighborhood aggregation used to score each patch against a reference set."},{"cited_title":"Winclip: Zero- /few-shot anomaly classification and segmentation","cited_arxiv_id":null,"evidence_quote":"Establishes the CLIP-based zero-shot anomaly detection protocol with normal/abnormal text prompts that FiSeCLIP's classification and segmentation heads build upon."},{"cited_title":"Adaclip: Adapting clip with hybrid learnable prompts for zero-shot anomaly de- tection","cited_arxiv_id":null,"evidence_quote":"Serves as the prompt-tuned state-of-the-art baseline AdaCLIP that FiSeCLIP claims to outperform without training."},{"cited_title":"Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection","cited_arxiv_id":null,"evidence_quote":"Supplies the MVTec-AD benchmark and the test-set characteristics on which the primary numbers and ablations are reported."}],"review_version":1}