REVIEW 6 cited by
ClipSAM: CLIP and SAM Collaboration for Zero-Shot Anomaly Segmentation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recently, foundational models such as CLIP and SAM have shown promising performance for the task of Zero-Shot Anomaly Segmentation (ZSAS). However, either CLIP-based or SAM-based ZSAS methods still suffer from non-negligible key drawbacks: 1) CLIP primarily focuses on global feature alignment across different inputs, leading to imprecise segmentation of local anomalous parts; 2) SAM tends to generate numerous redundant masks without proper prompt constraints, resulting in complex post-processing requirements. In this work, we innovatively propose a CLIP and SAM collaboration framework called ClipSAM for ZSAS. The insight behind ClipSAM is to employ CLIP's semantic understanding capability for anomaly localization and rough segmentation, which is further used as the prompt constraints for SAM to refine the anomaly segmentation results. In details, we introduce a crucial Unified Multi-scale Cross-modal Interaction (UMCI) module for interacting language with visual features at multiple scales of CLIP to reason anomaly positions. Then, we design a novel Multi-level Mask Refinement (MMR) module, which utilizes the positional information as multi-level prompts for SAM to acquire hierarchical levels of masks and merges them. Extensive experiments validate the effectiveness of our approach, achieving the optimal segmentation performance on the MVTec-AD and VisA datasets.
Forward citations
Cited by 6 Pith papers
-
Toward Long-Tailed Online Anomaly Detection through Class-Agnostic Concepts
The paper proposes the long-tailed online anomaly detection (LTOAD) benchmark and a class-agnostic concept-based framework that outperforms class-aware baselines in most offline settings and in the online setting.
-
TAGS: 3D Tumor-Adaptive Guidance for SAM
A SAM-based 3D tumor segmentation framework combining TotalSegmentator organ masks, CLIP text guidance, and multi-stage adapters outperforms several medical segmentation baselines on three CT datasets.
-
CLIP-Guided Label-Free Discriminative Region Scoring for Fine-Grained Classification
Random-crop regions scored by CLIP pseudo-label soft negative margin outperform SAM-mask regions for label-free fine-grained classification.
-
SAM-MI: A Mask-Injected Framework for Enhancing Open-Vocabulary Semantic Segmentation with SAM
SAM-MI improves open-vocabulary segmentation by injecting aggregated SAM masks as low- and high-frequency guidance into CLIP cost maps, with sparse text-guided point prompts for speed.
-
Enhancing Spatial Reasoning in Vision-Language Models via Chain-of-Thought Prompting and Reinforcement Learning
Scene-graph-based chain-of-thought prompting and GRPO training improve spatial reasoning accuracy in vision-language models, and GRPO degrades less than supervised fine-tuning when question wording is flipped.
-
StackCLIP: Clustering-Driven Stacked Prompt in Zero-Shot Industrial Anomaly Detection
Stacking multiple category names in a CLIP text prompt, along with cluster-specific alignment layers, improves zero-shot industrial defect detection and localization.
Discussion (0). Sign in to comment.