Pith. sign in

REVIEW 10 cited by

MedCLIP-SAMv2: Towards Universal Text-Driven Medical Image Segmentation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2409.19483 v4 pith:BTEPN4D3 submitted 2024-09-28 cs.CV cs.CL

Taha Koleilat , Hojat Asgariandehkordi , Hassan Rivaz , Yiming Xiao This is my paper · ORCID
classification cs.CVcs.CL
keywords segmentationmedicalimagemedclip-samv2modelszero-shotclinicalclip
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Segmentation of anatomical structures and pathological regions in medical images is essential for modern clinical diagnosis, disease research, and treatment planning. While significant advancements have been made in deep learning-based segmentation techniques, many of these methods still suffer from limitations in data efficiency, generalizability, and interactivity. As a result, developing precise segmentation methods that require fewer labeled datasets remains a critical challenge in medical image analysis. Recently, the introduction of foundation models like CLIP and Segment-Anything-Model (SAM), with robust cross-domain representations, has paved the way for interactive and universal image segmentation. However, further exploration of these models for data-efficient segmentation in medical imaging is still needed and highly relevant. In this paper, we introduce MedCLIP-SAMv2, a novel framework that integrates the CLIP and SAM models to perform segmentation on clinical scans using text prompts, in both zero-shot and weakly supervised settings. Our approach includes fine-tuning the BiomedCLIP model with a new Decoupled Hard Negative Noise Contrastive Estimation (DHN-NCE) loss, and leveraging the Multi-modal Information Bottleneck (M2IB) to create visual prompts for generating segmentation masks from SAM in the zero-shot setting. We also investigate using zero-shot segmentation labels within a weakly supervised paradigm to enhance segmentation quality further. Extensive testing across four diverse segmentation tasks and medical imaging modalities (breast tumor ultrasound, brain tumor MRI, lung X-ray, and lung CT) demonstrates the high accuracy of our proposed framework. Our code is available at https://github.com/HealthX-Lab/MedCLIP-SAMv2.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs

    cs.CV 2026-07 conditional novelty 6.0 of 10

    C-PTQ weights quantization error by per-channel Fisher information of the task loss, improving low-bit accuracy of multimodal LLMs by small margins over existing channel-wise scaling methods.

  2. Are Vision Foundation Models Ready for Out-of-the-Box Medical Image Registration?

    eess.IV 2025-07 conditional novelty 6.0 of 10

    On a benchmark of four breast MRI registration tasks, the SAM encoder outperforms classical optimization methods for gross breast alignment but not for fine fibroglandular tissue, and medical pre-training did not help.

  3. Grounding DINO-US-SAM: Text-Prompted Multi-Organ Segmentation in Ultrasound with LoRA-Tuned Vision-Language Models

    cs.CV 2025-06 conditional novelty 6.0 of 10

    LoRA-tuned Grounding DINO feeding boxes to a frozen SAM2 achieves strong text-prompted, box-free multi-organ ultrasound segmentation across 18 public datasets including three unseen domains.

  4. Brain Imaging Foundation Models, Are We There Yet? A Systematic Review of Foundation Models for Brain Imaging and Biomedical Research

    eess.IV 2025-06 conditional novelty 6.0 of 10

    A systematic review of brain imaging foundation models covering 86 models and 161 datasets, with a performance tournament, dataset atlas, and duplicated-data warnings.

  5. BiomedCoOp: Learning to Prompt for Biomedical Vision-Language Models

    cs.CV 2024-11 conditional novelty 6.0 of 10

    BiomedCoOp improves few-shot biomedical image classification by aligning learnable prompts with selectively pruned LLM-generated prompt ensembles and distilling their knowledge into BiomedCLIP.

  6. TextSAM-EUS: Text Prompt Learning for SAM to Accurately Segment Pancreatic Tumor in Endoscopic Ultrasound

    cs.CV 2025-07 conditional novelty 5.0 of 10

    TextSAM-EUS combines learned text prompts with LoRA-tuned SAM to segment pancreatic tumors in endoscopic ultrasound automatically, reaching 82.69% Dice on a public dataset.

  7. Deep Learning for Semen Analysis in Male Infertility: Computer Vision, Multimodal Fusion, and Clinical Translation

    cs.CV 2026-07 accept novelty 4.0 of 10

    A comprehensive review synthesizing AI-driven sperm analysis across computer vision tasks, multimodal fusion, and a staged clinical translation roadmap.

  8. Prompt Mechanisms in Medical Imaging: A Comprehensive Survey

    eess.IV 2025-06 conditional novelty 4.0 of 10

    A broad survey that organizes prompt mechanisms for medical image generation, segmentation, and classification into a two-dimensional taxonomy of core technologies and clinical applications.

  9. Zero-Shot Gaze-based Volumetric Medical Image Segmentation

    cs.CV 2025-05 conditional novelty 4.0 of 10

    Gaze-based prompts can drive zero-shot SAM-2 and MedSAM-2 segmentation of 3D abdominal CT organs, trading a roughly 0.08 Dice drop for an approximately 26 second per-organ speedup over manual bounding boxes.

  10. DCFormer: Efficient 3D Vision-Language Modeling with Decomposed Convolutions

    cs.CV 2025-02 conditional novelty 4.0 of 10

    A decomposed 3D convolution encoder for chest CT reaches competitive pathology detection and image-text retrieval with far fewer parameters and FLOPs than transformer or full 3D convolution baselines.

Pith tools