Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

MadCLIP: Few-shot Medical Anomaly Detection with CLIP

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read MadCLIP adapts CLIP with two separate adapter branches and learnable text prompts so that few-shot medical anomaly detection can outperform prior methods without memory banks or synthetic data.

desk verdict Solid new CLIP adaptation combo with honest ablations, but the SOTA claim rests on single-run AUCs and sub-1% margins; worth refereeing with a variance requirement. read the letter →

arxiv 2506.23810 v1 pith:6JLLW7GC submitted 2025-06-30 cs.CV

classification cs.CV
keywords medicalanomalydetectionfew-shotlearningCLIPadapterslearnablepromptsSigLIPlosssegmentationcross-datasetgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper seeks to establish that a frozen CLIP model can be adapted for few-shot medical anomaly detection by modeling normal and abnormal patterns separately. It introduces MadCLIP, which inserts two sets of learnable adapters into CLIP's vision encoder, one for normal and one for abnormal samples, and couples them with learnable text prompts and a SigLIP loss. Tested on six datasets over five modalities, the method is reported to beat prior state of the art in anomaly classification and segmentation, in both same-dataset and cross-dataset settings, while avoiding memory banks and synthetic data. If correct, medical anomaly detection would become practical with as few as 16 labeled samples per class.

What carries the argument

The central mechanism is a dual-branch adapter-prompt module inside a frozen CLIP vision encoder. At four selected layers, two learnable adapters (normal and abnormal) each refine the shared CLIP features, and two sets of learnable text prompts provide class-specific semantic targets. The decision score is the difference between cosine similarities to the normal and abnormal prompts, which is summed across layers for classification and interpolated for segmentation; SigLIP loss makes the image-text alignment well-posed when each prompt is paired with many images. This subtraction-based contrast is what separates the two classes.

What would settle it

Run MadCLIP and its closest baselines multiple times with different random seeds on the same six datasets; if the average AUC gaps, such as 0.22% in segmentation, shrink below seed-to-seed variance or change sign, the claim of state-of-the-art performance is not supported.

Watch

Extended reading notes

Core claim

The central claim is that CLIP, kept frozen, can be turned into a strong few-shot medical anomaly detector by giving it two complementary branches. The normal branch aligns visual features from normal images with a learnable 'normal' text prompt; the abnormal branch does the same for abnormal images and an 'abnormal' prompt. Training maximizes each branch's similarity to its own prompt while minimizing similarity to the opposing prompt, effectively subtracting one representation from the other to separate the two classes. A SigLIP loss handles the many-to-one association between images and the two prompt sets, and multi-level adapters at layers 6, 12, 18, and 24 supply both detection and segmentation features. The paper reports state-of-the-art AUC on average across six datasets, with the largest gains in anomaly classification and a smaller but positive gain in segmentation.

Load-bearing premise

The reported performance advantage over prior methods assumes the AUC differences, some below 1%, are larger than run-to-run variation; the main tables report single-point values without error bars or significance tests.

Editorial extensions

If this is right

  • Few-shot anomaly detection on medical images can work with 16 labeled samples per class and no memory bank, lowering the data and storage barrier for clinical deployment.
  • The same frozen CLIP backbone can serve both image-level classification and pixel-level segmentation through two task heads attached to the adapters.
  • Because the method transfers across datasets of the same modality, a model trained on one brain MRI set can be applied to another without retraining.
  • Replacing softmax-based CLIP loss with SigLIP loss yields consistent gains in this setting, suggesting the choice matters when text and images have many-to-one pairings.
  • Each component—separate adapters, opposite-class subtraction, learnable prompts, and SigLIP loss—contributes positively, so the full design is needed for the reported performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Not claimed by the paper: if the margin over the second-best method is genuinely above seed noise, the subtraction-based score could serve as an interpretability signal, where the gap between normal and abnormal alignment indicates how confidently an image is anomalous.
  • An extension the paper leaves implicit: the learned prompts and adapters could be frozen and applied zero-shot to unseen anatomical regions, since the design does not depend on a memory bank; the paper mentions zero-shot only as future work.
  • An editorial inference: avoiding a memory bank means inference memory and latency scale with the CLIP encoder alone, which could simplify deployment on resource-limited devices, though the paper does not measure this.
  • A testable extension: applying the same dual-branch adapter and SigLIP recipe to other modalities such as ultrasound or dermoscopy would show whether the gains generalize beyond the five modalities reported.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes MadCLIP, a few-shot medical anomaly detection method that adapts CLIP using a dual-branch vision encoder with separate normal and abnormal adapters, learnable text prompts, and SigLIP loss. The method is evaluated for image-level anomaly classification (AC) and pixel-level anomaly segmentation (AS) on six datasets spanning five modalities, under both same-dataset and cross-dataset protocols. The authors claim that MadCLIP outperforms existing state-of-the-art methods while avoiding memory banks and synthetic data, and they report ablation studies supporting the contribution of each component.

Significance. If the empirical claims are confirmed, MadCLIP is a practically attractive contribution: it is a relatively simple CLIP adaptation that requires no memory bank or synthetic abnormal data, shows potentially large gains on some datasets (e.g., HIS and Chest AC), and includes a cross-dataset evaluation. The public code release is a further strength. However, the central comparative claim currently rests on single-run AUC numbers, and the average AS margin over the second-best method is only 0.22%, which is within plausible seed-to-seed variation for few-shot training. The method itself is plausible and the ablations are directionally consistent, but the statistical evidence for 'outperforming all SOTA' is not yet established.

major comments (3)
  1. [§3, Tables 1–3] The main SOTA comparisons in Tables 1–3 report a single AUC per cell with no standard deviation, confidence interval, or significance test, although the ablation paragraph states that results are averaged over three seeds. This is load-bearing because the headline claim of 'best overall performance' rests on an average AS margin of 0.22% over MVFA (Table 1, Average AS column), with per-dataset margins of +0.27, +0.01, and +0.38 on tasks where AUC exceeds 97%. Such differences are within plausible seed-to-seed variance for 16-shot training, so the claim that MadCLIP outperforms all SOTA on AS is not yet supported. Please report mean and standard deviation over at least three (preferably five) seeds for Tables 1–3, include paired statistical tests or confidence intervals for the comparison against MVFA and the closest competitors, and apply the same reporting standard to the ablation percentages in §3.
  2. [§2, Loss Function] The composite loss L = Σ_i (λ1 Dice(ˆM^i, M) + λ2 Focal(ˆM^i, M) + λ3 SigLip(ˆc^i, c)) requires a ground-truth mask M for every training image, but the experimental section evaluates AC-only datasets (HIS, Chest, OCT17) that have no pixel-level annotations. The manuscript does not state how the segmentation losses are computed for those datasets (e.g., omitted, set to a constant, or trained on a subset). This ambiguity makes the reported AC results on those datasets difficult to reproduce and should be resolved explicitly.
  3. [§3, Cross-dataset analysis (Table 3)] The cross-dataset claim is supported by a comparison against a single baseline (MVFA). Since the paper's contribution explicitly includes cross-dataset generalization, the authors should either add the other few-shot SOTA methods (e.g., APRIL-GAN, MediCLIP) to Table 3, or soften the conclusion so it claims improvement over MVFA rather than general 'superior cross-dataset generalization.'
minor comments (6)
  1. [§2, Learnable Prompts and Setup] The symbol M is used both for the number of learnable prompt tokens (in p_n = [V_1]...[V_M]) and for the ground-truth mask (in the setup and loss function). This notational conflict should be resolved, for instance by using K for the number of prompt tokens.
  2. [§2, Dual Branch Architecture] The statement 'Dab = 1 − Dn' is not a meaningful relation for general probability distributions. Since the actual optimization objective uses subtraction of cosine similarities, the formal equality is unnecessary; please either define what is meant by complementarity or remove the equation.
  3. [§2, Inference] When computing SoftMax(S^i) for the patch-level score vector S^i = [S^i_n, S^i_ab], it should be specified whether the softmax is applied over the normal/abnormal channels or over spatial positions, and how the resulting maps are combined to produce the final AC score and AS mask.
  4. [Throughout] The spelling of the loss name is inconsistent: 'SigLip' and 'SigLIP' both appear. Please use a single convention.
  5. [§3, Table 1] There are minor typographical issues, such as 'A verage' in the Table 1 header and inconsistent capitalization of 'SoftMax'/'Softmax'.
  6. [§1, Introduction] The claim of 'first application of SigLIP loss in medical AD' is strong and not supported by a systematic survey; consider replacing 'first' with 'to the best of our knowledge' or conducting a broader related-work search.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the method is an empirical pipeline validated on held-out test sets, with no fitted parameter or self-citation chain defining the reported metric.

full rationale

The paper's derivation chain is self-contained and empirical. MadCLIP defines a dual-branch adapter architecture, learnable prompts, and a SigLIP-based loss in Section 2, then measures AC and AS as held-out AUC on standard benchmarks in Section 3. The anomaly score used at inference (e.g., S_i^n = O_i^n·t_n - O_i^n·t_ab) is the same contrastive objective used in training, but this is ordinary supervised classification rather than circularity: the labels and test images are external to the fitted parameters, and the reported AUC values come from unseen test sets. The claimed superiority over SOTA is an experimental comparison, not a consequence of the definitions. Ablations are performed on the same benchmark used for the final comparison, which creates a mild model-selection risk, but that is not circular reasoning and the paper discloses that the ablations are averaged over three seeds. There are no self-citations that carry load-bearing weight: none of the authors' prior works are invoked to justify design choices or to forbid alternatives. The SigLIP loss is justified by the architecture's many-to-one image-text pairing and is ablated empirically, so it is not an ansatz smuggled in via citation. The only substantive weakness is statistical: Tables 1-3 report single-point AUC values without error bars or significance tests, so small margins such as 0.22% in average AS may not exceed seed-to-seed variance. That concern affects evidential strength, not circularity. No step in the paper reduces, by its own equations or by self-citation, to its own inputs.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The method depends on a series of design assumptions, most importantly that CLIP features can be adapted to medical images with small linear layers and that the balanced 16-shot training setup represents the anomaly detection problem. Several hyperparameters are unreported, and the statistical reliability of the reported AUC gains is not established.

free parameters (6)
  • Number of learnable prompt tokens M = Not reported
    Length of learnable prompt vectors [V_i] and [W_i]; chosen by hand, no sensitivity analysis.
  • Number of synonym prompts k = Not reported
    Ensemble size for text prompts; affects text feature aggregation, not specified.
  • Feature layers for adapters = {6, 12, 18, 24}
    Manually selected layers in CLIP ViT-L/14 instead of using all layers; could affect performance.
  • Loss weights = lambda1=lambda2=lambda3=1
    Fixed to equal weights; no tuning or sensitivity analysis.
  • Objective token text = e.g., 'Brain'
    Hand-picked per modality; requires expert knowledge and may not transfer to unseen modalities.
  • Adapter hidden dimensions = Not reported
    Configuration of linear layers in adapters is unspecified, affecting capacity.
assumptions (5)
  • domain assumption CLIP features pretrained on natural images contain transferable representations for medical anomaly detection after light adaptation.
    Central to the design; if false, the adapters cannot bridge the domain gap. Invoked throughout Section 2.
  • ad hoc to paper Anomalous and normal distributions are complementary: Dab = 1 - Dn.
    Stated in Section 2 (Dual Branch Architecture) and used to justify the subtraction objective; not derived from data.
  • domain assumption A balanced few-shot training set with 16 or fewer samples per class is sufficient to learn generalized normal and abnormal representations.
    Necessary for the experimental claims; relies on the prior benchmark protocol of MVFA [13].
  • domain assumption AUC differences reported in Tables 1-3 are larger than seed-to-seed variance.
    No error bars or significance tests are reported; the central superiority claim relies on this.
  • domain assumption The medical text prompts are encoded meaningfully by the CLIP text encoder for medical concepts.
    The learnable prompts are initialized around standard CLIP templates; if text encoding is poor, the dual-branch alignment fails.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MadCLIP: Few-shot Medical Anomaly Detection with CLIP." pith.science (2026). https://pith.science/paper/6JLLW7GC

@misc{pith2026250623810,
  author       = {Pith},
  title        = {Pith review of: MadCLIP: Few-shot Medical Anomaly Detection with CLIP},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6JLLW7GC}},
  note         = {Machine review of arXiv:2506.23810}
}
read the original abstract

An innovative few-shot anomaly detection approach is presented, leveraging the pre-trained CLIP model for medical data, and adapting it for both image-level anomaly classification (AC) and pixel-level anomaly segmentation (AS). A dual-branch design is proposed to separately capture normal and abnormal features through learnable adapters in the CLIP vision encoder. To improve semantic alignment, learnable text prompts are employed to link visual features. Furthermore, SigLIP loss is applied to effectively handle the many-to-one relationship between images and unpaired text prompts, showcasing its adaptation in the medical field for the first time. Our approach is validated on multiple modalities, demonstrating superior performance over existing methods for AC and AS, in both same-dataset and cross-dataset evaluations. Unlike prior work, it does not rely on synthetic data or memory banks, and an ablation study confirms the contribution of each component. The code is available at https://github.com/mahshid1998/MadCLIP.

Figures

Figures reproduced from arXiv: 2506.23810 by the authors.

Figure 1
Figure 1. Overview of MadCLIP: A dual-branch design integrates adapters An and Aab into CLIP’s vision encoder to separately capture normal and abnormal features. Learn￾able text prompts V1, . . . , VM and W1, . . . , WM encode complementary semantics for AD. The outputs are image-level AC cˆ and AS mask Mˆ . a dual optimization process, maximizing multimodal (i.e., text and vision) simi￾larity within each class while minimizi… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Diffusion-Based Data Augmentation for Medical Image Segmentation

    cs.CV 2025-08 conditional novelty 3.0 of 10

    DiffAug augments medical training data with text-and-mask guided diffusion inpainting, filtered by a latent-space segmentation network, improving polyp and optic-disc segmentation.

Reference graph

Works this paper leans on

37 extracted references · 28 canonical work pages · cited by 1 Pith paper

  1. [1]

    arXiv:2107.02314 (2021)

    Baid, U., Ghodasara, S., Mohan, S., et al.: The rsna-asnr-miccai brats 2021 benchmark on brain tumor segmentation and radiogenomic classification. arXiv:2107.02314 (2021)

  2. [2]

    Scientific data 4(1), 1–13 (2017)

    Bakas, S., Akbari, H., Sotiras, A., et al.: Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features. Scientific data 4(1), 1–13 (2017)

  3. [3]

    Jama 318(22), 2199–2210 (2017)

    Bejnordi, B.E., Veta, M., Van Diest, P.J., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. Jama 318(22), 2199–2210 (2017)

  4. [4]

    Medical Image Analysis84, 102680 (2023)

    Bilic, P., Christ, P., Li, H.B., et al.: The liver tumor segmentation benchmark (lits). Medical Image Analysis84, 102680 (2023)

  5. [5]

    arXiv:2305.173822(4) (2023)

    Chen, X., Han, Y., Zhang, J.: A zero-/fewshot anomaly classification and segmen- tation method for cvpr 2023 vand workshop challenge tracks 1&2: 1st place on zero-shot ad and 4th place on few-shot ad. arXiv:2305.173822(4) (2023)

  6. [6]

    In: CVPR

    Deng, H., Li, X.: Anomaly detection via reverse distillation from one-class embed- ding. In: CVPR. pp. 9737–9746 (2022)

  7. [7]

    In: CVPR

    Ding, C., Pang, G., Shen, C.: Catching both gray and black swans: Open-set su- pervised anomaly detection. In: CVPR. pp. 7388–7398 (2022)

  8. [8]

    ACM Comp

    Fernando, T., Gammulle, H., Denman, S., Sridharan, S., Fookes, C.: Deep learning for medical anomaly detection–a survey. ACM Comp. Surveys54(7), 1–37 (2021)

Show all 37 references
  1. [9]

    In: WACV

    Gudovskiy,D.,Ishizaka,S.,Kozuka,K.:Cflow-ad:Real-timeunsupervisedanomaly detection with localization via conditional normalizing flows. In: WACV. pp. 98– 107 (2022)

  2. [10]

    Medical image analysis55, 216–227 (2019)

    Hu, J., Chen, Y., Yi, Z.: Automated segmentation of macular edema in oct using deep neural networks. Medical image analysis55, 216–227 (2019)

  3. [11]

    Biomolecules14(5), 590 (2024)

    Hua, L., Luo, Y., Qi, Q., Long, J.: Medicalclip: Anomaly-detection domain gener- alization with asymmetric constraints. Biomolecules14(5), 590 (2024)

  4. [12]

    In: ECCV

    Huang, C., Guan, H., Jiang, A., Zhang, Y., Spratling, M., Wang, Y.F.: Registration based few-shot anomaly detection. In: ECCV. pp. 303–319. Springer (2022) 10 Mahshid Shiri, Cigdem Beyan, and Vittorio Murino

  5. [13]

    In: CVPR

    Huang, C., Jiang, A., Feng, J., Zhang, Y., Wang, X., Wang, Y.: Adapting visual- language models for generalizable anomaly detection in medical images. In: CVPR. pp. 11375–11385 (2024)

  6. [14]

    In: AAAI

    Irvin, J., Rajpurkar, P., Ko, M., et al.: Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In: AAAI. pp. 590–597 (2019)

  7. [15]

    Journal of Magnetic Resonance Imaging 27(4), 685–691 (2008)

    Jack Jr, C.R., Bernstein, M.A., Fox, N.C., et al.: The alzheimer’s disease neu- roimaging initiative (adni): Mri methods. Journal of Magnetic Resonance Imaging 27(4), 685–691 (2008)

  8. [16]

    In: CVPR (2023)

    Jeong, J., Zou, Y., Kim, T., Zhang, D., Ravichandran, A., Dabeer, O.: Winclip: Zero-/few-shot anomaly classification and segmentation. In: CVPR (2023)

  9. [17]

    cell172(5), 1122–1131 (2018)

    Kermany, D.S., Goldbaum, M., Cai, W., et al.: Identifying medical diagnoses and treatable diseases by image-based deep learning. cell172(5), 1122–1131 (2018)

  10. [18]

    Scientific data11 (2024)

    Kulyabin, M., Zhdanov, A., Nikiforova, A., et al.: Octdl: Optical coherence tomog- raphy dataset for image-based deep learning methods. Scientific data11 (2024)

  11. [19]

    In: MICCAI Multi-Atlas Labeling Beyond Cranial Vault—Workshop Challenge

    Landman, B., Xu, Z., Igelsias, J., et al.: Miccai multi-atlas labeling beyond the cranial vault–workshop and challenge. In: MICCAI Multi-Atlas Labeling Beyond Cranial Vault—Workshop Challenge. vol. 5, p. 12 (2015)

  12. [20]

    In: WACV

    Li, Y., Goodge, A., Liu, F., Foo, C.S.: Promptad: Zero-shot anomaly detection using text prompts. In: WACV. pp. 1093–1102 (2024)

  13. [21]

    IEEE transactions on medical imaging 34(10), 1993–2024 (2014)

    Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al.: The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions on medical imaging 34(10), 1993–2024 (2014)

  14. [22]

    In: fourth international Conf

    Milletari, F., Navab, N., Ahmadi, S.A.: V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: fourth international Conf. on 3D vision. pp. 565–571. Ieee (2016)

  15. [23]

    In: ICML

    Radford, A., Kim, J.W., Hallacy, C., et al.: Learning transferable visual models from natural language supervision. In: ICML. pp. 8748–8763. PMLR (2021)

  16. [24]

    In: CVPR

    Ross, T.Y., Dollár, G.: Focal loss for dense object detection. In: CVPR. pp. 2980– 2988 (2017)

  17. [25]

    In: CVPR

    Roth, K., Pemula, L., Zepeda, J., Schölkopf, B., Brox, T., Gehler, P.: Towards total recall in industrial anomaly detection. In: CVPR. pp. 14318–14328 (2022)

  18. [26]

    In: CVPR

    Salehi, M., Sadjadi, N., Baselizadeh, S., et al.: Multiresolution knowledge distilla- tion for anomaly detection. In: CVPR. pp. 14902–14912 (2021)

  19. [27]

    In: ICCV

    Sheynin, S., Benaim, S., Wolf, L.: A hierarchical transformation-discriminating generative model for few shot anomaly detection. In: ICCV. pp. 8495–8504 (2021)

  20. [28]

    IEEE Transactions on Pattern Analysis and Machine Intelligence 46(3), 1819–1835 (2021)

    Su, J., Shen, H., Peng, L., Hu, D.: Few-shot domain-adaptive anomaly detection for cross-site brain images. IEEE Transactions on Pattern Analysis and Machine Intelligence 46(3), 1819–1835 (2021)

  21. [29]

    NIH Clinical Center: Bethesda, MD, USA (2019)

    Summers, R.: Nih chest x-ray dataset of 14 common thorax disease categories. NIH Clinical Center: Bethesda, MD, USA (2019)

  22. [30]

    In: CVPR

    Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised clas- sification and localization of common thorax diseases. In: CVPR. pp. 2097–2106 (2017)

  23. [31]

    arXiv:2210.10163 (2022)

    Wang, Z., Wu, Z., Agarwal, D., Sun, J.: Medclip: Contrastive learning from un- paired medical images and text. arXiv:2210.10163 (2022)

  24. [32]

    In: CVPR

    Yao, X., Li, R., Zhang, J., Sun, J., Zhang, C.: Explicit boundary guided semi- push-pull contrastive learning for supervised anomaly detection. In: CVPR. pp. 24490–24499 (2023) MadCLIP: Few-shot Medical Anomaly Detection with CLIP 11

  25. [33]

    In: ICCV

    Zhai, X., Mustafa, B., Kolesnikov, A., Beyer, L.: Sigmoid loss for language image pre-training. In: ICCV. pp. 11975–11986 (2023)

  26. [34]

    IEEE transactions on medical imaging 40(3), 879–890 (2020)

    Zhang, J., Xie, Y., Pang, G., et al.: Viral pneumonia screening on chest x-rays using confidence-aware anomaly detection. IEEE transactions on medical imaging 40(3), 879–890 (2020)

  27. [35]

    In: MICCAI

    Zhang, X., Xu, M., Qiu, D., Yan, R., Lang, N., Zhou, X.: Mediclip: Adapting clip for few-shot medical image anomaly detection. In: MICCAI. pp. 458–468. Springer (2024)

  28. [36]

    International Journal of Comp

    Zhou, K., Yang, J., Loy, C.C., Liu, Z.: Learning to prompt for vision-language models. International Journal of Comp. Vision (2022)

  29. [37]

    arXiv:2310.18961 (2023)

    Zhou, Q., Pang, G., Tian, Y., He, S., Chen, J.: Anomalyclip: Object-agnostic prompt learning for zero-shot anomaly detection. arXiv:2310.18961 (2023)

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.