Pith. sign in

REVIEW 4 major objections 5 minor 42 references

MADPOT: Medical Anomaly Detection with CLIP Adaptation and Partial Optimal Transport

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MADPOT claims that adapting CLIP with visual adapters, multiple learnable prompts aligned by Partial Optimal Transport, and contrastive learning achieves state-of-the-art medical anomaly classification and segmentation in few-shot…

desk verdict Strong few-shot pipeline; zero-shot SOTA claim outruns the evidence. read the letter →

arxiv 2507.06733 v1 pith:ANIWQVXR submitted 2025-07-09 cs.CV

Mahshid Shiri , Cigdem Beyan , Vittorio Murino This is my paper
classification cs.CV
keywords medicalanomalydetectionpartialoptimaltransportCLIPadaptationlearnablepromptscontrastivelearningfew-shotzero-shotsegmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Medical anomaly detection usually depends on large unlabeled archives or synthetic anomaly data. MADPOT tries to show that a pre-trained CLIP model, given only a handful of labeled normal and abnormal images, can instead be adapted with lightweight visual adapters and multiple learnable text prompts per class. The load-bearing idea is that aligning these prompts to image patches with Partial Optimal Transport, which moves only a fixed fraction of the probability mass, concentrates the alignment on small, informative abnormal regions instead of spreading it over healthy tissue, while contrastive learning keeps the normal and abnormal prompt clusters apart. The paper reports state-of-the-art anomaly classification and segmentation AUCs on the BMAD benchmark across five modalities, in few-shot, zero-shot, and cross-dataset tests, without synthetic data or memory banks. If the claims hold, accurate medical anomaly detection becomes feasible in data-scarce clinical settings.

What carries the argument

The central object is the transport plan between patch-level visual features and a set of K=4 learnable prompts per class, solved under Partial Optimal Transport with a fixed transported fraction (frac=0.8). Because POT relaxes the marginal equality constraint of classical optimal transport, the alignment is free to ignore a fifth of the mass, letting prompts attach to the most informative patches, typically small anomalies, rather than to normal tissue. Contrastive learning operates on class-fused prompt embeddings, enforcing intra-class cohesion and inter-class separation so that the multiple prompts stay diverse instead of collapsing to a single representation. On the vision side, residual adapters inserted after layer 12 and a projector attached to layer 24 map CLIP features into shared, then task-specific, spaces for classification and segmentation; the final anomaly score averages POT and contrastive logits from the two layers.

What would settle it

Re-run the BMAD few-shot protocol with ten random 16-sample train/eval splits, reporting mean and standard deviation of anomaly-classification AUC for MADPOT and MVFA; if the two means overlap within one standard deviation, the claimed state-of-the-art margin over MVFA is not established.

Watch

Extended reading notes

Core claim

The paper shows that learnable multi-prompt text tuning, constrained by Partial Optimal Transport to match only a fraction of the visual mass, and regularized by contrastive learning, adapts a frozen CLIP backbone to medical anomaly detection more effectively than fixed prompts, standard prompt learning, or prior adapter-only CLIP adaptation. The resulting model, MADPOT, separates normal and abnormal classes through multiple class-specific prompts, whose fused representations are pulled together within a class and pushed apart across classes, while POT maps each prompt to the most salient image patches. In the 16-shot few-shot regime, it reports the best average anomaly-classification AUC among compared methods (97.83%, versus 90.58% for the next best) and the best average segmentation AUC, and it again reports the highest classification AUC averages in zero-shot and cross-dataset evaluations, with segmentation performance competitive.

Load-bearing premise

The load-bearing premise is that the reported AUCs come from a fair, typical train/eval split and a zero-shot protocol comparable to the baselines, since the paper gives point estimates without variance and does not fully spell out the zero-shot setup.

Editorial extensions

If this is right

  • Anomaly detection in medical images can reach high AUC with as few as 16 labeled normal and abnormal images per class, avoiding reliance on large unlabeled archives.
  • A model trained on one dataset transfers to a different dataset of the same modality, such as ChestX-ray8 to NIH Chest or OCT17 to OCTDL, which could ease cross-hospital deployment.
  • Discarding a fraction of the transport mass lets prompts focus on small anomaly regions, which directly benefits anomaly localization as well as classification.
  • The pipeline removes the need for synthetic anomaly data and memory banks, making it lighter than approaches like MediCLIP and April-GAN.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test this, but the same partial-mass transport idea could transfer to industrial defect detection, where defects are also small and sparse, and a fixed fraction around 0.8 is plausible there as well.
  • An untested extension is to make the transport fraction per-image or per-layer rather than a fixed scalar; the ablations show sensitivity to this hyperparameter, so an adaptive schedule could improve robustness across modalities.
  • Because the evaluation uses a single train/eval split, the practical claim established is that this configuration is strong on BMAD under that split, not that it is guaranteed to beat all baselines under any split; resampling-based uncertainty estimates would be a natural next step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes MADPOT, a CLIP-based medical anomaly detection method that adds visual adapters and projectors to the CLIP vision encoder and learns multiple textual prompts for normal and abnormal classes. Prompt-image alignment is performed with partial optimal transport (POT), and contrastive learning (CL) is applied to fused class prompts. The paper evaluates anomaly classification (AC) and anomaly segmentation (AS) AUC on the BMAD benchmark, reporting state-of-the-art results in few-shot (16 samples per class), zero-shot, and cross-dataset settings, together with ablations over CL/OT/POT, adapter/projector configurations, and the POT fraction parameter.

Significance. If the reported few-shot results are reproducible, the method is a meaningful contribution: it avoids memory banks and synthetic data, shows large gains on datasets such as HIS and Chest, and is, to my knowledge, the first to apply POT-based prompt alignment to medical anomaly detection. The release of code and the use of a standard benchmark are assets. However, the significance is tempered by unresolved ambiguity in the zero-shot and cross-dataset evaluation protocols, the absence of variance estimates, and the apparent selection of the POT fraction on the evaluation benchmark. These issues directly affect the paper's central claim of state-of-the-art performance across all three settings.

major comments (4)
  1. [Section 4, 'Comparing Zero-Shot Performance with SOTA', Table 3] The zero-shot protocol is never defined. It is not stated which datasets are used as source training data, which are held out as targets, how many samples are used, or whether the same train/eval protocol is applied to APRIL-GAN and MVFA. Without this information, the claimed average AC gain of 7.99% is not a controlled comparison. Moreover, the abstract's unqualified claim of state-of-the-art zero-shot results is contradicted by Table 3, where MADPOT's average AS (91.56%) is below MVFA's (93.39%); the text attributes the deficit to RESC but does not resolve the contradiction. Please either define the protocol precisely or qualify the zero-shot claim to AC only.
  2. [Section 4, 'Cross-dataset analysis', Table 2] The cross-dataset table is uninterpretable as printed: the header lists 'Source Chest Brain OCT17 RESC' and 'Target NIHChest CheXpert ADNI OCTDL', but the rows contain six numeric entries. It is unclear which source-target pair each column denotes (for example, whether Chest is evaluated on both NIHChest and CheXpert, and which source corresponds to OCTDL). Please provide an explicit source/target mapping and state the exact training and evaluation splits used for each entry.
  3. [Tables 1-3] All AUC values appear to be single-run point estimates. Few-shot training with 16 samples per class is stochastic and depends on the particular training subset sampled; without multiple seeds or cross-validation splits, the reported margins (e.g., a 7.25% average AC gain in Table 1) cannot be assessed for statistical significance. Please report mean and standard deviation over at least 3-5 runs and specify the data split and seed handling.
  4. [Section 4, Implementation Details and ablation study on frac] The partial transport ratio frac is tuned over 0.4-0.9 and fixed at 0.8 after observing performance on the evaluation benchmark. The sensitivity analysis shows that AUC varies strongly with frac (AC drops from 99.00% to 81.61% between frac=0.8 and frac=0.4), so selecting the best value on test data can inflate the reported results. Please state clearly whether frac was chosen on a held-out validation set; if it was chosen on the test benchmark, the reported numbers should be interpreted as oracle-tuned and the claim of a parameter-free advantage should be revised.
minor comments (5)
  1. [Table 4] OT/POT alone collapse to near-random performance in Table 4 (e.g., AC 9.8% on HIS and 5.01% on Liver). The paper states that this shows POT is insufficient without CL, but offers no analysis or diagnostic. A brief explanation or additional ablation of why transport-only prompts fail would strengthen the component analysis.
  2. [Abstract and Section 4] The abstract and Section 4 use 'state-of-the-art' for zero-shot performance despite Table 3 showing that MADPOT's average AS is lower than MVFA's. Please qualify the claim to anomaly classification or introduce a composite metric that is consistently better.
  3. [Equation (1)] Equation (1)(a) and (1)(b) use `dis` and `sim` with inconsistent subscripts and summation ranges; please clarify the dimensions of the sums (G×k vs. G) and the temperature normalization.
  4. [Section 3, Partial Optimal Transport definition] The definition of the POT feasible set states '∥α∥1 ≥ ∥β∥1 = frac'; this is not a conventional way to state the partial transport constraint and should be rewritten with explicit marginal constraints involving the total mass frac.
  5. [Throughout] Minor copyedits: 'CoOP' should be 'CoOp'; the Table 5 caption has a stray comma in 'adapter and projector , showing'; and 'GDice'/'GDICE' are used inconsistently.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: MADPOT is an empirical pipeline; no claimed result reduces by construction to its own inputs.

full rationale

MADPOT does not present a derivation chain in which a prediction is equivalent to a fitted parameter or to a self-cited premise. The method is defined by training objectives (Eq. 1, Eq. 2) and losses; these are operational definitions, not outputs derived from the data they are evaluated on. The key design choices cite external, non-self prior work: adapters and projectors follow MVFA [19], prompt count follows PLOT [9], and layer choice is motivated by MVFA's ablation. None of these citations is authored by the present authors, so there is no load-bearing self-citation. The fractional transport ratio frac is tuned on the benchmark ('the final model uses 0.8'; the ablation shows peak performance at 0.8), which is a model-selection or soundness concern rather than a fitted parameter renamed as a prediction: no equation-level reduction makes the reported AUC a consequence of the frac choice by construction. The zero-shot SOTA claim is weakened by the paper's own Table 3 (average AS 91.56% vs. MVFA 93.39%) and by the unspecified zero-shot protocol, but an unsupported or inconsistent empirical claim is not circularity. Accordingly, there are no specific circular steps to report.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new particles or forces; the ledger lists the fitted hyperparameters and background assumptions that the claim depends on. The main concern is the benchmark-tuned frac and the unspecified temperature.

free parameters (4)
  • frac (partial transport ratio) = 0.8
    Controls the fraction of mass transported in POT; selected by scanning 0.4-0.9 on the BMAD benchmark (Brain, Liver, RESC) where performance peaks at 0.8, then used for all final results.
  • lambda (entropic regularization) = 0.1
    Regularization weight for the Sinkhorn/POT solver; fixed by default without reported tuning.
  • gamma (adapter residual ratio) = 0.2
    Residual ratio balancing frozen CLIP features and adapter outputs; chosen as the default value.
  • temperature tau (softmax scaling) = not specified
    Used in Eq. (1) and (2) for softmax scaling; the paper never states its value, leaving reproduction to guesswork.
assumptions (4)
  • domain assumption CLIP ViT-L/14 features, with linear adaptation, transfer to medical anomalies across modalities
    The whole method rests on the assumption that CLIP features contain sufficient signal for medical AD after shallow adaptation; Section 3.
  • domain assumption The BMAD benchmark and its train/test splits are representative of medical AD and the baselines use the same protocol
    All comparisons and conclusions are drawn from BMAD; Section 4.
  • standard math The transport plan computed by the inner loop is treated as fixed during outer-loop parameter updates
    This alternating optimization follows standard Sinkhorn/Dykstra practice; Section 3, paragraph 'Multimodal Feature Alignment'.
  • ad hoc to paper The single-run AUC is stable enough to compare methods without error bars
    The paper reports only point estimates; Section 4 tables.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MADPOT: Medical Anomaly Detection with CLIP Adaptation and Partial Optimal Transport." pith.science (2026). https://pith.science/paper/ANIWQVXR

@misc{pith2026250706733,
  author       = {Pith},
  title        = {Pith review of: MADPOT: Medical Anomaly Detection with CLIP Adaptation and Partial Optimal Transport},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ANIWQVXR}},
  note         = {Machine review of arXiv:2507.06733}
}
read the original abstract

Medical anomaly detection (AD) is challenging due to diverse imaging modalities, anatomical variations, and limited labeled data. We propose a novel approach combining visual adapters and prompt learning with Partial Optimal Transport (POT) and contrastive learning (CL) to improve CLIP's adaptability to medical images, particularly for AD. Unlike standard prompt learning, which often yields a single representation, our method employs multiple prompts aligned with local features via POT to capture subtle abnormalities. CL further enforces intra-class cohesion and inter-class separation. Our method achieves state-of-the-art results in few-shot, zero-shot, and cross-dataset scenarios without synthetic data or memory banks. The code is available at https://github.com/mahshid1998/MADPOT.

Figures

Figures reproduced from arXiv: 2507.06733 by the authors.

Figure 1
Figure 1. (a) Overview of MADPOT: Visual features are extracted via an adapter and projection on the CLIP vision encoder. Separate prompt sets for normal and abnormal are learned and fused into class-specific prompts. POT produces multiple logits, and CL is applied to the visual features and fused prompts. The final score is the average of the logits from POT and CL. (b) POT enables the model to capture localized features, wh… view at source ↗
Figure 2
Figure 2. Comparison with few-shot SOTA for 2, 4, and 8 shots per class (AUC %). For datasets with both AC and AS, the average of AC and AS is reported. (ω1 = ω2 = ω3 = 1). GDice(., .), F ocal(., .), and BCE(., .) correspond to Gener￾alized Dice, Focal, and Binary Cross Entropy losses. GDICE loss is specifically designed for highly imbalanced segmentation tasks, and Focal loss is beneficial in class imbalance scenarios, which… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 38 canonical work pages

  1. [1]

    arXiv:2107.02314 (2021)

    Baid, U., et al.: The rsna-asnr-miccai brats 2021 benchmark on brain tumor seg- mentation and radiogenomic classification. arXiv:2107.02314 (2021)

    arXiv 2021

  2. [2]

    Scientific data4, 1–13 (2017)

    Bakas, S., et al.: Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features. Scientific data4, 1–13 (2017)

    work page 2017

  3. [3]

    In: CVPR

    Bao, J., et al.: Bmad: Benchmarks for medical anomaly detection. In: CVPR. pp. 4042–4053 (2024)

    work page 2024

  4. [4]

    Jama318(22), 2199–2210 (2017)

    Bejnordi, B.E., et al.: Diagnostic assessment of deep learning algorithms for de- tection of lymph node metastases in women with breast cancer. Jama318(22), 2199–2210 (2017)

    work page 2017

  5. [5]

    SIAM Journal on Scientific Computing37(2), A1111–A1138 (2015)

    Benamou, J.D., et al.: Iterative bregman projections for regularized transportation problems. SIAM Journal on Scientific Computing37(2), A1111–A1138 (2015)

    work page 2015

  6. [6]

    Medical Image Analysis 84, 102680 (2023)

    Bilic, P., et al.: The liver tumor segmentation benchmark (lits). Medical Image Analysis 84, 102680 (2023)

    work page 2023

  7. [7]

    In: ECCV

    Cao, Y., et al.: Adaclip: Adapting clip with hybrid learnable prompts for zero-shot anomaly detection. In: ECCV. pp. 55–72 (2024)

    work page 2024

  8. [8]

    NeurIPS36, 8528–8541 (2023)

    Chang, W., et al.: Csot: Curriculum and structure-aware optimal transport for learning with noisy labels. NeurIPS36, 8528–8541 (2023)

    work page 2023

Show all 42 references
  1. [9]

    arXiv preprint arXiv:2210.01253 (2022)

    Chen, G., et al.: Plot: Prompt learning with optimal transport for vision-language models. arXiv preprint arXiv:2210.01253 (2022)

    2022 arXiv

  2. [10]

    arXiv:2305.17382 2(4) (2023)

    Chen, X., et al.: A zero-/fewshot anomaly classification and segmentation method. arXiv:2305.17382 2(4) (2023)

    2023 arXiv

  3. [11]

    In: CVPR

    Deng, H., Li, X.: Anomaly detection via reverse distillation from one-class embed- ding. In: CVPR. pp. 9737–9746 (2022)

    2022

  4. [12]

    In: CVPR

    Ding, C., et al.: Catching both gray and black swans: Open-set supervised anomaly detection. In: CVPR. pp. 7388–7398 (2022)

    2022

  5. [13]

    Journal of the American Statistical Association78(384), 837–842 (1983)

    Dykstra, R.L.: An algorithm for restricted least squares regression. Journal of the American Statistical Association78(384), 837–842 (1983)

    1983

  6. [14]

    In: CVPR

    Feng, C., et al.: Ot-filter: An optimal transport filter for learning with noisy labels. In: CVPR. pp. 16164–16174 (2023)

    2023

  7. [15]

    Advances in neural information processing systems 28 (2015)

    Frogner, C., et al.: Learning with a wasserstein loss. Advances in neural information processing systems 28 (2015)

    2015

  8. [16]

    In: WACV

    Gudovskiy, D., et al.: Cflow-ad: Real-time unsupervised anomaly detection with localization via conditional normalizing flows. In: WACV. pp. 98–107 (2022)

    2022

  9. [17]

    Medical image analysis55, 216–227 (2019)

    Hu, J., et al.: Automated segmentation of macular edema in oct using deep neural networks. Medical image analysis55, 216–227 (2019)

    2019

  10. [18]

    In: ECCV

    Huang, C., et al.: Registration based few-shot anomaly detection. In: ECCV. pp. 303–319 (2022)

    2022

  11. [19]

    In: CVPR

    Huang, C., et al.: Adapting visual-language models for generalizable anomaly de- tection in medical images. In: CVPR. pp. 11375–85 (2024)

    2024

  12. [20]

    In: AAAI

    Irvin, J., et al.: Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In: AAAI. pp. 590–597 (2019)

    2019

  13. [21]

    Journal of Magnetic Resonance Imaging27(4), 685–691 (2008)

    Jack Jr, C.R., et al.: The alzheimer’s disease neuroimaging initiative (adni): Mri methods. Journal of Magnetic Resonance Imaging27(4), 685–691 (2008)

    2008

  14. [22]

    cell172(5), 1122–1131 (2018) MADPOT 13

    Kermany, D.S., et al.: Identifying medical diagnoses and treatable diseases by image-based deep learning. cell172(5), 1122–1131 (2018) MADPOT 13

    2018

  15. [23]

    Physical Review E103(1), 012113 (2021)

    Koehl, P., et al.: Physics approach to the variable-mass optimal-transport problem. Physical Review E103(1), 012113 (2021)

    2021

  16. [24]

    Scientific data11 (2024)

    Kulyabin, M., et al.: Octdl: Optical coherence tomography dataset for image-based deep learning methods. Scientific data11 (2024)

    2024

  17. [25]

    In: MICCAI

    Landman, B., et al.: Miccai multi-atlas labeling beyond the cranial vault–workshop and challenge. In: MICCAI. vol. 5, p. 12 (2015)

    2015

  18. [26]

    In: AAAI

    Li, B., et al.: Unsupervised cross-domain image retrieval via prototypical optimal transport. In: AAAI. vol. 38, pp. 3009–3017 (2024)

    2024

  19. [27]

    In: CVPR

    Li, H., et al.: Global and local prompts cooperation via optimal transport for federated learning. In: CVPR. pp. 12151–12161 (2024)

    2024

  20. [28]

    IEEE transactions on medical imaging34(10), 1993–2024 (2014)

    Menze, B.H., et al.: The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions on medical imaging34(10), 1993–2024 (2014)

    2014

  21. [29]

    Foundations and Trends in Machine Learning11(5-6), 355–607 (2019)

    Peyré, G.e.a.: Computational optimal transport: With applications to data science. Foundations and Trends in Machine Learning11(5-6), 355–607 (2019)

    2019

  22. [30]

    In: ICML

    Radford, A., et al.: Learning transferable visual models from natural language supervision. In: ICML. pp. 8748–8763 (2021)

    2021

  23. [31]

    In: CVPR

    Roth, K., et al.: Towards total recall in industrial anomaly detection. In: CVPR. pp. 14318–14328 (2022)

    2022

  24. [32]

    In: CVPR

    Salehi, M., et al.: Multiresolution knowledge distillation for anomaly detection. In: CVPR. pp. 14902–14912 (2021)

    2021

  25. [33]

    The American Mathematical Monthly74(4), 402–405 (1967)

    Sinkhorn, R.: Diagonal equivalence to matrices with prescribed row and column sums. The American Mathematical Monthly74(4), 402–405 (1967)

    1967

  26. [34]

    NIH Clinical Center: Bethesda, MD, USA (2019)

    Summers, R.: Nih chest x-ray dataset of 14 common thorax disease categories. NIH Clinical Center: Bethesda, MD, USA (2019)

    2019

  27. [35]

    arXiv:2201.09366 (2022)

    Tu, R., et al.: Optimal transport for causal discovery. arXiv:2201.09366 (2022)

    2022 arXiv

  28. [36]

    338 (2008)

    Villani, C., et al.: Optimal transport: old and new, vol. 338 (2008)

    2008

  29. [37]

    In: CVPR

    Wang, X., et al.: Chestx-ray8: Hospital-scale chest x-ray database and benchmarks. In: CVPR. pp. 2097–2106 (2017)

    2017

  30. [38]

    In: AAAI

    Wang, Y., et al.: Probability-polarized optimal transport for unsupervised domain adaptation. In: AAAI. vol. 38, pp. 15653–15661 (2024)

    2024

  31. [39]

    In: CVPR

    Yao, X., et al.: Explicit boundary guided semi-push-pull contrastive learning for supervised anomaly detection. In: CVPR. pp. 24490–24499 (2023)

    2023

  32. [40]

    IEEE TMI40(3), 879–890 (2020)

    Zhang, J., et al.: Viral pneumonia screening on chest x-rays using confidence-aware anomaly detection. IEEE TMI40(3), 879–890 (2020)

    2020

  33. [41]

    In: MICCAI

    Zhang, X., et al.: Mediclip: Adapting clip for few-shot medical image anomaly detection. In: MICCAI. pp. 458–468 (2024)

    2024

  34. [42]

    IJCV (2022)

    Zhou, K., et al.: Learning to prompt for vision-language models. IJCV (2022)

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.