The reviewed record of science sign in
Pith

arxiv: 2605.05161 · v2 · pith:HQISPW25 · submitted 2026-05-06 · cs.CV

Wasserstein-Aligned Localisation for VLM-Based Distributional OOD Detection in Medical Imaging

Bernhard Kainz , Johanna P Mueller , Matthew Baugh , Cosmin Bercea This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-30 23:44 UTCgrok-4.3pith:HQISPW25record.jsonopen to challenge →

classification cs.CV
keywords zero-shot anomaly localisationvision-language modelsWasserstein distancemedical imagingout-of-distribution detectionbrain MRIoptimal transportreference selection
0
0 comments X

The pith

Selecting moderately similar healthy references via entropy-weighted Sliced Wasserstein distances on DINOv2 features improves zero-shot anomaly localisation in VLMs for brain MRI.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper reframes zero-shot anomaly localisation as a comparative inference task that identifies pathologies by contrasting them against reference distributions of normal anatomy. It introduces WALDO, a training-free method that selects those references from DINOv2 patch embeddings using entropy-weighted Sliced Wasserstein distances and exploits a non-monotonic relationship between reference similarity and localisation accuracy. Theoretical analysis shows that references of moderate similarity minimise a bias-variance trade-off in the comparative visual reasoning performed by the VLM. Experiments on the NOVA brain MRI benchmark report a 19 percent relative gain in mAP@30 when WALDO is paired with Qwen2.5-VL-72B, with consistent gains across other models and statistical confirmation via paired tests.

Core claim

WALDO reformulates zero-shot localisation as comparative inference against reference distributions of normal anatomy, using entropy-weighted Sliced Wasserstein distances on DINOv2 patch distributions to select references from a Goldilocks zone of moderate similarity; this selection, combined with self-consistency aggregation by weighted non-maximum suppression, yields 43.5 percent mAP@30 on the NOVA benchmark with Qwen2.5-VL-72B, a 19 percent relative improvement over zero-shot baselines.

What carries the argument

entropy-weighted Sliced Wasserstein distances on DINOv2 patch distributions, used to identify reference sets in the moderate-similarity Goldilocks zone that minimise bias-variance trade-off in VLM comparative reasoning

If this is right

  • Zero-shot VLM localisation becomes viable for rare pathologies without task-specific training data.
  • The same reference-selection procedure can be applied to other imaging modalities once DINOv2-style embeddings are available.
  • Self-consistency aggregation via weighted non-maximum suppression reduces false positives that arise from single-reference comparisons.
  • Performance scales with VLM size while the reference-selection step remains training-free and model-agnostic.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The approach could be extended to longitudinal scans by treating earlier time points as the reference distribution.
  • If the Goldilocks zone is modality-specific, the framework would require a cheap calibration step on each new scanner type.
  • Combining the Wasserstein selector with uncertainty estimates from the VLM itself might further tighten the bias-variance trade-off.

Load-bearing premise

DINOv2 patch distributions together with entropy-weighted Sliced Wasserstein distances can reliably identify reference sets whose moderate similarity minimises the bias-variance trade-off for anomaly localisation.

What would settle it

A controlled experiment that replaces the entropy-weighted Sliced Wasserstein reference selector with either random selection or selection by cosine similarity and measures whether mAP@30 on NOVA drops below the reported 43.5 percent for the same VLM.

Figures

Figures reproduced from arXiv: 2605.05161 by Bernhard Kainz, Cosmin Bercea, Johanna P Mueller, Matthew Baugh.

Figure 1
Figure 1. Figure 1: WALDO idea. The query image and healthy reference pool are processed view at source ↗
Figure 2
Figure 2. Figure 2: Qualitative results. Left three: NOVA brain MRI; Right three: VinDr view at source ↗
Figure 3
Figure 3. Figure 3: Error stratification analysis on NOVA. IoU distributions grouped by lesion view at source ↗
Figure 4
Figure 4. Figure 4: WALDO prompt example: Cerebral fat embolism ( view at source ↗
Figure 5
Figure 5. Figure 5: WALDO prompt example: Pleomorphic xanthoastrocytoma with dural view at source ↗
Figure 6
Figure 6. Figure 6: CXR WALDO prompt example: Cardiomegaly (IoU=0.80). Top row: 5 view at source ↗
read the original abstract

Zero-shot anomaly localisation via vision-language models (VLMs) offers a compelling approach for rare pathology detection, yet its performance is fundamentally limited by the absence of healthy anatomical context. We reformulate zero-shot localisation as a comparative inference problem in which anomalies are identified through structured comparison against reference distributions of normal anatomy. We introduce WALDO, a training-free framework grounded in optimal transport theory that enables comparative reasoning through: (i) entropy-weighted Sliced Wasserstein distances for anatomically-aware reference selection from DINOv2 patch distributions, (ii) Goldilocks zone sampling exploiting the non-monotonic relationship between reference similarity and localisation accuracy, and (iii) self-consistency aggregation via weighted non-maximum suppression. We theoretically analyse the Goldilocks effect through distributional divergence, and show that references with moderate similarity minimize a bias-variance trade-off in comparative visual reasoning. On the NOVA brain MRI benchmark, WALDO with Qwen2.5-VL-72B achieves $43.5_{\pm1.6}\%$ mAP@30 (95\% CI: [40.4, 46.7]), representing a 19\% relative improvement over zero-shot baselines. Cross-model evaluation shows consistent gains: GPT-4o achieves $32.0_{\pm6.5}\%$ and Qwen3-VL-32B achieves $32.0_{\pm6.6}\%$ mAP@30. Paired McNemar tests confirm statistical significance ($p<0.01$). Source code is available at https://github.com/bkainz/WALDO_MICCAI26_demo .

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper proposes WALDO, a training-free framework for zero-shot anomaly localization in medical images using vision-language models. It reformulates the task as comparative inference against reference distributions of normal anatomy, selected via entropy-weighted Sliced Wasserstein distances on DINOv2 patch features, with Goldilocks-zone sampling to exploit a posited non-monotonic similarity-accuracy relationship, and self-consistency via weighted non-maximum suppression. Theoretical analysis links distributional divergence to a bias-variance trade-off, and experiments on the NOVA brain MRI benchmark report 43.5±1.6% mAP@30 (19% relative gain over zero-shot baselines) with statistical significance across multiple VLMs.

Significance. If the core mechanism is validated, WALDO could offer a principled, training-free improvement to VLM-based OOD detection in medical imaging by grounding reference selection in optimal transport, with potential extension to other comparative reasoning tasks. The reported gains with confidence intervals and McNemar tests, plus open code, strengthen the empirical contribution if the Goldilocks assumption holds.

major comments (3)
  1. [§4] §4 (theoretical analysis of Goldilocks effect): the claim that moderate-similarity references minimize bias-variance in VLM comparative visual reasoning is derived from distributional divergence but lacks direct empirical ablation showing that the entropy-weighted Sliced Wasserstein selections occupy the accuracy peak (as opposed to any reference set or high-similarity matches).
  2. [§5] §5 (experiments on NOVA benchmark): the 19% mAP@30 improvement and p<0.01 McNemar results are attributed to the OT-based Goldilocks sampling, yet no ablation isolates the contribution of the reference selection mechanism versus other components (e.g., weighted NMS or VLM prompting), making it unclear whether the selection is load-bearing for the central claim.
  3. [§3.2] §3.2 (reference selection): the exact procedure for entropy-weighted Sliced Wasserstein distance computation on DINOv2 patches, including any post-hoc choices in reference sampling or weighting, is not fully specified, preventing verification that the selected sets empirically satisfy the moderate-similarity regime assumed in the theory.
minor comments (2)
  1. [§5] The abstract and §5 report 95% CIs and statistical tests but do not specify the number of runs or exact baseline implementations, which would aid reproducibility.
  2. [§3.3] Notation for the weighted NMS aggregation could be clarified with an equation or pseudocode to distinguish it from standard NMS.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed comments, which identify key areas where additional empirical validation and clarity would strengthen the manuscript. We address each major comment below and will revise the paper accordingly to incorporate the requested ablations and expanded specifications.

read point-by-point responses

  1. Referee: [§4] §4 (theoretical analysis of Goldilocks effect): the claim that moderate-similarity references minimize bias-variance in VLM comparative visual reasoning is derived from distributional divergence but lacks direct empirical ablation showing that the entropy-weighted Sliced Wasserstein selections occupy the accuracy peak (as opposed to any reference set or high-similarity matches).

    Authors: We agree that direct empirical evidence linking the selected references to the posited accuracy peak would better support the theoretical analysis. In the revised manuscript, we will add an ablation comparing mAP@30 performance when using entropy-weighted Sliced Wasserstein selected references versus random reference sets and high-similarity matches, to verify that the OT-based selections occupy the Goldilocks zone. revision: yes

  2. Referee: [§5] §5 (experiments on NOVA benchmark): the 19% mAP@30 improvement and p<0.01 McNemar results are attributed to the OT-based Goldilocks sampling, yet no ablation isolates the contribution of the reference selection mechanism versus other components (e.g., weighted NMS or VLM prompting), making it unclear whether the selection is load-bearing for the central claim.

    Authors: We acknowledge that isolating the reference selection mechanism is necessary to substantiate its role in the reported gains. We will add ablation experiments in the revision that replace the entropy-weighted Sliced Wasserstein reference selection with alternatives (e.g., random or similarity-thresholded selection) while holding weighted NMS and prompting fixed, to quantify its specific contribution to the 19% relative improvement. revision: yes

  3. Referee: [§3.2] §3.2 (reference selection): the exact procedure for entropy-weighted Sliced Wasserstein distance computation on DINOv2 patches, including any post-hoc choices in reference sampling or weighting, is not fully specified, preventing verification that the selected sets empirically satisfy the moderate-similarity regime assumed in the theory.

    Authors: We will expand §3.2 with a complete algorithmic description of the entropy-weighted Sliced Wasserstein computation on DINOv2 patches, including the number of projections, exact weighting formula, patch aggregation, and any sampling hyperparameters. We will also report the distribution of similarity scores for the selected references to confirm they satisfy the moderate-similarity regime. The open code repository already contains the precise implementation. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical evaluation on external benchmark with independent statistical tests

full rationale

The paper presents WALDO as a training-free framework using entropy-weighted Sliced Wasserstein distances on DINOv2 patches for reference selection, Goldilocks zone sampling justified by a claimed theoretical analysis of distributional divergence, and self-consistency aggregation. Performance is reported via direct measurement of mAP@30 on the external NOVA benchmark (43.5±1.6% with Qwen2.5-VL-72B, 19% relative gain, p<0.01 via McNemar), with cross-model consistency. No equations, fitted parameters, or predictions are shown to reduce by construction to inputs from the same data; the Goldilocks non-monotonicity is posited rather than derived from a self-referential fit. No self-citations appear in the provided text as load-bearing for the central claims. The derivation chain is self-contained against the external benchmark.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on pre-trained feature extractors (DINOv2, VLMs) and standard optimal-transport properties rather than new fitted constants or invented entities.

axioms (2)
  • domain assumption Sliced Wasserstein distance on DINOv2 patch distributions yields anatomically meaningful similarity for reference selection
    Invoked for the entropy-weighted reference selection step
  • domain assumption A non-monotonic relationship exists between reference similarity and localisation accuracy that can be exploited by Goldilocks zone sampling
    Central to the theoretical analysis and sampling procedure described in the abstract

pith-pipeline@v0.9.1-grok · 5838 in / 1498 out tokens · 31644 ms · 2026-06-30T23:44:54.826923+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 31 canonical work pages · 7 internal anchors

  1. [1]

    In: International conference on machine learning

    Arjovsky,M.,Chintala,S.,Bottou,L.:Wassersteingenerativeadversarialnetworks. In: International conference on machine learning. pp. 214–223. ICML - PMLR (2017)

    work page 2017

  2. [2]

    Qwen3-VL Technical Report

    Bai, S., Chen, K., Liu, X., Wang, P., et al.: Qwen3-VL technical report. arXiv preprint arXiv:2511.21631 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  3. [3]

    In: MICCAI

    Baugh, M., Tan, J., Müller, J.P., Dombrowski, M., Batten, J., Kainz, B.: Many tasks make light work: Learning to localise medical anomalies from multiple syn- thetic tasks. In: MICCAI. pp. 162–172. Springer (2023)

    work page 2023

  4. [4]

    In: International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging

    Baugh, M., Tan, J., Vlontzos, A., Müller, J.P., Kainz, B.: nnood: A framework for benchmarking self-supervised anomaly localisation methods. In: International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging. pp. 103–112. Springer (2022)

    work page 2022

  5. [5]

    arXiv preprint arXiv:2505.14064 (2025), NeurIPS 2025

    Bercea, C.I., Li, J., Raffler, P., Riedel, E.O., Schmitzer, L., Kurz, A., Bitzer, F., Roßmüller, P., Canisius, J., Beyrle, M.L., et al.: NOVA: A benchmark for anomaly localization and clinical reasoning in brain MRI. arXiv preprint arXiv:2505.14064 (2025), NeurIPS 2025

    work page arXiv 2025

  6. [6]

    In: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Bergmann, P., Fauser, M., Sattlegger, D., Steger, C.: Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings. In: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 4183–4192 (2020)

    work page 2020

  7. [7]

    Journal of Mathematical Imaging and Vision51(1), 22–45 (2015)

    Bonneel,N., Rabin,J., Peyré,G., Pfister,H.: Sliced andradon wassersteinbarycen- ters of measures. Journal of Mathematical Imaging and Vision51(1), 22–45 (2015)

    work page 2015

  8. [8]

    NeurIPS’2033, 1877–1901 (2020) 10 B

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Nee- lakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. NeurIPS’2033, 1877–1901 (2020) 10 B. Kainz et al

    work page 2033

  9. [9]

    IEEE TPAMI39(9), 1853–1865 (2017)

    Courty, N., Flamary, R., Tuia, D., Rakotomamonjy, A.: Optimal transport for domain adaptation. IEEE TPAMI39(9), 1853–1865 (2017)

    work page 2017

  10. [10]

    In: ICPR’21

    Defard, T., Setkov, A., Loesch, A., Audigier, R.: Padim: a patch distribution mod- eling framework for anomaly detection and localization. In: ICPR’21. pp. 475–489. Springer (2021)

    work page 2021

  11. [11]

    A Survey on In-context Learning

    Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Chang, B., et al.: A survey on in-context learning. In: ELNLP’24 - arXiv:2301.00234. pp. 1107–1128 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  12. [12]

    Statistical science pp

    Efron, B., Tibshirani, R.: Bootstrap methods for standard errors, confidence in- tervals, and other measures of statistical accuracy. Statistical science pp. 54–75 (1986)

    work page 1986

  13. [13]

    Gemini: A Family of Highly Capable Multimodal Models

    Gemini Team: Gemini: A family of highly capable multimodal models. arXiv:2312.11805 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  14. [14]

    Foundations and Trends in Machine Learning5(2–3), 123–286 (2012)

    Kulesza, A., Taskar, B., et al.: Determinantal point processes for machine learning. Foundations and Trends in Machine Learning5(2–3), 123–286 (2012)

    work page 2012

  15. [15]

    Medical image analysis42, 60–88 (2017)

    Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., Van Der Laak, J.A., Van Ginneken, B., Sánchez, C.I.: A survey on deep learning in medical image analysis. Medical image analysis42, 60–88 (2017)

    work page 2017

  16. [16]

    Psychometrika12, 153–157 (1947)

    McNemar, Q.: Note on the sampling error of the difference between correlated proportions. Psychometrika12, 153–157 (1947)

    work page 1947

  17. [17]

    In: MICCAI 2024

    Naval Marimont, S., Siomos, V., Baugh, M., Tzelepis, C., Kainz, B., Tarroni, G.: Ensembled cold-diffusion restorations for unsupervised anomaly detection. In: MICCAI 2024. vol. LNCS 15011, pp. 243–253. Springer (2024)

    work page 2024

  18. [18]

    Scientific Data9(1), 429 (2022)

    Nguyen, H.Q., Lam, K., Le, L.T., Pham, H.H., Tran, D.Q., Nguyen, D.B., Le, D.D., Pham, C.M., Tong, H.T., Dinh, D.H., et al.: VinDr-CXR: An open dataset of chest x-rays with radiologist’s annotations. Scientific Data9(1), 429 (2022)

    work page 2022

  19. [19]

    OpenAI Technical Report (2024)

    OpenAI: GPT-4o: Multimodal intelligence at scale. OpenAI Technical Report (2024)

    work page 2024

  20. [20]

    DINOv2: Learning Robust Visual Features without Supervision

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: Dinov2: Learning robust visual features without supervision. TMLR - arXiv:2304.07193 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  21. [21]

    Foundations and Trends®in Machine Learning11(5-6), 355–607 (2019)

    Peyré, G., Cuturi, M., et al.: Computational optimal transport: With applications to data science. Foundations and Trends®in Machine Learning11(5-6), 355–607 (2019)

    work page 2019

  22. [22]

    In: International conference on machine learning

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. ICML - PmLR (2021)

    work page 2021

  23. [23]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Roth, K., Pemula, L., Zepeda, J., Schölkopf, B., Brox, T., Gehler, P.: Towards total recall in industrial anomaly detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 14318–14328 (2022)

    work page 2022

  24. [24]

    Medical Image Analysis54, 30–44 (2019)

    Schlegl, T., Seeböck, P., Waldstein, S.M., Langs, G., Schmidt-Erfurth, U.: f- AnoGAN: Fast unsupervised anomaly detection with generative adversarial net- works. Medical Image Analysis54, 30–44 (2019)

    work page 2019

  25. [25]

    In: ECCV

    Schlüter, H.M., Tan, J., Hou, B., Kainz, B.: Natural synthetic anomalies for self- supervised anomaly detection and localization. In: ECCV. pp. 474–489. Springer (2022)

    work page 2022

  26. [26]

    DINOv3

    Siméoni,O.,Vo,H.V.,Seitzer,M.,Baldassarre,F.,Oquab,M.,etal.:DINOv3:Self- supervised learning with gram anchoring. arXiv preprint arXiv:2508.10104 (2025) WALDO 11

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  27. [27]

    De- tecting outliers with foreign patch interpolation.arXiv preprint arXiv:2011.04197, 2020

    Tan, J., Hou, B., Batten, J., Qiu, H., Kainz, B., et al.: Detecting outliers with foreign patch interpolation. Machine Learning for Biomedical Imaging - arXiv:2011.041971(April 2022 issue), 1–27 (2022)

    work page arXiv 2011

  28. [28]

    Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution

    Wang, P., Bai, S., et al.: Qwen2-VL: Enhancing vision-language model’s perception at any resolution. arXiv:2409.12191 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  29. [29]

    Self-Consistency Improves Chain of Thought Reasoning in Language Models

    Wang, X., Wei, J., Schuurmans, D., et al.: Self-consistency improves chain of thought reasoning. In: ICLR - arXiv:2203.11171 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  30. [30]

    NeurIPS’2235, 24824–24837 (2022)

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D., et al.: Chain-of-thought prompting elicits reasoning in large language models. NeurIPS’2235, 24824–24837 (2022)

    work page 2022

  31. [31]

    box_2d": [y1, x1, y2, x2],

    Zimmerer, D., Full, P.M., Isensee, F., Jäger, P., Adler, T., Petersen, J., Köhler, G., Ross, T., Reinke, A., Kascenas, A., et al.: Mood 2020: A public benchmark for out- of-distribution detection and localization on medical images. IEEE transactions on medical imaging41(10), 2728–2738 (2022) 12 B. Kainz et al. Supplementary Material Figures 4, 5, and 6 sh...

    work page 2020