Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Training-Free Zero-Shot Anomaly Detection in 3D Brain MRI with 2D Foundation Models

T0 review · 3 major / 6 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A fully training-free pipeline detects anomalies in 3D brain MRI by pooling multi-axis 2D foundation-model features into cubic tokens and scoring those tokens by rarity across a test batch.

desk verdict Sensible tokenization bridge to 3D ZSAD, but the patient-level AUROC is probably dataset identity; the segmentation results are more convincing. read the letter →

arxiv 2602.15315 v1 pith:BFWJYYWT submitted 2026-02-17 cs.CV stat.ML

Tai Le-Gia , Jaehyun Ahn This is my paper
classification cs.CVstat.ML
keywords magneticresonanceimagingzero-shotanomalydetection3DbrainMRIvolumetrictokenizationfoundationmodelsbatch-basedsegmentationmetastasis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Zero-shot anomaly detection usually stops at 2D. This paper shows that a batch of 3D brain MRI volumes can be turned into compact collections of volumetric tokens entirely without training: a frozen 2D vision transformer processes every slice along three anatomical axes, slice features are averaged into roughly 10-millimeter cubic patches, and a distance-based mutual-similarity score flags tokens that have no close match in the rest of the batch. On BraTS-2025 metastases against healthy IXI volumes, the pipeline reports patient-level AUROC of 96.9% and voxel Dice of 41.3% on T2-weighted scans, outperforming CLIP-based zero-shot baselines and approaching supervised references. The significance is that volumetric anomaly detection need not depend on reconstructing normal anatomy or on text prompts.

What carries the argument

Central object is the multi-axis 3D patch token. For each of the three anatomical planes, a frozen 2D vision transformer (DINOv2-L/14) extracts slice features; patch-aligned average pooling over p consecutive slices collapses the depth dimension and yields one feature per p×p×p voxel cube; a fixed Gaussian random projection based on the Johnson–Lindenstrauss lemma reduces each axis's features to 128 dimensions; the three axes are concatenated into a single 384-dimensional token per cube. These tokens form the collections on which the mutual similarity vector and rarity scoring operate, exploiting the Doppelgänger assumption that normal tokens have multiple close matches elsewhere in the batc

What would settle it

Register a batch in which all patients share the same implanted tumor mimics at the same atlas coordinates and run the pipeline: if the consistent-anomaly handling works, those lesions should be excluded from mutual matching and still detected; if they instead score as normal, the rarity machinery is short-circuited. Alternatively, a cohort enriched with sub-100 mm³ metastases that are clearly visible in native MRI should reproduce the reported true-positive rate of about 0.23.

Watch

Extended reading notes

Core claim

The paper's central claim is that the Doppelgänger principle—normal anatomical patches repeat across subjects, anomalies do not—remains valid at the token level in 3D if tokens are constructed properly. The authors construct each 3D token by aggregating 2D DINOv2 features over corresponding blocks of axial, coronal, and sagittal slices, concatenating the three views after a random projection. These localized cubic tokens restore the 3D context that slice-wise pipelines lose, and feeding them to batch-based scoring yields volumetric anomaly maps that localize lesions without any medical training. The authors report that on T2-weighted brain MRI the training-free method reaches 96.9% patient-l

Load-bearing premise

The load-bearing premise is that after atlas registration every normal anatomical region in a test volume has near-duplicate token representations somewhere in the test batch, while the anomaly is rare enough that its tokens remain mutual outliers; if a patient's normal anatomy deviates strongly from the batch, that healthy tissue will be scored as anomalous.

Editorial extensions

If this is right

  • Training-free volumetric anomaly detection becomes practical on a single consumer GPU: roughly 4 seconds per volume and under 10 GB VRAM for 180 volumes.
  • No text prompts or medical fine-tuning are needed, so the method transfers to new anomaly types (glioma, stroke) and modalities (T1w, T2w) with no adaptation.
  • Aggressive dimensionality reduction to 128 dimensions preserves performance, so large cohorts can be scored while keeping quadratic mutual-similarity computation tractable.
  • The method degrades gracefully with batch size; even 15 volumes in a batch gives non-trivial segmentation, implying usefulness in small or sequential test sets.
  • Voxel-level segmentation from zero-shot tokens can reach Dice scores comparable to supervised CLIP baselines, making it a reference for unsupervised volumetric detection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same cubic-token recipe should work for other 3D medical modalities (e.g., CT or MRA) whenever a reasonable atlas registration and background mask exist, since the pipeline contains no brain-specific learned component.
  • Editorial inference: replacing the single-resolution pooling with multi-scale cubes (for example p and 2p) could raise the reported 23% true-positive rate for sub-100 mm³ lesions, a testable extension the authors mention as future work.
  • Editorial inference: the near-parity of two-axis (axial plus coronal) and three-axis fusion suggests that a two-thirds reduction in slicing cost may be possible with only a small Dice loss; this could be validated by measuring runtime and accuracy on a held-out cohort.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper proposes CoDeGraph3D, a training-free batch-based zero-shot anomaly detection method for 3D brain MRI. It tokenizes volumes into cubic 3D patches by pooling DINOv2 features extracted from axial, coronal, and sagittal slices in 14-slice blocks, applies a fixed random projection to 128 dimensions, concatenates the three views, and feeds the resulting token collections to the MuSc/CoDeGraph mutual-similarity pipeline. Experiments on mixed batches of 115 healthy IXI volumes and 65 pathological volumes (BraTS-2025 METS T1w/T2w; also BraTS-2021 GLI and ATLAS R2.0) report patient-level AUROC up to 96.9% (T2w) and voxel Dice 41.3%, outperforming CLIP-based zero-shot baselines and roughly matching supervised references. The paper also reports ablations on projection dimension, viewpoint combinations, batch size, and backbone.

Significance. If the empirical claims withstand the domain-shift confound, the paper would be a useful contribution: it demonstrates that a fully training-free, frozen-2D-encoder pipeline can produce volumetric anomaly scores at about 4 seconds per volume on a single GPU, with no prompts or fine-tuning. The multi-axis tokenization is simple and the ablations (Tables 3, 4, 5, 7) are informative. Strengths include the breadth of anomaly types (metastasis, glioma, stroke), explicit limitations with LTPR, and a formal patch-averaging lemma in Appendix D. However, the headline patient-level AUROC currently cannot be distinguished from dataset-separation performance, and the segmentation evaluation would benefit from non-oracle thresholds and error bars.

major comments (3)
  1. [§4.1, Tables 1–2, 6] The AC evaluation is confounded by dataset identity. The test batch is 115 IXI normal volumes plus 65 BraTS/ATLAS pathological volumes; the two populations come from different acquisition domains. A batch method that scores tokens by rarity can attain high patient AUROC by separating scanner/protocol differences rather than pathology. The DAE baseline in Table 1 illustrates exactly this: 99.8% patient AUROC on T2w with only 25.0% Dice. CoDeGraph3D's 96.9% is near that ceiling. Please add a control where healthy volumes from a second dataset/scanner are added to the batch (or, equivalently, split the healthy cohort by site) to show that domain identity is not the main driver, and report whether top-scoring voxels are lesion voxels. Without this, the zero-shot anomaly detection claim is overstated.
  2. [§4.1 Evaluation Metrics] Dice-max is an oracle-threshold metric: the threshold is chosen after seeing ground truth. It therefore reports an upper bound, not a deployable operating point. The main tables should also give a fixed-threshold Dice (e.g., Otsu or a predetermined percentile) or emphasize the voxel AP columns. The main quantitative tables also lack error bars; Table 3 shows both AUROC and Dice vary with the random projection seed (±0.3 in Dice at k=128), so single-run headline numbers in Tables 1–2 and 6 do not establish robustness.
  3. [§3.1 and §4.1] The MSV hyperparameter K (number of nearest-neighbor distances averaged) is not reported or ablated. The text states only that CoDeGraph is run "using its default configuration." Since MuSc/CoDeGraph performance is known to depend on K, and the batch contains only 65 abnormal volumes, the reported Dice/AUROC may be sensitive to this choice. Please state the value of K and provide a sensitivity analysis analogous to Table 3.
minor comments (6)
  1. [Table 4] The abbreviation 'P-AUC' is used in the table but not defined; the text uses 'Patient-level AUROC'. Please standardize.
  2. [Table 6] The ATLAS R2.0 experiment uses IXI volumes registered to MNI152 instead of SRI-24, while all other experiments use SRI-24. This preprocessing difference should be flagged in the table caption or the main text, as it may affect comparability.
  3. [§4.1 Preprocessing] The description of histogram standardization ('Nyúl et al. 2000 using a fixed reference template') does not specify which reference template was used. Please add details for reproducibility.
  4. [Appendix C.1] The supervised CLIP baselines are trained only on tumor-containing slices (at most ten per axis). This selection bias is discussed only in the appendix; it should be acknowledged in the main text when these numbers are quoted as supervised references.
  5. [§4.2 LTPR] The lesion-wise true positive rate analysis is valuable, but the reported LTPR values (0.23 for <100 mm³, 0.83 for >1000 mm³) lack confidence intervals. Given the small lesion counts, please add intervals or a per-lesion scatter plot.
  6. [§2 and §4] RAPTOR is cited as a related train-free 3D embedding method but is not compared experimentally. A sentence explaining why it is not a suitable baseline (e.g., volume-level rather than local anomaly scoring) would be helpful.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the 3D tokenization is a forward, parameter-free pipeline and is validated against external baselines and held-out anomaly types; only a minor test-set hyperparameter choice and a non-load-bearing self-citation prevent a 0.

full rationale

The claimed derivation chain is not circular. Eqs. (1)-(3) define a fixed feature-extraction operation (multi-axis slice encoding, patch-aligned averaging, JL random projection) with no parameters fitted to the anomaly labels, and the batch-level MSV/CoDeGraph scoring is a generic algorithm from prior work (Li et al., 2024; Gia and Ahn, 2025). The paper's self-citation to CoDeGraph is not load-bearing in a circular sense: CoDeGraph3D is evaluated here against external CLIP-based zero-shot baselines, a supervised DAE, and on independent anomaly types (BraTS-2021 GLI, ATLAS R2.0), so the central result does not reduce to the authors' prior claims. The only near-circular concern is that k=128 and the four-layer averaging are selected from ablations run on the same T2w evaluation batch (Section 5, Table 3), making the headline Dice mildly optimistic; however, Table 3 shows performance is stable for k=50-200 and Table 6 transfers without retuning, so this is test-set hyperparameter selection rather than a construction-level circularity. The IXI-vs-BraTS domain difference could inflate patient-level AUROC, but that is an external-validity/correctness caveat, not a definitional reduction, and the voxel Dice and cross-dataset results provide independent supporting evidence.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The 3D patch token is a constructed feature representation, not a postulated entity. The ledger records the fitted/selected parameters and the domain assumptions that the central claim rests on.

free parameters (3)
  • random_projection_dimension k = 128
    Chosen from ablation on the BraTS-2025 T2w test set (Table 3) to balance Dice and compute; k=50 already stabilizes, k=128 selected for final model.
  • feature_layers (DINOv2) = 4 layers: 6,12,18,24
    Selected for the final model; not ablated in the paper. Averaging over these layers is a design choice, not derived.
  • MuSc/CoDeGraph K (nearest-neighbor count) = default (not stated)
    Inherited from Gia and Ahn (2025) via 'default configuration'; affects MSV averaging and consistent-anomaly exclusion, but not specified in this paper.
assumptions (5)
  • domain assumption Doppelgänger assumption: normal structures recur across samples, anomalies are rare and distinctive
    Central premise of batch-based ZSAD; stated in Section 3.1. Underlies the mutual-similarity scoring.
  • domain assumption DINOv2 features trained on natural images transfer to brain MRI slices
    The method uses frozen DINOv2-L/14; the paper validates empirically (Table 7 with CLIP backbone) but does not prove the transfer.
  • domain assumption Atlas registration to SRI-24/MNI152 sufficiently aligns brains so that corresponding tokens represent similar anatomy
    Preprocessing in Section 4.1; if registration fails, normal tokens would not find matches, causing false positives.
  • standard math Johnson–Lindenstrauss lemma: Gaussian random projection preserves pairwise distances up to (1±ϑ) with high probability
    Used in Section 3.2.2 to justify projecting token features to k=128 dimensions.
  • ad hoc to paper Patch-averaging sensitivity conditions of Lemma 1 (Appendix D): average feature difference over anomalous slices ≥ Δ0, and non-anomalous differences ≤ ε
    Lemma 1 assumes these conditions to show token-level separability; they are not empirically verified for DINOv2 features or brain MRI.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Training-Free Zero-Shot Anomaly Detection in 3D Brain MRI with 2D Foundation Models." pith.science (2026). https://pith.science/paper/BFWJYYWT

@misc{pith2026260215315,
  author       = {Pith},
  title        = {Pith review of: Training-Free Zero-Shot Anomaly Detection in 3D Brain MRI with 2D Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BFWJYYWT}},
  note         = {Machine review of arXiv:2602.15315}
}
read the original abstract

Zero-shot anomaly detection (ZSAD) has gained increasing attention in medical imaging as a way to identify abnormalities without task-specific supervision, but most advances remain limited to 2D datasets. Extending ZSAD to 3D medical images has proven challenging, with existing methods relying on slice-wise features and vision-language models, which fail to capture volumetric structure. In this paper, we introduce a fully training-free framework for ZSAD in 3D brain MRI that constructs localized volumetric tokens by aggregating multi-axis slices processed by 2D foundation models. These 3D patch tokens restore cubic spatial context and integrate directly with distance-based, batch-level anomaly detection pipelines. The framework provides compact 3D representations that are practical to compute on standard GPUs and require no fine-tuning, prompts, or supervision. Our results show that training-free, batch-based ZSAD can be effectively extended from 2D encoders to full 3D MRI volumes, offering a simple and robust approach for volumetric anomaly detection.

Figures

Figures reproduced from arXiv: 2602.15315 by the authors.

Figure 1
Figure 1. Anomaly segmentation on 3D T2-weighted BraTS scans. Yellow contours show ground-truth boundaries, and blue regions denote predicted anomaly masks. Rows illustrate high-, medium-, and low-Dice cases for CoDeGraph3D and APRIL-GAN baselines. 5. Ablation Studies We quantitatively verify our tokenization strategy by testing it against alternative ap￾proaches. For all ablations, we report the results on the T2w modality o… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MorphologyFM: A Foundation Model for Morphology-Aware Representation Learning from ECG and Pulse Oximetry Waveforms

    eess.SP 2026-07 conditional novelty 5.0 of 10

    Morphology-aware masking plus cross-modal ECG–SpO2 pretraining on MIMIC yields stronger transfer than MAE, contrastive, Barlow Twins, and JEPA on several clinical prediction tasks.

Reference graph

Works this paper leans on

13 extracted references · 4 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Raptor: Scalable train-free embeddings for 3d medical volumes lever- aging pretrained 2d foundation models.arXiv preprint arXiv:2507.08254,

    Ulzee An, Moonseong Jeong, Simon A Lee, Aditya Gorla, Yuzhe Yang, and Sriram Sankararaman. Raptor: Scalable train-free embeddings for 3d medical volumes lever- aging pretrained 2d foundation models.arXiv preprint arXiv:2507.08254,

    arXiv

  2. [4]

    Unlike 2D batch-based ZSAD pipelines, where pairwise similarity computation with complexityO B2N 2k is often the primary bottleneck, our batch-based 3D formulation shifts most of the computational bur- den to the volumetric token construction step. This occurs because constructing volumetric tokens requires applying the 2D backbone to every slice along al...

    2023

  3. [8]

    URLhttps://doi.org/10.1007/ s10916-025-02272-2

    doi: 10.1007/s10916-025-02272-2. URLhttps://doi.org/10.1007/ s10916-025-02272-2. Bjoern H Menze, Andras Jakab, Stefan Bauer, Jayashree Kalpathy-Cramer, Keyvan Fara- hani, Justin Kirby, Yuliya Burren, Nicole Porz, Johannes Slotboom, Roland Wiest, et al. The multimodal brain tumor image segmentation benchmark (brats).IEEE transactions on medical imaging, 34...

    doi:10.1007/s10916-025-02272-2 1993

  4. [12]

    For WinCLIP, we strictly follow the official settings (Jeong et al., 2023), employing the ViT-B/16+ backbone with an input resolution of 240×240 and the standard prompt ensemble

    and APRIL-GAN (Chen et al., 2023). For WinCLIP, we strictly follow the official settings (Jeong et al., 2023), employing the ViT-B/16+ backbone with an input resolution of 240×240 and the standard prompt ensemble. For Anoma- lyCLIP and APRIL-GAN, we use a frozen ViT-L/14-336 CLIP encoder, pretrained by 16 Training-Free Zero-Shot 3D MRI Anomaly Detection T...

    2023

  5. [24]

    Patient-level anomaly classification results obtained by aggregating slice-wise CLS-token anomaly scores (max over slices)

    [6, 12, 18, 24] 17 Le-Gia Ahn Table 10:WinCLIP Patient-Level Performance with Slice-wise Aggregation. Patient-level anomaly classification results obtained by aggregating slice-wise CLS-token anomaly scores (max over slices). Dataset AUROC AP BraTS-2025 METS (T1w) 86.9 78.7 BraTS-2025 METS (T2w) 58.4 42.8 BraTS-2021 GLI (T2w) 75.4 68.8 ATLAS R2.0 (T1w) 62...

    2025

  6. [2000]

    Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193,

    Maxime Oquab, Timoth´ ee Darcet, Th´ eo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193,

    arXiv

  7. [2019]

    Kitamura, Sarthak Pati, Luciano M

    Ujjwal Baid, Satyam Ghodasara, Suyash Mohan, Michel Bilello, Evan Calabrese, Errol Co- lak, Keyvan Farahani, Jayashree Kalpathy-Cramer, Felipe C. Kitamura, Sarthak Pati, Luciano M. Prevedello, Jeffrey D. Rudie, Chiharu Sako, Russell T. Shinohara, Timo- thy Bergquist, Rong Chai, James Eddy, Julia Elliott, Walter Reade, Thomas Schaffter, Thomas Yu, Jiaxin Z...

    2021

  8. [2021]

    Spyridon Bakas, Hamed Akbari, Aristeidis Sotiras, Michel Bilello, Martin Rozycki, Justin S Kirby, John B Freymann, Keyvan Farahani, and Christos Davatzikos

    URLhttps://arxiv.org/abs/2107.02314. Spyridon Bakas, Hamed Akbari, Aristeidis Sotiras, Michel Bilello, Martin Rozycki, Justin S Kirby, John B Freymann, Keyvan Farahani, and Christos Davatzikos. Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features.Scientific data, 4(1):1–13,

    arXiv

Show all 13 references
  1. [2022]

    Analysis of the miccai brain tumor segmentation–metastases (brats-mets) 2025 lighthouse challenge: Brain metastasis segmentation on pre-and post-treatment mri

    Nazanin Maleki, Raisa Amiruddin, Ahmed W Moawad, Nikolay Yordanov, Athanasios Gkampenis, Pascal Fehringer, Fabian Umeh, Crystal Chukwurah, Fatima Memon, Bojan Petrovic, et al. Analysis of the miccai brain tumor segmentation–metastases (brats-mets) 2025 lighthouse challenge: Br...

    2025

  2. [2023]

    15 Le-Gia Ahn Appendix A. Using Alternate 2D ViT Encoders To demonstrate that CoDeGraph3D is adaptable to different foundation models, we replaced the default DINOv2 encoder with the CLIP visual encoder (ViT-L/14@336px) while keeping all other hyperparameters fixed. Table 7:Ef...

    2025

  3. [2024]

    Xuhai Chen, Yue Han, and Jiangning Zhang. A zero-/few-shot anomaly classification and segmentation method for cvpr 2023 vand workshop challenge tracks 1&2: 1st place on zero-shot ad and 4th place on few-shot ad.arXiv preprint arXiv:2305.17382,

    2023 arXiv

  4. [2025]

    Unsupervised brain lesion segmentation from mri using a convolutional au- toencoder

    Hans E Atlason, Askell Love, Sigurdur Sigurdsson, Vilmundur Gudnason, and Lotta M Ellingsen. Unsupervised brain lesion segmentation from mri using a convolutional au- toencoder. InMedical Imaging 2019: Image Processing, volume 10949, pages 372–378. SPIE,

    2019

  5. [2026]

    doi: https://doi.org/10

    ISSN 1361-8415. doi: https://doi.org/10. 1016/j.media.2025.103763. URLhttps://www.sciencedirect.com/science/article/ pii/S1361841525003093. Sook-Lei Liew, Bethany P Lo, Miranda R Donnelly, Artemis Zavaliangos-Petropulu, Jes- sica N Jeong, Giuseppe Barisano, Alexandre Hutton, J...

    2025

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.