Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

Raptor: Scalable Train-Free Embeddings for 3D Medical Volumes Leveraging Pretrained 2D Foundation Models

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Raptor shows that a frozen 2D vision model, applied to orthogonal slices and compressed by random projections, produces train-free embeddings that beat medical-volume pretrained models on ten benchmarks.

desk verdict Raptor is a genuinely train-free and useful 3D embedding recipe, but the headline regression wins look like an MLP-vs-linear-head artifact; the classification gains are real but thin. read the letter →

arxiv 2507.08254 v1 pith:LRBULARB submitted 2025-07-11 eess.IV cs.CVcs.LG

Ulzee An , Moonseong Jeong , Simon A. Lee , Aditya Gorla , Yuzhe Yang , Sriram Sankararaman This is my paper
classification eess.IVcs.CVcs.LG
keywords 3Dmedicalvolumestrain-freeembeddingsrandomprojectionsDINOv2volumetricclassificationregressiondimensionalityreductionfoundationmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that a frozen 2D image foundation model, never trained on medical data, can serve as a general-purpose feature extractor for 3D medical volumes if its slice-wise outputs are averaged and compressed properly. The proposed method, Raptor, applies DINOv2-L to every cross-section of a volume along three orthogonal axes, mean-pools the slice features per axis, and reduces each with random projections, producing an embedding of $3Kp^2$ numbers with no training. On ten classification and regression benchmarks the paper reports state-of-the-art scores, beating medical-volume pretrained models by 3–14% in accuracy while using embeddings up to ~99% smaller than the raw voxels and smaller than competing latents. A sympathetic reader would care because it suggests that expensive 3D pretraining may be avoidable for many volumetric tasks, widening access to high-dimensional medical analysis.

What carries the argument

The load-bearing object is the Raptor embedding $\Phi(x)=\operatorname{concat}_{i=1}^3 R\,\frac{1}{D}\sum_{j=1}^D \phi(s_{ij})$, where $\phi$ is the frozen DINOv2-L vision transformer (a transformer that encodes images as grids of patch tokens), $s_{ij}$ are slices in the axial, coronal, and sagittal directions, and $R\in\mathbb{R}^{K\times d}$ is a Gaussian random projection matrix. The mean-pooling aggregates the volume dimension while the random projection compresses the token dimension, and the Johnson–Lindenstrauss lemma supplies the pairwise-distance guarantee. The whole pipeline is linear after the encoder, which also makes the runtime $O(p^2 d N (D+K))$ and sub-cubic in volume size.

What would settle it

Measure Raptor's AUC on a dataset whose only signal is a small, localized abnormality (a 10–16px sphere or fracture line): the paper's size simulation predicts chance-level AUC ~0.5 at that scale, so a real clinical dataset where human readers reliably detect such lesions while Raptor stays at chance would settle whether the embeddings preserve small-feature information.

Watch

Extended reading notes

Core claim

The central claim is that distance structure in DINOv2-L slice embeddings survives both averaging and random projection, so the resulting volume descriptor preserves enough semantic and spatial information to outperform models pretrained on medical volumes. Concretely, for two volumes the Raptor distance $d_{\text{Raptor}}$ is shown to remain between a constant times the raw embedding distance and a constant times $\sqrt{D}\,d_{\text{raw}}$, provided slice-level embedding differences are aligned across slices ($\alpha_j>0$). The paper reports that this holds empirically on most of its ten tasks, where Raptor reaches the best AUROC/accuracy in six of nine classification datasets and the best $r^2$ in nine of ten regression regions, on embeddings of size $3\times 100\times 16\times 16$ (or ten projections for Raptor-B).

Load-bearing premise

The method assumes that averaging features over all slices in each of three directions does not cancel the signal distinguishing two volumes: the formal guarantee requires slice-level embedding differences to point in roughly the same direction ($\alpha_j>0$), and the paper's own results show this fails on Fracture3D's axial view and on empty slices, where averaged differences can vanish.

Editorial extensions

If this is right

  • A 256³ volume becomes a 76,800-dimensional vector (K=100) or 7,680-dimensional vector (K=10) with no training, which fits on-chip for downstream logistic regression or small MLPs.
  • Raptor-B at K=10 matches SuPreM's average classification accuracy while being 28.8× smaller in embedding size, so the compression itself does not explain the accuracy gap.
  • Because the encoder is frozen and model-agnostic, replacing DINOv2-L with a stronger 2D foundation model should transfer directly to volumes without re-architecting anything.
  • The ~6.5s per 256³ volume on a single consumer GPU, with no training run, makes large-scale volume analysis feasible in data-scarce laboratories.
  • The theory ties class separability in the original embedding space to an $\Omega(\beta)$ separation guarantee after compression, so Raptor's downstream behavior is not purely empirical.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • [Editorial] The same 'orthogonal slices + frozen 2D encoder + random projection' recipe should transfer to non-medical volumetric data, such as microscopy volumes or 3D objects, since nothing in the pipeline is modality-specific.
  • [Editorial] The size-detection simulation suggests a sharp spatial-resolution floor: features much smaller than a 16px ViT patch are likely to be averaged away, so tasks requiring detection of tiny lesions or thin connected structures would need patch-level or multi-scale pooling rather than whole-slice averaging.
  • [Editorial] A testable prediction is that Raptor embeddings with different random seeds at K=100 are nearly interchangeable (the paper reports standard deviation below 0.001 in AUC), implying downstream classifiers should be stable across seeds even when the underlying 2D encoder is changed.
  • [Editorial] The paper's own observation of negative $\alpha_j$ in Fracture3D predicts that datasets with abrupt slice-to-slice changes, such as volumes containing multiple disconnected organs, will degrade toward chance, and fixing this may require a robust aggregation other than mean-pooling.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. Raptor (Random Planar Tensor Reduction) is a train-free method for embedding 3D medical volumes. It applies a frozen 2D DINOv2-L encoder to all slices in three orthogonal axes, mean-pools the slice-level token embeddings per axis, compresses the resulting 3 x d x p^2 tensor with random projections to dimension K per patch, and flattens the result into a 3Kp^2 vector. The authors evaluate Raptor on six 3D MedMNIST classification tasks, three additional classification datasets (CC-CCII, CTRG-C, CTRG-B), and a UK Biobank brain MRI regression benchmark with ten groups of imaging-derived phenotypes. They compare against 3D ResNet, MAE-finetuned ViT, and several medical-volume pretraining methods (SuPreM, MISFM, Merlin, VoCo, SLIViT), reporting that Raptor achieves the highest average classification and regression scores while using embeddings far smaller than raw volumes and smaller than most competing latent spaces. The paper also provides ablation studies on the number of random projections, the number of viewpoints, and a controlled simulation of spatial resolution limits, plus a formal analysis of distance preservation in Appendices A.5 and A.6.

Significance. The core idea is appealing and practically valuable: a trained 3D model can be bypassed entirely by reusing a frozen 2D foundation model plus random projections, which is genuinely train-free for embedding generation and scales sub-cubically with volume size. The classification results are credible and broadly consistent with the claim that Raptor is competitive with or better than several medical-volume foundation models on the benchmarks tested. The paper is also honest in reporting the failure modes: the size-detection simulation shows AUC near 0.5 at 16 px, and the Appendix A.6 analysis explicitly notes the alpha_j > 0 assumption is violated in Fracture3D. If the regression comparison is made head-matched and the reported aggregate numbers are corrected, the contribution would be a solid, reproducible baseline for resource-constrained volumetric analysis. The main weakness is that the paper's headline claims overstate the measured margins and rest in part on an unfair predictor-class comparison for the regression benchmark.

major comments (4)
  1. [Section 4.1 and Table 5] The regression comparison is not head-matched: all baselines receive a single linear layer above their latent space, while Raptor embeddings are fed to an MLP tuned up to 3 layers (Appendix A.2). Since the UKBB regression benchmark produces the largest relative gains (+24% over Merlin, +30% over SuPreM), these margins may reflect predictor capacity rather than embedding quality. Please rerun the regression benchmark with a common predictor class (for example, ridge regression or the same MLP architecture) for all embeddings, and report both configurations, or restrict the regression claims accordingly.
  2. [Abstract, Section 4.2, Section 6] The numerical claims are inconsistent across the paper. The abstract states +3% over SuPreM and +6% over MISFM, while Section 4.2 reports average classification improvements of +2% over SuPreM and +4% over MISFM, and the Discussion states a '3% accuracy gain over the next best approach (SuPreM)'. Please specify whether these are AUROC or accuracy, whether they include all ten tasks or only the classification tasks, and reconcile the numbers in all three places.
  3. [Appendix A.6 and Figure A.3] The lower bound d_Raptor >= (1-epsilon)/D * alpha_min * d_raw requires alpha_j > 0 for every slice, but the paper itself reports negative alpha_j in the axial view of Fracture3D and Figure A.3 shows peaks at alpha = 0 due to empty slices. Under these violations the lower bound does not hold, so the claim that 'cluster centers remain Omega(beta)-separated, thereby guaranteeing class separability' is not established for those datasets. Please state the guarantee as explicitly conditional and report the alpha_j diagnostics for all benchmark datasets, not only a subset.
  4. [Tables 3, 4, and 5] The main results report only a single split per dataset without repeated runs or confidence intervals. The headline margins over SuPreM are small (about 1-2 AUROC points on average) and Raptor does not win on every dataset, so the 'superior performance' claim would be more convincing with variance estimates, at least for the closest competitor and on the datasets where the margins are within a few points.
minor comments (4)
  1. [Appendix A.6] The notation for the partial sum is confusing: S_j is introduced as S_j = sum_{k=1}^j Delta_k but written as 'Pk=j k=1'; please clarify the indexing and align the alpha_j assumption with the induction step.
  2. [Table A.5] The class percentages for CTRG-C sum to more than 100% and the dataset is described as multi-label, while the paper reports AUROC and ACC in the MedMNIST style; please clarify how accuracy is computed for multi-label tasks.
  3. [Table 1] The method name is typeset inconsistently as 'V oCo' with an extra space in several places; please fix the typography.
  4. [Section 5.3] The size-detection simulation shows AUC ~0.5 at 16px, which is an honest limitation, but the abstract and Discussion do not mention this resolution limit; consider adding one sentence to the Discussion to balance the scalability claims.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: Raptor's frozen-encoder, random-projection pipeline is train-free and results are evaluated on held-out test splits; only a non-load-bearing self-citation (SLIViT baseline) appears.

full rationale

Raptor's embedding is computed entirely from a frozen DINOv2-L encoder applied to slices in three orthogonal planes, mean-pooling over slices, and a random Gaussian projection matrix with entries drawn i.i.d. from N(0,1); none of these steps fit parameters to the benchmark labels. The downstream logistic-regression and MLP heads are trained on train/validation splits and evaluated on reported test splits, so the headline accuracies and r2 values are genuine held-out predictions rather than fitted inputs renamed as predictions. The Appendix A.6 distance-preservation result is an explicit conditional analysis: it assumes alpha_j > 0 alignment of slice-level embedding differences and, by the paper's own admission, this assumption is violated in the axial view of Fracture3D and by empty slices (Figure A.3), so the bound is not used to manufacture the empirical result. The only self-citation of note is the SLIViT baseline (Avram et al., 2024), which includes co-authors of the present paper; it is used as a comparison method and not as the load-bearing justification for Raptor's design, which instead rests on the external DINOv2 model (Oquab et al., 2023) and standard JL-lemma theory. The unequal downstream head class for regression (tuned MLP for Raptor versus a single linear layer for baselines) is a legitimate experimental-fairness concern, but it is not circularity, because the MLP is fit only on training labels and scored on held-out test data. No step in the derivation reduces by construction to its own inputs.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

Raptor introduces no new physical or mathematical entities. It relies on standard random projection theory, the pretrained DINOv2-L encoder, and a domain-specific smoothness assumption about slice-level embeddings. The key load-bearing assumptions are the alignment of slice differences and the representational sufficiency of DINOv2 features for medical slices.

free parameters (4)
  • K: number of random projections = 100 (Raptor), 10 (Raptor-B)
    Chosen by hand based on ablations in Section 5.1; affects embedding size and accuracy.
  • L2 regularization penalty (logistic regression) = Tuned over {0.01, 0.1, 1.0, 10.0, 100.0}
    Standard downstream hyperparameter selected on validation sets (Section 4.1).
  • MLP depth and width (regression) = Up to 3 layers, 256 hidden units (Appendix A.2)
    Selected on validation sets; creates an asymmetry versus the linear head used for baselines.
  • Random projection seed = Not reported for main results; three seeds in Section 5.1
    The random matrix R is sampled once, but the specific seed is not given for the headline tables, so the reported numbers are not tied to a reproducible realization.
assumptions (5)
  • standard math Johnson-Lindenstrauss lemma: random projections approximately preserve pairwise distances with high probability for K = O(eps^-2 log n)
    Invoked in Section 3.3 and proved in Appendix A.5.
  • domain assumption DINOv2-L features are semantically meaningful for medical slice images
    Required for the whole approach; empirically supported by the benchmark results and the variance analysis in Appendix A.3.
  • ad hoc to paper Slice-level DINOv2 embedding differences between volumes are smoothly aligned (alpha_j > 0)
    This is the core assumption of the distance-preservation bound in Appendix A.6 (equation involving Delta_j^T S_{j-1} >= alpha_j). The paper itself shows this fails for some datasets.
  • domain assumption DINOv2's pretraining data does not contain the benchmark test sets
    The paper argues web-crawled images are unlikely to include Medical MNIST or the other benchmarks (Section 3.2), but this cannot be fully verified.
  • domain assumption A linear or shallow MLP readout can exploit the information in the embeddings
    The paper evaluates Raptor with logistic regression or a 3-layer MLP; the claim of state-of-the-art is contingent on this readout protocol (Sections 4.1-4.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Raptor: Scalable Train-Free Embeddings for 3D Medical Volumes Leveraging Pretrained 2D Foundation Models." pith.science (2026). https://pith.science/paper/LRBULARB

@misc{pith2026250708254,
  author       = {Pith},
  title        = {Pith review of: Raptor: Scalable Train-Free Embeddings for 3D Medical Volumes Leveraging Pretrained 2D Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LRBULARB}},
  note         = {Machine review of arXiv:2507.08254}
}
read the original abstract

Current challenges in developing foundational models for volumetric imaging data, such as magnetic resonance imaging (MRI), stem from the computational complexity of training state-of-the-art architectures in high dimensions and curating sufficiently large datasets of volumes. To address these challenges, we introduce Raptor (Random Planar Tensor Reduction), a train-free method for generating semantically rich embeddings for volumetric data. Raptor leverages a frozen 2D foundation model, pretrained on natural images, to extract visual tokens from individual cross-sections of medical volumes. These tokens are then spatially compressed using random projections, significantly reducing computational complexity while retaining semantic information. Extensive experiments on ten diverse medical volume tasks verify the superior performance of Raptor over state-of-the-art methods, including those pretrained exclusively on medical volumes (+3% SuPreM, +6% MISFM, +10% Merlin, +13% VoCo, and +14% SLIViT), while entirely bypassing the need for costly training. Our results highlight the effectiveness and versatility of Raptor as a foundation for advancing deep learning-based methods for medical volumes.

Figures

Figures reproduced from arXiv: 2507.08254 by the authors.

Figure 1
Figure 1. Comparisons between Raptor and state-of-the-art methods on diverse medical volume tasks. Raptor achieves superior performance across classification (AUROC↑) and regres￾sion (mean r 2 ↑, indicated with +) tasks while remaining entirely train-free. human-computer interaction (Jiang et al., 2023), and au￾tonomous systems (Huang et al., 2025). However, learning from volumetric modalities, represented as 3-dimensional vo… view at source ↗
Figure 2
Figure 2. Flowchart visualizing the computation of Raptor embeddings from a medical volume. Raptor leverages a pretrained 2D image foundational model to encode the semantics of the 3D volumes in three different axial views. The resulting high-dimensional tensor is then low-rank approximated with random projections and aggregated. We obtain the final Raptor embedding by flattening the projections of the volumes. There is no tr… view at source ↗
Figure 3
Figure 3. Triangulation of a feature from a lower dimension. Visualization of the intuition behind triaxial sampling. Features reduced to orthogonal lower dimensions can be used to triangulate features in the original space (e.g. in 2D or 3D). The quantities in typical settings are D = 256, d = 1024, T = 16, and p = 256/16 = 16 (using DINOv2-L), resulting in Raptor embeddings of size 768 × K based on the choice of K. We infer… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Prediction accuracy with limited training data. We measure the effect of limiting the number of training samples between 10 ∼ 500 on the final test accuracy on the Synapse dataset. Shaded area represents 95% CI. (0.729) of the performance achieved with the full 1, 230 …
Figure 6
Figure 6. Figure 6: Overall accuracy of each method in the context of their embedding sizes. Average performance of each method on the classification and regression datasets according to the size of the latent space inferred by each method (single precision floats). substantial space savi…
Figure 7
Figure 7. Figure 7: Examples of the “location” and “size” detection simu￾lated tasks. The “location” task asks whether a digit appears in an expected location in a volume. The “size” task asks whether a digit of varying sizes is present in a volume [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Training-Free Zero-Shot Anomaly Detection in 3D Brain MRI with 2D Foundation Models

    cs.CV 2026-02 conditional novelty 6.0 of 10

    CoDeGraph3D aggregates multi-axis 2D DINOv2 features into 3D patch tokens and uses batch-level rarity scoring to detect and segment brain MRI anomalies with no training.

  2. MorphologyFM: A Foundation Model for Morphology-Aware Representation Learning from ECG and Pulse Oximetry Waveforms

    eess.SP 2026-07 conditional novelty 5.0 of 10

    Morphology-aware masking plus cross-modal ECG–SpO2 pretraining on MIMIC yields stronger transfer than MAE, contrastive, Barlow Twins, and JEPA on several clinical prediction tasks.

Reference graph

Works this paper leans on

13 extracted references · 4 canonical work pages · cited by 2 Pith papers

  1. [3]

    Ebrahimi, A., Luo, S., and Chiong, R

    URL https:// openreview.net/forum?id=YicbFdNTTy. Ebrahimi, A., Luo, S., and Chiong, R. Introducing transfer learning to 3d resnet-18 for alzheimer’s disease detection on mri images. In 2020 35th international conference on image and vision computing New Zealand (IVCNZ), pp. 1–6. IEEE,

    work page 2020

  2. [6]

    Llava-med: Training a large language-and-vision assistant for biomedicine in one day

    Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., and Gao, J. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. arXiv preprint arXiv:2306.00890,

    arXiv

  3. [7]

    G., Blazes, M., Wu, Y ., Lee, C

    Liu, Z., Xu, H., Woicik, A., Shapiro, L. G., Blazes, M., Wu, Y ., Lee, C. S., Lee, A. Y ., and Wang, S. Octcube: a 3d foundation model for optical coherence tomogra- phy that improves cross-dataset, cross-disease, cross- device and cross-modality analysis. arXiv preprint arXiv:2408.11227, 2024b. Ma, J., He, Y ., Li, F., Han, L., You, C., and Wang, B. Seg-...

    arXiv

  4. [8]

    V-net: Fully con- volutional neural networks for volumetric medical image segmentation

    Milletari, F., Navab, N., and Ahmadi, S.-A. V-net: Fully con- volutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV), pp. 565–571. Ieee,

    work page 2016

  5. [9]

    URL https://arxiv.org/abs/2307. 15189. arXiv:2307.15189. Ning, J., Zhao, H., Lan, L., Sun, P., and Feng, Y . A computer-aided detection system for the detection of lung nodules based on 3d-resnet. Applied Sciences, 9(24): 5544,

    arXiv

  6. [12]

    Large-scale 3d medical image pre-training with geometric context priors

    Wu, L., Zhuang, J., and Chen, H. Large-scale 3d medical image pre-training with geometric context priors. arXiv preprint arXiv:2410.09890, 2024a. Wu, L., Zhuang, J., and Chen, H. V oCo: A Simple-yet-Effective V olume Contrastive Learning Framework for 3D Medical Image Analysis, April 2024b. URL http://arxiv.org/abs/2402. 17300. arXiv:2402.17300 [eess]. Xi...

    arXiv

  7. [13]

    Appendix A.1

    12 Random Planar Tensor Reduction A. Appendix A.1. Using alternate image foundation models in Raptor Table A.1. Effectiveness of other 2D ViT encoders METHOD ENCODER AUC ACC RAPTOR SAM 0.850 0.800 CLIP 0.870 0.813 LLAVA-MED 0.869 0.812 MEDSAM 0.872 0.808 DINO V2-L 0.907 0.835 As our approach is agnostic to the choice of the image foundation model, we expl...

    work page 2024

  8. [2019]

    Dinov2: Learning robust visual features without supervision

    Oquab, M., Darcet, T., Moutakanni, T., V o, H., Szafraniec, M., Khalidov, V ., Fernandez, P., Haziza, D., Massa, F., El- Nouby, A., et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193,

    arXiv

Show all 13 references
  1. [2020]

    Kingma, D. P. and Welling, M. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings,

    2014

  2. [2021]

    X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al

    Chang, A. X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012,

    arXiv

  3. [2022]

    Mis-fm: 3d medical image segmentation using foun- dation models pretrained on a large-scale unannotated dataset

    Wang, G., Wu, J., Luo, X., Liu, X., Li, K., and Zhang, S. Mis-fm: 3d medical image segmentation using foun- dation models pretrained on a large-scale unannotated dataset. arXiv preprint arXiv:2306.16925,

    arXiv

  4. [2023]

    Lai, H., Jiang, Z., Yao, Q., Wang, R., He, Z., Tao, X., Wei, W., Lv, W., and Zhou, S. K. E3d-gpt: Enhanced 3d visual foundation for medical vision-language model. arXiv preprint arXiv:2410.14200,

    arXiv

  5. [2024]

    URL http://arxiv.org/abs/2406. 06512. arXiv:2406.06512 [cs]. Bycroft, C., Freeman, C., Petkova, D., Band, G., Elliott, L. T., Sharp, K., Motyer, A., Vukcevic, D., Delaneau, O., O’Connell, J., et al. The uk biobank resource with deep phenotyping and genomic data. Nature, 562(77...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.