Pith. sign in

REVIEW 4 major objections 7 minor 20 references

Rad-JEPA 3D claims that a compact 19M-parameter encoder, trained by predicting hidden features of masked CT cubes, captures the spatial structure that lets a 3–4B language model beat much larger 3D medical vision-language models on spatial

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 00:33 UTC pith:TPFPAP5H

load-bearing objection A well-ablated, plausible 3D CT JEPA assembly whose novelty is real but moderate, and whose headline 'structure substitutes for scale' claim is not yet backed by a controlled comparison. the 4 major comments →

arxiv 2607.26196 v1 pith:TPFPAP5H submitted 2026-07-28 cs.CV

Rad-JEPA 3D: Radiology Joint-Embedding Predictive Model for 3D Computed Tomography

Quoc-Huy Trinh , Minh-Van Nguyen , Ulas Bagci This is my paper
classification cs.CV
keywords self-supervised learning3D computed tomographyjoint-embedding predictive architectureMambagrouped-query attentionspatial reasoningmedical vision-language modelrepresentation learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Rad-JEPA 3D is a self-supervised framework that learns 3D CT representations by predicting, from a heavily masked view, the latent features of a complete scan. The authors' central claim is that this joint-embedding predictive objective, combined with a hybrid Mamba–GQA encoder and layer-wise orthogonal regularization (HSOR), induces coarse spatial structure that earlier contrastive and reconstruction-based encoders lack. As a result, a complete vision–language model with only 3–4B parameters reaches competitive or better results than 7.6B baselines on closed-ended VQA and achieves the best average score on the SpatialMed spatial-reasoning benchmark. The work argues that spatial organization in the encoder, not raw language-model scale, is what drives volumetric reasoning.

Core claim

Rad-JEPA 3D's discovery is that a student encoder, trained to predict an EMA teacher's latent features at masked cube positions (75% of the volume), learns volumetric representations that are unusually discriminative for organ-level and spatial tasks. The encoder tokenizes a 128×256×256 CT volume into 8×8×8 cubes and processes them with a hybrid H-Mamba block that routes each token to either a sequential Mamba-2 branch (inter-slice continuity) or a grouped-query attention branch (cross-plane context). Hidden States Orthogonal Regularization (HSOR) aligns student–teacher hidden states at every layer and penalizes weight correlation, reducing feature redundancy. The paper reports that the resu

What carries the argument

The load-bearing machinery is the H-Mamba encoder and the HSOR regularization. H-Mamba layers mix a Mamba-2 state-space branch (linear-time sequential modeling of inter-slice continuity) with a grouped-query attention branch (all-to-all cross-plane context via rotary-position-encoded grid coordinates), combined per token by a lightweight router with a load-balancing loss. HSOR drives the cross-correlation of layer-wise student–teacher hidden states toward the identity and applies a soft orthogonality penalty to encoder weight matrices, keeping intermediate representations consistent and less redundant. Together they turn the JEPA objective—predicting the latent features of masked cubes—into

Load-bearing premise

Every headline improvement assumes that all baseline models were evaluated under exactly the same protocol—same prompts, decoding settings, training data, and evaluation harness—as Rad-JEPA 3D; the supplementary that would document these details is not included in the paper.

What would settle it

A decisive test would be a head-to-head experiment that keeps the language model fixed (for example, Qwen2.5-7B-Instruct) and swaps only the visual encoder—Rad-JEPA 3D against DCFormer-SigLP and M3D-CLIP—on the same VQA and spatial prompts. If the margins shrink or reverse, the claimed spatial-structure advantage is an artifact of protocol or language-model pairing rather than of the encoder.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the claim holds, compact 3D medical vision–language models are viable: a ~19M-parameter visual encoder plus a 3–4B LLM can match systems twice as large on volumetric reasoning, easing deployment constraints.
  • The consistent gains on organ recognition, abnormality detection, and volume-magnitude reasoning indicate that JEPA-style latent prediction is a stronger pretraining objective than contrastive or reconstruction-based methods specifically for tasks that require knowing where things are.
  • The sharp optimum at 75% masking suggests that the amount of context withheld during pretraining is a sensitive hyperparameter; other volumetric modalities should tune it rather than reuse video or image defaults.
  • The model still trails on directional (DIR) reasoning, which the authors attribute to a missing spatial component; fixing this would be a natural next step and a test of whether the encoder truly captures all spatial relations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • We infer that a controlled experiment fixing the language model and swapping only the visual encoder would isolate how much of the reported margin comes from Rad-JEPA 3D's representation versus from the LLM or decoding procedures; such an experiment is not reported in the paper.
  • We infer that the per-token routing scheme suggests a broader design principle: for volumetric data, letting the model assign each patch to sequential or global processing may be more parameter-efficient than fixed hybrid stacks, and could transfer to other 3D modalities.
  • We infer that the frozen k-NN probing results imply the pretrained features are directly usable for few-shot label mapping on new CT cohorts without fine-tuning, a practical consequence the paper leaves implicit.
  • We infer that if the scale-substitution claim is right, it challenges the assumption that larger language models are the primary driver of 3D medical VLM performance, redirecting research attention to geometric fidelity in visual encoders.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. Rad-JEPA 3D proposes a self-supervised 3D CT representation learning method combining a joint-embedding predictive objective with a hybrid Mamba/GQA encoder and a layer-wise Hidden States Orthogonal Regularization (HSOR). The visual encoder is pretrained on ~120k CT volumes and then connected to Qwen2.5-3B or Qwen3-4B for VQA and spatial reasoning. The paper reports frozen-encoder k-NN gains over Swin UNETR, Models Genesis, and VoCo-B, closed-ended M3D-VQA accuracy of 81.66%, open-ended gains over Med3DVLM, and the best average on the SpatialMed benchmark (58.16%). The central claim is that the induced spatial structure can substitute for raw language-model scale.

Significance. If the reported results are taken at face value, the paper offers a compact 19M-parameter 3D encoder that achieves competitive or state-of-the-art VQA and spatial-reasoning results with a small LLM, which would be practically useful for resource-constrained medical imaging. The k-NN probing design is a sound way to isolate encoder quality, and most internal arithmetic in Tables 1, 4, 5, and 6 is consistent. The hybrid Mamba-GQA block and HSOR are clearly described, and the ablations are well formed. However, the central interpretational claim — that spatial structure substitutes for language-model scale — is not supported by the current comparisons, because the LLM, protocol, and encoder all change across the headline rows. The manuscript also omits evaluation-protocol details and contains arithmetic errors in Table 3. These issues are load-bearing and require additional experiments or a substantial reframing.

major comments (4)
  1. [Abstract; Results, Tables 1–3] The claim that 'the induced spatial structure can substitute for raw language-model scale' is not supported by any controlled experiment. In Tables 1–3, the proposed 4.0B model differs from Med3DVLM in the vision encoder, the LLM (Qwen3-4B vs Qwen2.5-7B-Instruct), and the instruction-tuning/decoding protocol. The paper's own comparisons show that changing only the LLM/generator from Qwen2.5-3B to Qwen3-4B changes closed-ended accuracy from 81.06 to 81.66 and SpatialMed from 52.64 to 58.16. To support the scale-substitution mechanism, the authors should provide an experiment with a fixed LLM and training protocol while swapping only the vision encoder (e.g., Rad-JEPA 3D vs M3D-CLIP vs DCFormer). Without this, the open-ended margins and spatial averages could be attributable to the newer LLM or decoding choices rather than to the encoder.
  2. [Experimental Setup; Tables 1–3] Baseline comparability is not established. The manuscript states that 'additional architectural, optimization, and decoding configurations are provided in the supplementary,' but the supplementary is not included. Prompt templates, maximum token counts, sampling/decoding settings, LoRA ranks, instruction-tuning epochs, and pretraining budgets for the baselines are absent. The reported +33.8 BLEU-1 open-ended margin and the closed-ended/spatial averages could therefore be an artifact of protocol mismatch rather than representation quality. The authors should report the full evaluation and tuning protocol and, where feasible, re-run baselines under the same inference conditions.
  3. [Table 3] The average column in Table 3 contains arithmetic errors that affect the headline comparison. For M3D-LaMed, the four sub-scores 0.12, 27.13, 33.22, 13.30 have mean 18.44, not the reported 21.92. Med-2E3's row has mean 40.80, not 41.19, and BTB3D's row has mean 31.99, not 31.86. The M3D-LaMed discrepancy changes the apparent ordering of the weakest baseline and must be corrected and re-verified across all tables.
  4. [Figure 2; Representation learning comparison] The k-NN probing comparison in Figure 2 does not control for pretraining data or compute. The paper does not report the pretraining corpus size, number of epochs, or preprocessing used for Swin UNETR, Models Genesis, and VoCo-B. If those baselines were pretrained on substantially smaller datasets, the reported margins (e.g., +2.6 on Organ) may reflect data/budget rather than architectural or objective superiority. Please report the pretraining budgets and ideally retrain or evaluate the baselines under the same 120k-volume setting used for Rad-JEPA 3D.
minor comments (7)
  1. [Experimental Setup] The text says 'closed-loop and open-loop' M3D-VQA test sets; the correct terms used elsewhere are 'closed-ended' and 'open-ended.'
  2. [Figure 2 caption] The caption refers to 'RadJepa-Hybrid' while the paper uses 'Rad-JEPA 3D'; please unify the terminology.
  3. [Table 2] If the 3.0B variant is also highlighted, note that its BERTScore (90.77) is below Med3DVLM (92.18); the claim that the model 'leads on every metric' applies only to the 4.0B variant.
  4. [Table 4] The first row of Table 4 has no component labels; add a 'baseline' label to make the ablation structure immediately clear.
  5. [Method, Eq. (6)] The notation W is used both for the collection of weight matrices and for a flattened 2D matrix; please distinguish these to avoid confusion in the Gram-matrix penalty definition.
  6. [Implementation Details] The description of the encoder as 'eight Mamba-2 layers and six GQA-based Transformer layers' is unclear relative to the H-Mamba layer that contains both branches; please specify the total layer count and how the branches are interleaved.
  7. [Ablation Studies] Ablation results are reported without error bars or repeated runs; given that some effects are small (e.g., Table 4 Organ gains), a statistical statement would strengthen the complementarity claim.

Circularity Check

1 steps flagged

No equation-level circularity; one load-bearing self-benchmark (SpatialMed) and uncontrolled LLM/decoding comparisons weaken the scale-substitution claim, but no derivation reduces to its inputs by construction.

specific steps
  1. self citation load bearing [Experimental Setup / Dataset; Results / Spatial Reasoning comparison; References [Trinh et al., 2026]]
    "For the evaluation set, we evaluate medical image visual question answering on both closed-loop and open-loop M3D-VQA test sets and spatial reasoning on the Multiple choice questions benchmark from SpatialMed [Trinh et al., 2026]."

    SpatialMed is cited as an external benchmark, but it is prior work by the same group: current first author Quoc-Huy Trinh and senior author Ulas Bagci are both authors of the cited SpatialMed paper. The paper's central spatial claim—'the induced spatial structure can substitute for raw language-model scale on volumetric reasoning tasks'—is measured largely on this self-authored benchmark. The benchmark's task design and baseline protocol are therefore imported by self-citation rather than independently anchored, making the 'best spatial-reasoning score' a self-defined success criterion for the spatial half of the headline.

full rationale

The pretraining objective (L = L_JEPA + gamma_align L_align + gamma_reg L_reg + lambda_bal L_bal) and the downstream evaluations are measurements, not fitted quantities renamed as predictions. The k-NN probing in Figure 2 and Tables 4–6 is an external probe of frozen encoder features, and the M3D-VQA closed/open-ended scores are against a benchmark external to this group. There is no step where an equation reduces to its own input or where a tuned parameter is relabeled as a predicted result. The main circularity-adjacent issue is the self-authored SpatialMed benchmark: the paper's strongest mechanism claim is supported by best-average spatial reasoning on a benchmark introduced by the same first and senior authors, and no independent spatial benchmark is supplied. A second, non-circular but important validity gap is that the scale-substitution conclusion is inferred from comparisons that simultaneously change the vision encoder, the LLM (Qwen2.5-3B vs Qwen3-4B vs Qwen2.5-7B), and decoding settings ('Additional architectural, optimization, and decoding configurations are provided in the supplementary'), so the 3B-to-4B gains could reflect LLM generation rather than Rad-JEPA's spatial representations. These are comparability and self-benchmark concerns, not circular derivation; hence the score is 3 rather than higher.

Axiom & Free-Parameter Ledger

6 free parameters · 7 axioms · 0 invented entities

The central claim rests on: (i) the premise that masked latent prediction on cubes teaches the coarse spatial structure needed by downstream VLMs — a design hypothesis tested only via final benchmarks; (ii) comparability of all baselines, asserted but not documented; (iii) validity of the self-authored SpatialMed benchmark for the headline spatial claim; and (iv) standard SSL stability assumptions (EMA teacher, stop-gradient). The free parameters (mask ratio, cube size, HSOR weights, EMA schedule) were selected on the same tasks used for the headline numbers, so the reported margins are conditional on that selection. No invented entities are introduced.

free parameters (6)
  • Mask ratio = 0.75
    Selected via ablation on the Plane/Phase/Organ probing tasks (Table 5: 0.75 gives 55.3 mean vs 52.2–52.9 for neighbors), then interpreted as evidence for the spatial-inference mechanism.
  • HSOR and router loss weights = γ_align=0.1, γ_reg=1e-6, λ_bal=0.005, λ_off=0.005
    Chosen 'after carefully tuning' (Training Objective); all downstream k-NN and VQA numbers use these values, so the reported gains are conditional on this selection.
  • Cube size = 8×8×8
    Selected via ablation (Table 6); 16³ coarsens too far, 4×16×16 fragments cross-slice anatomy; all main results use the best setting.
  • EMA decay τ = unspecified
    The teacher EMA schedule is never stated. All JEPA and JEPA+HSOR results depend on it, and prior JEPA works require a carefully scheduled τ; its absence blocks reproduction and makes isolating the HSOR contribution impossible.
  • Representation and predictor dims = d=384; predictor 192, 4 layers
    Architecture constants chosen without external benchmarking; the results depend on them.
  • LoRA configuration for VLM tuning = unspecified
    Rank, target modules, and data mix for instruction tuning are deferred to an absent supplementary; downstream VQA and spatial numbers depend on them.
axioms (7)
  • domain assumption Masked latent prediction on 75%-masked cube sequences induces coarse spatial/geometric structure useful to downstream VLMs.
    Section 'Masked Denoising in Representation Space' and Table 5. This is the paper's design hypothesis; it is validated only indirectly by final benchmark scores.
  • domain assumption Frozen-encoder k-NN accuracy on Plane/Phase/Organ labels faithfully ranks representation quality for VQA and spatial reasoning.
    Figure 2 / 'Representation learning comparison'; the probe protocol (layer, normalization, distance) is partially specified.
  • domain assumption Baseline SSL models (Swin UNETR, Models Genesis, VoCo-B) were pretrained on data comparable to the ~120k volumes used here.
    Figure 2; the paper never states baseline pretraining data scale, so k-NN margins may conflate method with data scale.
  • domain assumption Published baseline VLM scores (Tables 1–3) were produced under instruction-tuning, prompt, and decoding protocols compatible with the authors'.
    Tables 1–3; no decoding or prompt details are given, and the open-ended BLEU-1 margin (+33.8) is suspiciously large.
  • standard math EMA teacher with stop-gradient produces stable, non-degenerate targets.
    Section 'Masked Denoising in Representation Space'; standard BYOL/I-JEPA assumption.
  • standard math The min(m,n)-Gram weight-orthogonality penalty (Eq. 6–7) is well-posed for wide matrices.
    Section 'Weight orthogonality'; the authors themselves discuss the m<n ill-posedness and choose the smaller Gram, which is mathematically sound.
  • domain assumption SpatialMed benchmark (authored by the same group) is a valid, unbiased test of spatial reasoning.
    Table 3; the benchmark is a self-authored preprint (Trinh et al. 2026), so the headline spatial claim rests on the benchmark's objectivity being independent of its authors' model design.

pith-pipeline@v1.3.0-alltime-deepseek · 12986 in / 32801 out tokens · 293649 ms · 2026-08-01T00:33:14.896345+00:00 · methodology

0 comments
read the original abstract

Self-supervised pretraining is central to 3D medical image analysis, where unlabeled CT volumes are abundant but expert annotations are scarce. Yet existing volumetric encoders often fail to preserve the coarse spatial and geometric structure that downstream reasoning depends on, limiting their performance on organ disentanglement, abnormality detection, and spatial understanding when paired with language models. We introduce Rad-JEPA 3D, a joint-embedding predictive framework that learns volumetric CT representations by predicting the latent features of a complete scan from a masked view. At its core is a hybrid H-Mamba encoder that fuses a Mamba state-space branch, which models inter-slice continuity through sequential scanning, with a grouped-query attention branch, which captures cross-plane spatial context, combined through a lightweight per-token router. To improve the quality of intermediate representations, we further propose Hidden States Orthogonal Regularization (HSOR), which aligns student-teacher hidden states and reduces feature redundancy throughout the encoder. This layer-wise regularization produces more consistent and discriminative volumetric representations, leading to improved performance on organ recognition and spatial reasoning tasks. Pretrained on approximately 120,000 CT scans, Rad-JEPA 3D attains state-of-the-art results despite its compact size: with only 4.0B total parameters, it achieves competitive results with state-of-the-art on closed-ended VQA and the best average spatial-reasoning score on the Spatial-Med benchmark. Ablation studies confirm that the hybrid block and HSOR contribute complementary gains, and that the induced spatial structure can substitute for raw language-model scale on volumetric reasoning tasks.

Figures

Figures reproduced from arXiv: 2607.26196 by Minh-Van Nguyen, Quoc-Huy Trinh, Ulas Bagci.

Figure 1
Figure 1. Figure 1: Overall architecture of Rad-JEPA 3D. A CT volume is tokenized into cubes and passed to two weight-sharing en [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Frozen-encoder k-NN probing accuracy on three CT tasks. Encoders are self-supervised and kept frozen; a k-NN probe classifies their representations, isolating rep￾resentation quality from fine-tuning. Tasks are ordered by increasing granularity: scan plane (chance 50.0%), contrast phase (chance 10.0%), and organ (chance 2.9%). RadJepa￾Hybrid leads on all three, with the largest margin on the finest-grained… view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of unidirectional and bidirectional [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

20 extracted references · 15 linked inside Pith

  1. [1]

    Gqa: Training generalized multi-query transformer models from multi-head checkpoints

    Joshua Ainslie, James Lee-Thorp, Michiel De Jong, Yury Zemlyanskiy, Federico Lebr´on, and Sumit Sanghai. Gqa: Training generalized multi-query transformer models from multi-head checkpoints. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4895–4901,

    2023

  2. [5]

    Masked and predictive self-supervised foun- dation models for 3d brain mri.arXiv preprint arXiv:2606.13315,

    Esra Erg¨un, Hersh Chandarana, Dan Sodickson, and G ¨ozde ¨Unal. Masked and predictive self-supervised foun- dation models for 3d brain mri.arXiv preprint arXiv:2606.13315,

    Pith/arXiv arXiv

  3. [6]

    Li, Aravind R

    Thomas Z. Li, Aravind R. Krishnan, Lianrui Zuo, John M. Still, Kim L. Sandler, Fabien Maldonado, Thomas A. Lasko, and Bennett A. Landman. Self-supervised learn- ing of imaging and clinical signatures using a multimodal joint-embedding predictive architecture.arXiv preprint arXiv:2509.15470,

    arXiv

  4. [8]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101,

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101,

    Pith/arXiv arXiv

  5. [11]

    V-jepa 2.1: Unlocking dense features in video self-supervised learning.arXiv preprint arXiv:2603.14482,

    Lorenzo Mur-Labadia, Matthew Muckley, Amir Bar, Mido Assran, Koustuv Sinha, Mike Rabbat, Yann LeCun, Nico- las Ballas, and Adrien Bardes. V-jepa 2.1: Unlocking dense features in video self-supervised learning.arXiv preprint arXiv:2603.14482,

    Pith/arXiv arXiv

  6. [12]

    Unimedvl: Unifying medical multi- modal understanding and generation through observation- knowledge-analysis.arXiv preprint arXiv:2510.15710,

    Junzhi Ning, Wei Li, Cheng Tang, Jiashi Lin, Chenglong Ma, Chaoyang Zhang, Jiyao Liu, Ying Chen, Shujian Gao, Lihao Liu, et al. Unimedvl: Unifying medical multi- modal understanding and generation through observation- knowledge-analysis.arXiv preprint arXiv:2510.15710,

    Pith/arXiv arXiv

  7. [13]

    Fernando P ´erez-Garc´ıa, Harshita Sharma, Sam Bond- Taylor, Kenza Bouzid, Valentina Salvatelli, Maxim- ilian Ilse, Shruthi Bannur, Daniel C

    arXiv:2304.07193. Fernando P ´erez-Garc´ıa, Harshita Sharma, Sam Bond- Taylor, Kenza Bouzid, Valentina Salvatelli, Maxim- ilian Ilse, Shruthi Bannur, Daniel C. Castro, Anton Schwaighofer, Matthew P. Lungren, et al. RAD-DINO: Exploring scalable medical image encoders beyond text supervision.arXiv preprint arXiv:2401.10815,

    Pith/arXiv arXiv

  8. [14]

    Arnold, and William Speier

    Ashwath Radhachandran, Vedrana Ivezi ´c, Shreeram Athreya, Ronit Anilkumar, Corey W. Arnold, and William Speier. Us-jepa: A joint embedding predictive architecture for medical ultrasound.arXiv preprint arXiv:2602.19322,

    arXiv

  9. [15]

    Dinov3.arXiv preprint arXiv:2508.10104,

    Oriane Sim ´eoni, Huy V V o, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Micha ¨el Ramamonjisoa, et al. Dinov3.arXiv preprint arXiv:2508.10104,

    Pith/arXiv arXiv

  10. [17]

    Virchow: A million-slide digital pathology foun- dation model.arXiv preprint arXiv:2309.07778,

    Eugene V orontsov, Alican Bozkurt, Adam Casson, George Shaikovski, Michal Zelechowski, Siqi Liu, Kristen Sev- erson, Eric Zimmermann, James Hall, Neil Tenenholtz, et al. Virchow: A million-slide digital pathology foun- dation model.arXiv preprint arXiv:2309.07778,

    Pith/arXiv arXiv

  11. [18]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191,

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191,

    Pith/arXiv arXiv

  12. [19]

    Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and St´ephane Deny

    doi: 10.1038/s41746-025-02035-w. Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and St´ephane Deny. Barlow twins: Self-supervised learning via redundancy reduction. InInternational conference on machine learning, pages 12310–12320. PMLR,

    doi:10.1038/s41746-025-02035-w

  13. [20]

    Gotway, and Jianming Liang

    Zongwei Zhou, Vatsal Sodha, Jiaxuan Pang, Michael B. Gotway, and Jianming Liang. Models genesis.Medical Image Analysis, 67:101840, 2021

    2021

  14. [2017]

    V olta-3d: Self-supervised learning for brain mri using 3d volumet- ric token alignment.arXiv preprint arXiv:2605.16775,

    Amy Makawana, Abhijeet Parida, Marius George Lingu- raru, Julia Ive, and Syed Muhammad Anwar. V olta-3d: Self-supervised learning for brain mri using 3d volumet- ric token alignment.arXiv preprint arXiv:2605.16775,

    Pith/arXiv arXiv

  15. [2021]

    M3d: Advancing 3d medical image analysis with multi-modal large language models.arXiv preprint arXiv:2404.00578,

    Fan Bai, Yuxin Du, Tiejun Huang, Max Q-H Meng, and Bo Zhao. M3d: Advancing 3d medical image analysis with multi-modal large language models.arXiv preprint arXiv:2404.00578,

    Pith/arXiv arXiv

  16. [2022]

    Beyond medical diagnostics: How medical multimodal large language models think in space

    Quoc-Huy Trinh, Xi Ding, Yang Liu, Zhenyue Qin, Xingjian Li, Gorkem Durak, Halil Ertugrul Aktas, Elif Keles, Ulas Bagci, and Min Xu. Beyond medical diagnostics: How medical multimodal large language models think in space. arXiv preprint arXiv:2603.13800,

    Pith/arXiv arXiv

  17. [2023]

    V-jepa 2: Self-supervised video models enable understanding, pre- diction and planning.arXiv preprint arXiv:2506.09985, 2025a

    Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, et al. V-jepa 2: Self-supervised video models enable understanding, pre- diction and planning.arXiv preprint arXiv:2506.09985, 2025a. Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Matthe...

    Pith/arXiv arXiv

  18. [2024]

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin

    arXiv:2404.08471. Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transform- ers. InProceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV), pages 9650–9660,

    Pith/arXiv arXiv

  19. [2025]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee

    URL https://arxiv.org/abs/2502.09838. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916,

    Pith/arXiv arXiv

  20. [2026]

    Lost in volume: The ct- spatialvqa benchmark for evaluating semantic-spatial un- derstanding of 3d medical vision-language models.arXiv preprint arXiv:2605.08787,

    Mashrafi Monon, Umaima Rahman, Asif Hanif, Numan Saeed, and Mohammad Yaqub. Lost in volume: The ct- spatialvqa benchmark for evaluating semantic-spatial un- derstanding of 3d medical vision-language models.arXiv preprint arXiv:2605.08787,

    Pith/arXiv arXiv