Learning How Much, Not Just What: Cross-Patient Burden Order for CT Vision-Language Pretraining

Guoliang You; Haifan Gong; Xiaomeng Chu

REVIEW 4 major objections 4 minor 12 references

Weak cross-patient order can teach CT models disease severity, not just presence, without longitudinal data.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 00:56 UTC pith:ZIEVLR4R

load-bearing objection Spectrum's cross-patient burden-order idea is genuinely new and the gains are real, but the rule-based scorer that generates all order supervision is unvalidated, so the 'burden-aware' claim needs direct evidence. the 4 major comments →

arxiv 2608.00231 v1 pith:ZIEVLR4R submitted 2026-07-31 cs.CV

Learning How Much, Not Just What: Cross-Patient Burden Order for CT Vision-Language Pretraining

Guoliang You , Haifan Gong , Xiaomeng Chu This is my paper

classification cs.CV

keywords CT vision-language pretrainingburden ordercross-patient relation miningBurden-Direction Alignmentzero-shot abnormality diagnosisanatomy-conditioned representationradiology report gradingordinal representation learning

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that current CT vision-language pretraining only learns which findings appear in a scan, collapsing words like 'mild' and 'extensive' into a present/absent signal. It proposes Spectrum, which mines weak lower-to-higher burden relations between different patients from labels and report wording, and enforces them with Burden-Direction Alignment: the pathology-conditioned image delta is aligned to the report delta at whole-study and organ scopes, with auxiliary losses that separate the selected direction from its reverse. The result is a burden-aware representation space without longitudinal repeat scans. If correct, this turns ordinary cross-sectional hospital data into a scalable source of severity supervision.

Core claim

On its own terms, the paper establishes that 'how much' can be learned from cross-sectional data by treating report language as a weak ordinal signal. Spectrum's Burden-Direction Alignment takes a mined lower-to-higher pair of patients for a specific pathology, conditions both modalities on a pathology embedding, uses a target-conditioned aligner to make the lower-burden image comparable to the higher-burden one, and aligns the resulting image delta with the direct text delta. Direction, endpoint, and triplet-composition losses keep the orientation well-posed. The paper reports 85.6 zero-shot AUROC on CT-RATE and 72.7 on RAD-ChestCT, with consistent gains in linear probing and retrieval, and

What carries the argument

Burden-Direction Alignment (BDA): a training-time objective that aligns pathology-conditioned image deltas with report deltas along a mined lower-to-higher burden direction. A fixed rule-based scorer mines cross-patient ordered pairs from labels and report cues; a target-conditioned gated residual aligner removes between-patient variation before differencing; and direction, endpoint-consistency, and triplet-composition losses keep the order well-posed. BDA operates in feature space, is discarded after training, and lives entirely within anatomy-conditioned whole-study and organ scopes.

Load-bearing premise

The claim depends on the rule-based burden scorer's cross-patient score differences reflecting actual visual disease burden rather than report wording or style; if scores diverge from true image burden, BDA would anchor the representation to a text-derived direction that is not the real severity direction.

What would settle it

Take a held-out set of cross-patient pairs for one pathology, have radiologists annotate which image carries more disease, and compare their ordering with the rule-based scorer's ordering. If Spectrum's feature-space direction aligns with the scorer whenever the scorer disagrees with radiologists (or if the zero-shot AUROC gain disappears when training on scorer-mined pairs whose order is radiologist-confirmed), the burden-direction claim would be falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

CT vision-language pretraining can encode graded severity without longitudinal data, making severity ordering available at the scale of routine cross-sectional cohorts.
Zero-shot abnormality diagnosis improves on both in-domain and external datasets, suggesting the learned burden direction transfers across scanners and populations.
Frozen-encoder linear probing and retrieval improve alongside diagnosis, indicating the order signal reorganizes the whole representation space, not just the final classifier.
Improvements appear for every organ group, including pathologies not explicitly mapped to an organ, so the burden signal propagates beyond its direct supervision scopes.
Because BDA modules are discarded after training, the method adds no inference-time cost or architectural change at deployment.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same cross-patient order-mining recipe could be tested on chest X-ray or MRI vision-language pretraining, where reports also contain graded severity language; the gains may transfer if the scorer is adapted to each modality's terminology.
The hand-written burden scorer could be replaced by a calibrated language-model ranker, which might increase mining precision and reduce sensitivity to report wording style; this is a natural next ablation.
A clinically testable extension: if the learned burden direction is truly visual, it should predict radiologist-annotated severity scores (e.g., lesion volume or extent) on held-out pairs better than correspondence-only representations; that prediction could be measured directly.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Spectrum's cross-patient burden-order idea is genuinely new and the gains are real, but the rule-based scorer that generates all order supervision is unvalidated, so the 'burden-aware' claim needs direct evidence.

read the letter

You should know about this paper because it proposes something I haven't seen in CT VLP: mining weak cross-patient lower-to-higher burden pairs from reports and using them as supervision to align image and text deltas. That's a clever way to get graded signal without longitudinal data. The method is well-engineered — anatomy-conditioned scopes, a target-conditioned aligner to remove between-patient variation, and a set of direction/endpoint/composition losses. The ablation shows BDA adds 8.9 AUROC over a matched no-BDA baseline (76.7 to 85.6), and the gains transfer externally to RAD-ChestCT. I think this is a solid contribution.

Where I get uneasy is the rule-based burden scorer φ_p (Eq. 4). Every ordered relation is mined by hand-written rules over report language (negation, size cues, severity descriptors, uncertainty). Since eligible pairs are both label-positive, the label term can't rank them — the order is entirely determined by these language cues. The paper never validates that the score differences are monotonically related to actual visual burden. The only direct evidence is Figure 5, one lung-nodule triplet whose size cues are part of the scorer. That's close to circular. If φ_p mostly encodes report style or verbosity, then BDA is aligning image deltas to a text axis that isn't really burden. The +8.9 AUROC could then come from a report-correlated axis that helps classification without making the representation genuinely burden-aware.

There's also a protocol wrinkle: the zero-shot numbers include a label-supervised warm-up (L_warm). That's fine as a method choice, but it means the headline comparison to CT-CLIP etc. isn't apples-to-apples unless those baselines also used labels. The internal ablation matches the architecture, so BDA's contribution is still credible, but the absolute "85.6 zero-shot" should be described carefully.

Also, no error bars, no multiple seeds, no code release. For a paper whose central claim is a new training signal, that's a reproducibility gap.

Net: I'd send this to peer review — the idea is important enough, and the method is coherent. But the reviewers should push hard for a validation of the scorer: e.g., evaluate ordering against lesion size or a manual severity subset, and show that the learned direction correlates with burden beyond report wording. Fixing that would make this a strong paper; right now it's a good paper with an unproven central mechanism.

For your reading group: worth a session. I'd cite it if I worked in CT VLP.

Referee Report

4 major / 4 minor

Summary. The paper proposes Spectrum, an anatomy-conditioned CT vision-language pretraining framework that jointly learns whole-study and organ-level representations. In addition to standard global and organ contrastive correspondence, it introduces Burden-Direction Alignment (BDA): a fixed rule-based scorer mines ordered lower-to-higher cross-patient relations for organ-mapped pathologies from labels and report language (Eq. 4); BDA then aligns pathology-conditioned image deltas with report deltas at whole-study and organ scopes, with auxiliary terms for forward-vs-reverse direction, endpoint consistency, and ordered-triplet composition (Eqs. 7–8). BDA modules are used only during training. The paper reports zero-shot AUROC of 85.6 on CT-RATE and 72.7 on external RAD-ChestCT, with gains in linear probing and retrieval. A matched ablation attributes +8.9 AUROC to BDA (Table 4).

Significance. If the central claim is established, the contribution is meaningful: it offers a scalable way to inject graded 'how much' supervision from cross-sectional data, complementing anatomy-aware correspondence without requiring longitudinal follow-up. The paper has notable strengths: the anatomy-conditioned scope design is principled, the BDA ablation is a matched internal control (Table 4), the method transfers to an external dataset (Table 1), and BDA modules are discarded at inference, so inference cost is unchanged. However, the burden-order signal itself is not independently validated, and the 'zero-shot' protocol includes label-supervised warm-up. These issues are load-bearing for the paper's main interpretation, so the significance currently remains conditional.

major comments (4)

[§III.C, Eq. (4)]
[§III.D]
[Tables 1, 4; §IV.C]
[Fig. 5, Eq. (8)]

minor comments (4)

[§IV.A]
[§III.C]
[§IV.B]
[§II.A]

Circularity Check

1 steps flagged

Figure 5's burden-order validation is trained by L_dir, so the reported forward-reverse preference is the training objective itself; main zero-shot/linear-probing results remain external and independent.

specific steps

fitted input called prediction [Section III-C, Eq. (8) and Section IV-D, Fig. 5 (Model-derived burden-order geometry)]
"The direction term enforces s_fwd ≥ s_rev + μ while keeping the textual reference fixed in the selected a→b direction. ... For the lung-nodule triplet in Fig. 5, the lower-to-medium, medium-to-higher, and lower-to-higher gaps are +0.034, +0.011, and +0.060, respectively. ... All three forward–reverse similarity gaps are positive, so the trained model prefers the displayed direction to its reverse."

The positive forward-minus-reverse similarity gaps are not predicted by the model; they are directly optimized by L_dir in Eq. (8), which is applied to the same selected cross-patient relations used in training. The lung-nodule triplet was itself mined by the rule-based scorer from report size cues (Eq. (4)), so displaying that the model 'prefers' the mined direction is equivalent to confirming that the training loss was minimized on a training example. This makes Fig. 5 a fitted-input-called-prediction validation rather than an independent test of burden-order learning.

full rationale

The core derivation is not circular: Spectrum's Burden-Direction Alignment is a training objective, and the headline numbers (85.6 zero-shot AUROC on CT-RATE, 72.7 on RAD-ChestCT, linear-probing AUROC, retrieval gains) are measured on external or held-out benchmarks and do not reduce to the mined order. The only clear circular step is the qualitative burden-order geometry in Fig. 5, where the reported forward-reverse preference is exactly what L_dir enforces on selected relations; showing positive gaps for a triplet that was selected by the same rule-based scorer is showing the optimization target rather than an emergent prediction. The unvalidated nature of the rule-based scorer φ_p is an assumption about the text-to-visual burden link, not a constructional circularity, and the main benchmark comparisons retain independent content. Hence the paper is partially circular in one supporting evidence item but not in its central external evaluations.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 1 invented entities

The central claim depends on hand-designed scoring rules, manual thresholds, and tuned loss weights, plus domain assumptions that report language and organ masks encode true burden. The invented entity 'burden direction' is a latent construct with no direct external validation. These are the main costs the reader pays beyond standard contrastive pretraining.

free parameters (5)

Rule-based burden scorer thresholds δ, η, s_min = 0.75 each
Manual thresholds in Eq. (4) determine which cross-patient pairs count as lower-to-higher. They are chosen by hand, not learned, and directly control the BDA training signal.
BDA loss weights λ_BDA, λ_dir, λ_end, λ_comp = 0.64, 0.50, 0.25, 0.03
Tuned hyperparameters that balance the burden-order objective against correspondence. λ_BDA directly controls how much burden signal enters training.
Direction margin μ and temperature τ_o = 0.10 and 0.07
Margin for forward-reverse direction loss and temperature for organ contrastive loss; both are hand-set.
Top-K selected relations and max eligible triplets = K=32, max 8 per mini-batch
Manual caps on how many mined relations are used per batch; affects which patients and pathologies contribute.
Rule-based scoring function φ_p
The scorer's parsing rules for negation, size, severity, and uncertainty are described only verbally and not specified in equations or code. This is a large hand-designed component of the training signal.

axioms (6)

domain assumption Cross-patient report score differences are monotone in true visual disease burden.
Equation (4) treats s_b - s_a ≥ δ as evidence that patient b has more disease burden than patient a; if report language is inconsistent across patients, the mined order is wrong.
domain assumption The target-conditioned aligner can remove between-patient variation while preserving burden.
Section III.C, Eq. (6) assumes the residual after aligning lower-burden to higher-burden visual features isolates burden from habitus, protocol, and anatomy.
domain assumption The fixed pathology-to-organ map is valid.
Section III.A maps 16 of 18 labels to lung/heart/esophagus/aorta; if this mapping is wrong, organ-scope BDA constrains the wrong anatomical direction.
domain assumption CT-RATE labels and organ masks are reliable.
Warm-up and relation mining use labels and masks; errors in these annotations propagate into the burden-order signal.
standard math Contrastive losses and cosine direction matching are valid objectives.
The method relies on standard InfoNCE/contrastive assumptions and L2-normalized cosine alignment; no new mathematical machinery is introduced.
ad hoc to paper Discarding BDA modules after training preserves inference quality.
Section III.D states BDA acts only in feature space and is removed at inference; this assumes the training-time effect transfers to the frozen encoder and projection heads.

invented entities (1)

Pathology-conditioned burden direction in feature space no independent evidence
purpose: A per-pathology latent axis representing lower-to-higher disease burden; BDA aligns image and text deltas along this direction.
The burden direction is the central construct but is never measured directly against an external severity score. The only direct evidence is one qualitative triplet in Fig. 5, which uses the same report-derived ordering that trains the model.

pith-pipeline@v1.3.0-alltime-deepseek · 13376 in / 12712 out tokens · 118382 ms · 2026-08-04T00:56:08.103590+00:00 · methodology

0 comments

read the original abstract

Volumetric CT vision-language pretraining learns 3D representations from scan-report pairs, but global and anatomy-aware objectives supervise only correspondence: they establish what is present and leave how much unconstrained. Nothing separates a mild from an extensive case of the same finding along a consistent direction, so the graded burden language in reports collapses into a present/absent signal. Longitudinal supervision would supply this order, but patient-matched CT pairs are scarce at scale; cross-sectional cohorts already encode weak burden cues across different patients. We introduce Spectrum, an anatomy-conditioned framework that represents each study at whole-study and organ scopes. For each organ-mapped pathology, a rule-based scorer mines confidence-filtered lower-to-higher pairs of different patients, and Burden-Direction Alignment (BDA) aligns the pathology-conditioned image delta with the report delta at each scope, separating that direction from its reverse. Because the endpoints are different people, a target-conditioned aligner first makes them comparable, so the delta reflects burden rather than between-patient variation. BDA further separates the selected direction from its reverse, anchors it to the observed higher-burden endpoint, and enforces consistency across ordered triplets. Since every pair is drawn within a single pathology, BDA is designed to constrain intra-class structure that image-report contrast alone never touches. Spectrum attains 85.6 zero-shot AUROC on CT-RATE and 72.7 on external RAD-ChestCT, with consistent gains in linear probing and retrieval. Weak cross-patient order is thus a scalable complement to anatomy-aware correspondence, yielding burden-aware CT representations without longitudinal data.

Figures

Figures reproduced from arXiv: 2608.00231 by Guoliang You, Haifan Gong, Xiaomeng Chu.

**Figure 2.** Figure 2: Architecture of Spectrum. A mined relation orders two different patients, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Figure 3: Delta construction in BDA for pathology p at scope q. A target-conditioned aligner maps endpoint a (lower burden) toward b (higher) before differencing, giving dI , while the aligner-free text path gives dT . The endpoints are different patients, so only the residual difference isolates burden from habitus and protocol. input. The image and text adapters are shared across studies, anatomical scopes, and … view at source ↗

**Figure 4.** Figure 4: Global-prediction AUROC by organ group on CT [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

12 extracted references · 3 linked inside Pith

[5]

Gong, H.; Huang, W.; Zhang, H.; Wang, Y.; Wan, X.; Shen, H.;Li,G.;andLi,H.2024

VQAMix: Conditional Triplet Mixup for Medical Visual Question Answering.IEEE Transactions on Medical Imag- ing, 41(11): 3332–3343. Gong, H.; Huang, W.; Zhang, H.; Wang, Y.; Wan, X.; Shen, H.;Li,G.;andLi,H.2024. IntensityConfusionMatters:An Intensity-DistanceGuidedLossforBronchusSegmentation. In2024 IEEE International Conference on Multimedia and Expo (ICM...

2024
[6]

Jain,S.;Agrawal,A.;Saporta,A.;Truong,S.;Duong,D.N.; Bui, T.; Chambon, P.; Zhang, Y.; Lungren, M

BCNet: Bronchus Classifi- cation via Structure Guided Representation Learning.IEEE Transactions on Medical Imaging, 44(1): 489–498. Jain,S.;Agrawal,A.;Saporta,A.;Truong,S.;Duong,D.N.; Bui, T.; Chambon, P.; Zhang, Y.; Lungren, M. P.; Ng, A. Y.; etal.2021. RadGraph:ExtractingClinicalEntitiesandRela- tions from Radiology Reports. InThirty-fifth Conference on...

2021
[7]

Lin, W.; Zhao, Z.; Zhang, X.; Wu, C.; Zhang, Y.; Wang, Y.; and Xie, W

Ct-glip: 3d grounded language-image pretraining with ct scans and radiology reports for full-body scenarios.arXiv preprint arXiv:2404.15272. Lin, W.; Zhao, Z.; Zhang, X.; Wu, C.; Zhang, Y.; Wang, Y.; and Xie, W

arXiv
[9]

Radford, A.; Kim, J

Vision foundationmodelsforcomputedtomography.arXivpreprint arXiv:2501.09001. Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

Pith/arXiv arXiv
[10]

Wang,F.;Zhou,Y.;Wang,S.;Vardhanabhuti,V.;andYu,L

Comprehensive language-image pre-training for 3D medical image understanding.arXiv preprint arXiv:2510.15042. Wang,F.;Zhou,Y.;Wang,S.;Vardhanabhuti,V.;andYu,L. 2022a. Multi-granularity cross-modal alignment for gener- alized medical visual representation learning.Advances in neural information processing systems, 35: 33536–33549. Wang,S.;Gong,H.;Cui,R.;Wa...

arXiv
[12]

InProceedings of the IEEE/CVFconferenceoncomputervisionandpatternrecog- nition, 18123–18133

Lit: Zero-shot trans- fer with locked-image text tuning. InProceedings of the IEEE/CVFconferenceoncomputervisionandpatternrecog- nition, 18123–18133. Zhang,S.;Xu,Y.;Usuyama,N.;Xu,H.;Bagga,J.;Tinn,R.; Preston,S.;Rao,R.;Wei,M.;Valluri,N.;etal.2025. Amul- timodal biomedical foundation model trained from fifteen million image–text pairs.Nejm Ai, 2(1): AIoa240...

2025
[2021]

Gong,H.;Chen,G.;Liu,S.;Yu,Y.;andLi,G.2021

Machine-learning- basedmultipleabnormalitypredictionwithlarge-scalechest computedtomographyvolumes.Medicalimageanalysis,67: 101857. Gong,H.;Chen,G.;Liu,S.;Yu,Y.;andLi,G.2021. Cross- ModalSelf-AttentionwithMulti-TaskPre-TrainingforMed- icalVisualQuestionAnswering. InProceedingsofthe2021 InternationalConferenceonMultimediaRetrieval,456–460. ACM. Gong, H.; C...

2021
[2022]

InEuropean conference on computer vision, 1–21

Making the most of text semantics to improve biomedical vision–language pro- cessing. InEuropean conference on computer vision, 1–21. Springer. Burges, C.; Shaked, T.; Renshaw, E.; Lazier, A.; Deeds, M.; Hamilton,N.;andHullender,G.2005.Learningtorankusing gradient descent. InProceedings of the 22nd international conference on Machine learning, 89–96. Cao,...

2005
[2023]

InInternational Conference on Medical Image Computing and Computer- Assisted Intervention, 525–536

Pmc-clip: Contrastive language-image pre-training using biomedical documents. InInternational Conference on Medical Image Computing and Computer- Assisted Intervention, 525–536. Springer. Müller,P.;Kaissis,G.;Zou,C.;andRueckert,D.2022. Joint learning of localized representations from medical images and reports. InEuropean conference on computer vision, 68...

2022
[2024]

Bannur, S.; Hyland, S.; Liu, Q.; Perez-Garcia, F.; Ilse, M.; Castro,D.C.;Boecking,B.;Sharma,H.;Bouzid,K.;Thieme, A.; et al

M3d: Advancing 3d medical image analysis with multi-modal large language models.arXiv preprint arXiv:2404.00578. Bannur, S.; Hyland, S.; Liu, Q.; Perez-Garcia, F.; Ilse, M.; Castro,D.C.;Boecking,B.;Sharma,H.;Bouzid,K.;Thieme, A.; et al

Pith/arXiv arXiv
[2025]

Draelos,R.L.;Dov,D.;Mazurowski,M.A.;Lo,J.Y.;Henao, R.; Rubin, G

Curia: A Multi-Modal Foundation Model for Radiology.arXiv preprint arXiv:2509.06830. Draelos,R.L.;Dov,D.;Mazurowski,M.A.;Lo,J.Y.;Henao, R.; Rubin, G. D.; and Carin, L

Pith/arXiv arXiv
[2026]

Wang, Z.; Wu, Z.; Agarwal, D.; and Sun, J

Costal Cartilage Segmenta- tionwithTopologyGuidedDeformableMamba:Methodand Benchmark.ExpertSystemswithApplications,300:130085. Wang, Z.; Wu, Z.; Agarwal, D.; and Sun, J. 2022b. Med- clip:Contrastivelearningfromunpairedmedicalimagesand text. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 3876–3887. Wu, C.; Zhang, ...

2022

[1] [5]

Gong, H.; Huang, W.; Zhang, H.; Wang, Y.; Wan, X.; Shen, H.;Li,G.;andLi,H.2024

VQAMix: Conditional Triplet Mixup for Medical Visual Question Answering.IEEE Transactions on Medical Imag- ing, 41(11): 3332–3343. Gong, H.; Huang, W.; Zhang, H.; Wang, Y.; Wan, X.; Shen, H.;Li,G.;andLi,H.2024. IntensityConfusionMatters:An Intensity-DistanceGuidedLossforBronchusSegmentation. In2024 IEEE International Conference on Multimedia and Expo (ICM...

2024

[2] [6]

Jain,S.;Agrawal,A.;Saporta,A.;Truong,S.;Duong,D.N.; Bui, T.; Chambon, P.; Zhang, Y.; Lungren, M

BCNet: Bronchus Classifi- cation via Structure Guided Representation Learning.IEEE Transactions on Medical Imaging, 44(1): 489–498. Jain,S.;Agrawal,A.;Saporta,A.;Truong,S.;Duong,D.N.; Bui, T.; Chambon, P.; Zhang, Y.; Lungren, M. P.; Ng, A. Y.; etal.2021. RadGraph:ExtractingClinicalEntitiesandRela- tions from Radiology Reports. InThirty-fifth Conference on...

2021

[3] [7]

Lin, W.; Zhao, Z.; Zhang, X.; Wu, C.; Zhang, Y.; Wang, Y.; and Xie, W

Ct-glip: 3d grounded language-image pretraining with ct scans and radiology reports for full-body scenarios.arXiv preprint arXiv:2404.15272. Lin, W.; Zhao, Z.; Zhang, X.; Wu, C.; Zhang, Y.; Wang, Y.; and Xie, W

arXiv

[4] [9]

Radford, A.; Kim, J

Vision foundationmodelsforcomputedtomography.arXivpreprint arXiv:2501.09001. Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

Pith/arXiv arXiv

[5] [10]

Wang,F.;Zhou,Y.;Wang,S.;Vardhanabhuti,V.;andYu,L

Comprehensive language-image pre-training for 3D medical image understanding.arXiv preprint arXiv:2510.15042. Wang,F.;Zhou,Y.;Wang,S.;Vardhanabhuti,V.;andYu,L. 2022a. Multi-granularity cross-modal alignment for gener- alized medical visual representation learning.Advances in neural information processing systems, 35: 33536–33549. Wang,S.;Gong,H.;Cui,R.;Wa...

arXiv

[6] [12]

InProceedings of the IEEE/CVFconferenceoncomputervisionandpatternrecog- nition, 18123–18133

Lit: Zero-shot trans- fer with locked-image text tuning. InProceedings of the IEEE/CVFconferenceoncomputervisionandpatternrecog- nition, 18123–18133. Zhang,S.;Xu,Y.;Usuyama,N.;Xu,H.;Bagga,J.;Tinn,R.; Preston,S.;Rao,R.;Wei,M.;Valluri,N.;etal.2025. Amul- timodal biomedical foundation model trained from fifteen million image–text pairs.Nejm Ai, 2(1): AIoa240...

2025

[7] [2021]

Gong,H.;Chen,G.;Liu,S.;Yu,Y.;andLi,G.2021

Machine-learning- basedmultipleabnormalitypredictionwithlarge-scalechest computedtomographyvolumes.Medicalimageanalysis,67: 101857. Gong,H.;Chen,G.;Liu,S.;Yu,Y.;andLi,G.2021. Cross- ModalSelf-AttentionwithMulti-TaskPre-TrainingforMed- icalVisualQuestionAnswering. InProceedingsofthe2021 InternationalConferenceonMultimediaRetrieval,456–460. ACM. Gong, H.; C...

2021

[8] [2022]

InEuropean conference on computer vision, 1–21

Making the most of text semantics to improve biomedical vision–language pro- cessing. InEuropean conference on computer vision, 1–21. Springer. Burges, C.; Shaked, T.; Renshaw, E.; Lazier, A.; Deeds, M.; Hamilton,N.;andHullender,G.2005.Learningtorankusing gradient descent. InProceedings of the 22nd international conference on Machine learning, 89–96. Cao,...

2005

[9] [2023]

InInternational Conference on Medical Image Computing and Computer- Assisted Intervention, 525–536

Pmc-clip: Contrastive language-image pre-training using biomedical documents. InInternational Conference on Medical Image Computing and Computer- Assisted Intervention, 525–536. Springer. Müller,P.;Kaissis,G.;Zou,C.;andRueckert,D.2022. Joint learning of localized representations from medical images and reports. InEuropean conference on computer vision, 68...

2022

[10] [2024]

Bannur, S.; Hyland, S.; Liu, Q.; Perez-Garcia, F.; Ilse, M.; Castro,D.C.;Boecking,B.;Sharma,H.;Bouzid,K.;Thieme, A.; et al

M3d: Advancing 3d medical image analysis with multi-modal large language models.arXiv preprint arXiv:2404.00578. Bannur, S.; Hyland, S.; Liu, Q.; Perez-Garcia, F.; Ilse, M.; Castro,D.C.;Boecking,B.;Sharma,H.;Bouzid,K.;Thieme, A.; et al

Pith/arXiv arXiv

[11] [2025]

Draelos,R.L.;Dov,D.;Mazurowski,M.A.;Lo,J.Y.;Henao, R.; Rubin, G

Curia: A Multi-Modal Foundation Model for Radiology.arXiv preprint arXiv:2509.06830. Draelos,R.L.;Dov,D.;Mazurowski,M.A.;Lo,J.Y.;Henao, R.; Rubin, G. D.; and Carin, L

Pith/arXiv arXiv

[12] [2026]

Wang, Z.; Wu, Z.; Agarwal, D.; and Sun, J

Costal Cartilage Segmenta- tionwithTopologyGuidedDeformableMamba:Methodand Benchmark.ExpertSystemswithApplications,300:130085. Wang, Z.; Wu, Z.; Agarwal, D.; and Sun, J. 2022b. Med- clip:Contrastivelearningfromunpairedmedicalimagesand text. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 3876–3887. Wu, C.; Zhang, ...

2022