Pith. sign in

REVIEW 3 major objections 6 minor 44 references

A lightweight adapter turns frozen whole-volume CT models into anatomy-aware vision-language systems that keep global context and beat both global and fine-grained baselines on zero-shot finding classification.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 10:50 UTC pith:U2L6A7IY

load-bearing objection Solid lightweight adapter paper: real gains over frozen CT VLMs and matched adapters, but the “beats existing fine-grained methods” line only holds for frozen proxies, which the authors already admit in Limitations. the 3 major comments →

arxiv 2607.27154 v1 pith:U2L6A7IY submitted 2026-07-29 cs.CV cs.AI

Anatomy Contextualized Adaption of CT Foundation Models

Roshan Kenia , Stephanie L McNamara , William Lotter This is my paper
classification cs.CV cs.AI
keywords CT foundation modelsvision-language pre-trainingfine-grained alignmentanatomy contextualizationzero-shot classificationinter-anatomy transformerparameter-efficient adaptation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Whole-volume CT vision-language models compress an entire scan into one embedding, which washes out organ-level signal. Pure fine-grained methods fix that by matching each organ to its report text, but they throw away the cross-organ context clinicians use and usually train everything from scratch. This paper introduces Anatomy Contextualized Adaptation (ACA): keep the foundation model frozen, carve its feature maps into organ embeddings with a standard segmenter, let a small transformer mix information across organs, and train only that adapter plus projection heads with both per-organ and full-report contrastive losses. On two public CT datasets and two backbones, ACA improves zero-shot finding classification over the original models and over frozen-embedding versions of prior fine-grained designs, while training in under an hour once embeddings are cached. The learned attention also tracks anatomically sensible organ-to-organ links, and an optional finding-specific pooling step can squeeze out a bit more accuracy at inference.

Core claim

Adapting frozen CT foundation representations with anatomy decomposition, an inter-anatomy transformer, and joint anatomy-level plus scan-level text alignment yields stronger zero-shot finding classification than either whole-volume foundation models or fine-grained adapters that lack cross-anatomy context, at a fraction of the cost of training fine-grained models from scratch.

What carries the argument

Anatomy Contextualized Adaptation (ACA): TotalSegmentator-based anatomy tokens from a frozen CT backbone, refined by a transformer with spatial-position and anatomy-type embeddings, then aligned via combined per-anatomy and mean-pooled scan-level contrastive losses to report text.

Load-bearing premise

The paper treats frozen-embedding reimplementations of prior fine-grained methods as fair stand-ins for that literature, and assumes automated organ masks plus LLM-extracted organ text are good enough supervision.

What would settle it

Retrain the original end-to-end fine-grained methods on the same Merlin and CT-RATE splits and check whether they still lose to ACA on the same zero-shot finding AUROC tables; if they clearly win, the comparative claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Frozen CT foundation models can be upgraded to organ-level vision-language alignment without full backbone retraining.
  • Cross-organ attention is especially useful for relative findings such as organomegaly and multi-site vascular calcification.
  • Caching backbone embeddings makes anatomy-aware adaptation practical in under an hour of training.
  • Inference-time pooling restricted to finding-relevant organs can further lift zero-shot scores when global scan supervision is weak.
  • The same adapter design can be swapped onto other CT foundation backbones because it is decoupled from the frozen encoder.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the adapter truly recovers most of LoRA end-to-end gain, hospitals could ship anatomy-aware updates as small plug-ins rather than re-pretraining foundation models.
  • Weak self-attention on the diagonal suggests future designs might drop identity reinforcement and spend capacity only on inter-organ routing.
  • Extending the same contextualized tokens to report generation or outcome prediction is a natural next test the paper leaves open.
  • Errors in segmentation vocabulary or LLM organ labels would hit rare or out-of-taxonomy findings first, so those cases are the stress test for clinical deployment.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Anatomy Contextualized Adaptation (ACA), a lightweight module that adapts frozen CT vision–language foundation models (Merlin, CT-CLIP) for anatomy-level alignment without end-to-end retraining. TotalSegmentator masks are used to pool backbone feature maps into 44 anatomy embeddings; an inter-anatomy transformer with spatial-centroid and anatomy-type encodings contextualizes them; training combines a soft-target anatomy-level contrastive loss (Lanatomy, Eq. 2–4) with a scan-level report loss (Lscan, Eq. 5–6). On Merlin and CT-RATE, ACA improves zero-shot finding AUROC over frozen global VLMs and over several frozen-embedding fine-grained/global adapters (MLP, adapted fVLM/ViSD-Boost, Spatial Transformer), in- and (on 7 shared findings) out-of-distribution (Table 1), with ablations of loss terms and anatomy-guided pooling (Table 2), attention visualizations (Fig. 3), and a qualitative organomegaly case (Fig. 4). Training is reported as under one hour once embeddings are cached; code is released.

Significance. If the narrower empirical claim holds—that a cheap adapter on frozen CT foundation features can recover anatomy-level signal, add cross-anatomy context, and beat both whole-volume baselines and other frozen adapters on zero-shot finding classification—the work is a useful, practical contribution to CT foundation-model adaptation. Strengths include matched training setups, explicit global vs fine-grained ablations, bootstrap uncertainty, per-finding tables, a LoRA end-to-end check suggesting frozen adaptation captures most of the gain, and released code. The compute-efficiency angle is timely. Significance is tempered by the fact that comparisons to “existing fine-grained methods” are to frozen-embedding reimplementations rather than published end-to-end FVLP systems, so the paper’s main advance is best read as parameter-efficient anatomical contextualization of existing backbones, not as displacing the FVLP literature.

major comments (3)
  1. [Abstract; §4.3; §5; Table 1; Limitations] Abstract, §5 opening, and Table 1 caption/claim state that ACA “consistently outperforms … existing fine-grained methods.” §4.3 and Limitations correctly state that fVLM† and ViSD-Boost† are frozen-embedding adaptations of their pooling/VQ modules trained only with Lanatomy, not the original end-to-end systems. That matched-adapter comparison is scientifically useful, but it does not support the broader literature-beating phrasing. Please align Abstract, Results, and Conclusion with the Limitations wording (e.g., “frozen-embedding adaptations of fine-grained alignment modules”), or add true end-to-end FVLP baselines on the same splits. As written, Table 1’s strongest comparative reading is overstated relative to the experimental design.
  2. [§3.3 Eq. (2)–(4); Fig. 5; Limitations] Lanatomy depends on LLM-extracted per-anatomy text and binary normality labels for the soft target T (Eq. 2; Fig. 5). These labels are not validated against radiologist annotations, yet they directly shape positives (normal–normal and same-patient pairs) and which anatomies enter the sum in Eq. 4. A brief sensitivity check (e.g., hard diagonal only; subsample of human-audited normality; or ablating the normal–normal term) would show whether reported anatomy-level gains are robust to this supervision noise. Without it, part of the gap vs global baselines could reflect easier soft targets rather than better visual representations.
  3. [§4.2; Table 1; Tables 11–14] Out-of-distribution evaluation is restricted to 7 shared findings (Tables 13–14), while in-distribution macros use 30 (Merlin) and 18 (CT-RATE) findings. Macro averages in Table 1 therefore mix incompatible finding sets across columns. Please report overlapping-7 macros side-by-side for all in-distribution models (partially present in Appendix Tables 11–12) in the main table, and avoid implying full cross-dataset generalization beyond those 7 labels.
minor comments (6)
  1. [Title; Abstract] Title and running text use “Adaption”; standard spelling is “Adaptation.” Fix throughout for professionalism.
  2. [Fig. 2] Fig. 2 mixes Merlin-only, overlapping, and CT-RATE-only findings on one axis; a clearer split or ordering (e.g., shared findings first) would help readers parse green/red deltas.
  3. [Table 2; Appendix B] Anatomy-guided inference uses α tuned on validation macro-AUROC (Appendix B). State the selected α per dataset in the main text or Table 2 caption so the “+ Anatomy-Guided” rows are reproducible without the appendix.
  4. [§3.3 Eq. (5)] Eq. (5) writes ftext_scan in one place and f txt_scan elsewhere; unify notation with the rest of §3.3.
  5. [§6 Discussion] LoRA comparison in Discussion (0.8159 vs 0.8081) does not match Table 1’s ACA Merlin number (0.8213). Clarify which checkpoint/pooling variant is compared so the “majority of end-to-end benefit” claim is numerically consistent.
  6. [§2 Related Work] Cite and briefly position related organ-token / cross-anatomy contrastive work more carefully (e.g., CA-GCL is in the bibliography but barely discussed relative to the inter-anatomy transformer claim).

Circularity Check

0 steps flagged

No significant circularity: empirical contrastive adaptation evaluated on held-out organizer labels, not a derivation that re-encodes its inputs as predictions.

full rationale

ACA is a lightweight adapter trained with anatomy- and scan-level contrastive losses on frozen Merlin/CT-CLIP embeddings, then scored by zero-shot AUROC on held-out Merlin and CT-RATE test splits using each dataset’s ground-truth finding labels and fixed text prompts. Soft targets (Eq. 2) use LLM normality and same-patient pairs only inside the training objective; they do not define the test metric. Attention heatmaps (Fig. 3) and the splenomegaly case study (Fig. 4) are post-hoc interpretations, not load-bearing predictions. Ablations (Table 2) and baselines (MLP, Spatial Transformer, adapted fVLM/ViSD-Boost) are independent architectural controls, not self-definitional reductions. Comparative phrasing against “existing fine-grained methods” is a scope/fairness issue (frozen proxies vs end-to-end originals), not circularity. No fitted parameter is renamed as a first-principles prediction, and no uniqueness theorem or ansatz is imported via self-citation to force the result. The derivation chain is ordinary empirical ML against external benchmarks.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 2 invented entities

The result rests on standard contrastive VL learning plus domain tooling (TotalSegmentator, frozen Merlin/CT-CLIP encoders, LLM report parsing). No new physical entities. Free parameters are ordinary ML hyperparameters and a few design choices (λ=1, 44 grouped anatomies, temperatures, anatomy-guided α). Load-bearing domain assumptions are segmentation quality, LLM extraction fidelity, and fairness of frozen-embedding baseline adaptations.

free parameters (5)
  • λ (weight on Lscan) = 1
    Set to 1 without sweep in main experiments; ablations remove terms entirely rather than tuning λ continuously.
  • contrastive temperatures τ, τscan = init 0.07
    Learnable, init 0.07, clamped to [0.001, 0.5]; standard CLIP-style free parameters.
  • anatomy-guided blend α = selected on val set
    Chosen on validation from {0,0.2,...,1.0} to maximize macro AUROC for guided pooling variant.
  • transformer depth/width/heads and projection dims = Table 6 values
    Architecture hyperparameters (2 layers, 4 heads, hidden 1024, proj 512→256) chosen by authors; not derived.
  • 44-group TotalSegmentator taxonomy = 44 groups
    Manual condensation of 117 TS classes into 44 groups (Appendix Table 3) defines the token vocabulary.
axioms (6)
  • domain assumption Frozen Merlin/CT-CLIP intermediate feature maps retain sufficient anatomy-local signal when mean-pooled under TotalSegmentator masks.
    Core of embedding construction (§3.1); if features are only globally pooled semantics, anatomy tokens cannot recover fine-grained findings.
  • domain assumption TotalSegmentator segmentations are accurate enough on Merlin and CT-RATE for presence masks and centroids.
    Absent/incorrect masks drop or misassign anatomies (§3.1); paper excludes empty masks but does not quantify segmentation error impact.
  • domain assumption Qwen3-4B-Instruct extractions of per-anatomy descriptions and normality labels are adequate soft-supervision targets.
    Used for Lanatomy soft matrix T and skipping all-normal anatomies (§3.3, Fig. 5); acknowledged as possible label noise in Limitations.
  • domain assumption Symmetric InfoNCE-style contrastive alignment with soft targets is a valid training objective for zero-shot prompt classification at test time.
    Standard VL assumption linking train losses (Eqs. 3–6) to cosine scoring vs hand-written finding prompts (Appendix B).
  • ad hoc to paper Adapted frozen-embedding fVLM query pooling and ViSD-Boost VQ-VAE pipelines are fair compute-matched fine-grained baselines.
    Section 4.3 and Limitations explicitly redefine prior methods; comparative claim depends on this design choice.
  • standard math Standard transformer attention and ℓ2-normalized cosine similarity behave as usual (linear algebra / ML background).
    Used throughout inter-anatomy transformer and losses without novel proof obligations.
invented entities (2)
  • Anatomy Contextualized Adaptation (ACA) module no independent evidence
    purpose: Name for the trainable stack: anatomy pooling from frozen FM + type/spatial embeddings + inter-anatomy transformer + dual projection/losses.
    Engineering construct, not a new biological or physical entity; independent evidence is the empirical tables, not an external measurement of a postulated object.
  • Inter-anatomy spatial position encoding pa (44-D normalized centroid distances) independent evidence
    purpose: Supply geometric relationships among organs to the transformer.
    Defined in §3.1; a featurization choice rather than a discovered latent organ.

pith-pipeline@v1.2.0-daily-grok45 · 26747 in / 3838 out tokens · 83279 ms · 2026-07-30T10:50:36.104077+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Anatomy Contextualized Adaption of CT Foundation Models." pith.science (2026). https://pith.science/paper/U2L6A7IY

@misc{pith2026260727154,
  author       = {Pith},
  title        = {Pith review of: Anatomy Contextualized Adaption of CT Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U2L6A7IY}},
  note         = {Machine review of arXiv:2607.27154}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

CT vision-language foundation models have demonstrated promising performance across downstream tasks, but are typically trained with whole-volume representations that dilute fine-grained anatomical signals. Fine-grained vision-language pre-training addresses this by aligning anatomy-level visual features with anatomy-specific text, but in doing so discards the global context that whole-volume models provide. Furthermore, existing fine-grained approaches train from scratch, making them computationally expensive. We introduce Anatomy Contextualized Adaptation (ACA), a lightweight framework that adapts frozen CT foundation model representations for anatomy-level vision-language alignment while enhancing global contextualization. ACA uses TotalSegmentator to decompose CT volumes into anatomy-level embeddings, which are refined via a transformer that captures cross-anatomy relationships, and aligned to both per-anatomy and scan-level text extracted from radiology reports. Evaluated on Merlin and CT-RATE, ACA consistently outperforms both the frozen foundation model baselines and existing fine-grained methods in zero-shot finding classification, while requiring less than one hour of training once embeddings are cached. The attention weights learned by ACA's inter-anatomy transformer additionally indicate plausible cross-anatomy context routing. Altogether, these results support ACA as a lightweight approach for adapting CT foundation models to anatomically grounded vision-language alignment while preserving and enhancing global anatomical context.

Figures

Figures reproduced from arXiv: 2607.27154 by Roshan Kenia, Stephanie L McNamara, William Lotter.

Figure 1
Figure 1. Figure 1: Overview of ACA. Anatomy-level visual embeddings are constructed from a frozen CT foundation model using TotalSegmentator segmentations, while anatomy￾level text embeddings are extracted from radiology reports using an LLM and the corresponding frozen text encoder. An inter-anatomy transformer, augmented with spatial position and anatomy type embeddings, contextualizes the anatomy embeddings across the ful… view at source ↗
Figure 2
Figure 2. Figure 2: Per-finding AUROC comparison between the original models and the ACA anatomy-guided model for In-Distribution zero-shot evaluation. Green lines represent improvement over the baseline, Red lines represent regression. * These findings’ names from CT-RATE are simplified to match with Merlin: Coronary Artery Wall Calcifica￾tion = Coronary Calcification, Arterial Wall Calcification = Atherosclerosis. \mathcal … view at source ↗
Figure 3
Figure 3. Figure 3: Each heatmap shows the mean attention weights between fifteen major anatom￾ical structures, averaged over all transformer layers, attention heads, and test scans. Left: ACA model trained on Merlin and evaluated on the Merlin test set. Right: ACA model trained on CT-RATE and evaluated on the CT-RATE validation set. Sample counts per query anatomy reflect each respective dataset. of-distribution AUROC improv… view at source ↗
Figure 4
Figure 4. Figure 4: Two CT scans from the Merlin dataset. The right patient has splenomegaly and the left does not. Despite the spleens appearing of comparable size in isolation, ACA correctly identifies the right patient as positive while all fine-grained baselines predict negative. The relative proportions of surrounding organs provide a discriminative signal that single-organ embeddings cannot capture. force anatomy identi… view at source ↗
Figure 5
Figure 5. Figure 5: Prompt templates used for structured anatomy-level information extraction from CT radiology reports. Since reports describe findings globally rather than per or￾gan, a three-stage pipeline is applied. The mention detection prompt screens whether a given anatomy is discussed at all, avoiding spurious extractions for absent organs. The information extraction prompt isolates organ-specific descriptive text, u… view at source ↗
Figure 6
Figure 6. Figure 6: Example anatomy-specific findings extracted from a radiology report using Qwen3-4B-Instruct, along with the corresponding binary normality label used in the soft-target contrastive loss described in Section 3.3. Anatomical structures with no reported findings receive a default normal description [PITH_FULL_IMAGE:figures/full_fig_p023_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Example positive and negative text prompts used for zero-shot finding clas￾sification. For each finding, a set of positive prompts describing the pathology and negative prompts describing normal appearance are encoded by the frozen text en￾coder and averaged to obtain class-level embeddings f + and f − [PITH_FULL_IMAGE:figures/full_fig_p024_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 5 linked inside Pith

  1. [1]

    In: Proceed- ings of the 58th annual meeting of the association for computational linguistics

    Abnar, S., Zuidema, W.: Quantifying attention flow in transformers. In: Proceed- ings of the 58th annual meeting of the association for computational linguistics. pp. 4190–4197 (2020)

    2020

  2. [2]

    arXiv preprint arXiv:2106.08254 (2021)

    Bao, H., Dong, L., Piao, S., Wei, F.: Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254 (2021)

    Pith/arXiv arXiv 2021

  3. [3]

    medRxiv (2025)

    Beeche, C., Kim, J., Tavolinejad, H., Zhao, B., Sharma, R., Duda, J., Gee, J., Dako, F., Verma, A., Morse, C., et al.: A pan-organ vision-language model for generalizable 3d ct representations. medRxiv (2025)

    2025

  4. [4]

    Nature pp

    Blankemeier, L., Kumar, A., Cohen, J.P., Liu, J., Liu, L., Van Veen, D., Gardezi, S.J.S., Yu, H., Paschali, M., Chen, Z., et al.: Merlin: a computed tomography vision–language foundation model and dataset. Nature pp. 1–11 (2026)

    2026

  5. [5]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Cao, W., Zhang, J., Shui, Z., Wang, S., Chen, Z., Li, X., Lu, L., Ye, X., Zhang, Q., Liang, T., et al.: Boosting vision semantic density with anatomy normality mod- eling for medical vision-language pre-training. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 23041–23050 (2025)

    2025

  6. [6]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Cao, W., Zhang, J., Xia, Y., Mok, T.C., Li, Z., Ye, X., Lu, L., Zheng, J., Tang, Y., Zhang, L.: Bootstrapping chest ct image understanding by distilling knowl- edge from x-ray expert models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11238–11247 (2024)

    2024

  7. [7]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 9650–9660 (2021)

    2021

  8. [8]

    In: proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    Carreira, J., Zisserman, A.: Quo vadis, action recognition? a new model and the kinetics dataset. In: proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 6299–6308 (2017)

    2017

  9. [9]

    Advances in Neural Information Processing Systems35, 16664–16678 (2022)

    Chen, S., Ge, C., Tong, Z., Wang, J., Song, Y., Wang, J., Luo, P.: Adaptformer: Adapting vision transformers for scalable visual recognition. Advances in Neural Information Processing Systems35, 16664–16678 (2022)

    2022

  10. [10]

    In: International conference on machine learning

    Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for con- trastive learning of visual representations. In: International conference on machine learning. pp. 1597–1607. PmLR (2020)

    2020

  11. [11]

    Nature Communications (2025)

    Gao, Z., Zhang, G., Liang, H., Liu, J., Ma, L., Wang, T., Guo, Y., Chen, Y., Yan, Z., Chen, X., et al.: A lung ct vision foundation model facilitating disease diagnosis and medical imaging. Nature Communications (2025)

    2025

  12. [12]

    In: European Conference on Com- puter Vision

    Hamamci, I.E., Er, S., Sekuboyina, A., Simsar, E., Tezcan, A., Simsek, A.G., Esirgun, S.N., Almas, F., Doğan, I., Dasdelen, M.F., et al.: Generatect: Text- conditional generation of 3d chest ct volumes. In: European Conference on Com- puter Vision. pp. 126–143. Springer (2024)

    2024

  13. [13]

    Nature Biomedical Engineering pp

    Hamamci, I.E., Er, S., Wang, C., Almas, F., Simsek, A.G., Esirgun, S.N., Dogan, I., Durugol, O.F., Hou, B., Shit, S., et al.: Generalist foundation models from a multimodal dataset for 3d computed tomography. Nature Biomedical Engineering pp. 1–19 (2026) 16 R. Kenia et al

    2026

  14. [14]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 16000–16009 (2022)

    2022

  15. [15]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 9729–9738 (2020)

    2020

  16. [16]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    He, X., Li, C., Zhang, P., Yang, J., Wang, X.E.: Parameter-efficient model adapta- tion for vision transformers. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 37, pp. 817–825 (2023)

    2023

  17. [17]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    He, Y., Guo, P., Tang, Y., Myronenko, A., Nath, V., Xu, Z., Yang, D., Zhao, C., Si- mon, B., Belue, M., et al.: Vista3d: A unified segmentation foundation model for 3d medical imaging. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 20863–20873 (2025)

    2025

  18. [18]

    Iclr1(2), 3 (2022)

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: Lora: Low-rank adaptation of large language models. Iclr1(2), 3 (2022)

    2022

  19. [19]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Huang, S.C., Shen, L., Lungren, M.P., Yeung, S.: Gloria: A multimodal global-local representation learning framework for label-efficient medical image recognition. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 3942–3951 (2021)

    2021

  20. [20]

    arXiv preprint arXiv:2404.15272 (2024)

    Lin, J., Xia, Y., Zhang, J., Yan, K., Cao, K., Lu, L., Luo, J., Zhang, L.: Ct-glip: 3d grounded language-image pretraining with ct scans and radiology reports for full-body scenarios. arXiv preprint arXiv:2404.15272 (2024)

    arXiv 2024

  21. [21]

    Advances in neural information pro- cessing systems35, 36889–36901 (2022)

    Liu, Y.C., Ma, C.Y., Tian, J., He, Z., Kira, Z.: Polyhistor: Parameter-efficient multi-task adaptation for dense vision tasks. Advances in neural information pro- cessing systems35, 36889–36901 (2022)

    2022

  22. [22]

    In: European conference on computer vision

    Müller, P., Kaissis, G., Zou, C., Rueckert, D.: Joint learning of localized represen- tations from medical images and reports. In: European conference on computer vision. pp. 685–701. Springer (2022)

    2022

  23. [23]

    Nature Communications16(1), 1523 (2025)

    Niu, C., Lyu, Q., Carothers, C.D., Kaviani, P., Tan, J., Yan, P., Kalra, M.K., Whitlow, C.T., Wang, G.: Medical multimodal multitask foundation model for lung cancer screening. Nature Communications16(1), 1523 (2025)

    2025

  24. [24]

    arXiv preprint arXiv:2501.09001 (2025)

    Pai, S., Hadzic, I., Bontempi, D., Bressem, K., Kann, B.H., Fedorov, A., Mak, R.H., Aerts, H.J.: Vision foundation models for computed tomography. arXiv preprint arXiv:2501.09001 (2025)

    Pith/arXiv arXiv 2025

  25. [25]

    In: International conference on machine learning

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PmLR (2021)

    2021

  26. [26]

    arXiv preprint arXiv:2501.14548 (2025)

    Shui, Z., Zhang, J., Cao, W., Wang, S., Guo, R., Lu, L., Yang, L., Ye, X., Liang, T., Zhang, Q., et al.: Large-scale and fine-grained vision-language pre-training for enhanced ct image understanding. arXiv preprint arXiv:2501.14548 (2025)

    Pith/arXiv arXiv 2025

  27. [27]

    Advances in neural information processing systems35, 33536–33549 (2022)

    Wang, F., Zhou, Y., Wang, S., Vardhanabhuti, V., Yu, L.: Multi-granularity cross- modal alignment for generalized medical visual representation learning. Advances in neural information processing systems35, 33536–33549 (2022)

    2022

  28. [28]

    IEEE Transactions on Neural Networks and Learning Systems (2025)

    Wang, H., Guo, S., Ye, J., Deng, Z., Cheng, J., Li, T., Chen, J., Su, Y., Huang, Z., Shen, Y., et al.: Sam-med3d: a vision foundation model for general-purpose seg- mentation on volumetric medical images. IEEE Transactions on Neural Networks and Learning Systems (2025)

    2025

  29. [29]

    In: Proceedings of the 2022 Conference on Empir- ical Methods in Natural Language Processing

    Wang, Z., Wu, Z., Agarwal, D., Sun, J.: Medclip: Contrastive learning from un- paired medical images and text. In: Proceedings of the 2022 Conference on Empir- ical Methods in Natural Language Processing. pp. 3876–3887 (2022) Anatomy Contextualized Adaption 17

    2022

  30. [30]

    Radiology: Artificial Intelligence 5(5), e230024 (2023)

    Wasserthal, J., Breit, H.C., Meyer, M.T., Pradella, M., Hinck, D., Sauter, A.W., Heye, T., Boll, D.T., Cyriac, J., Yang, S., et al.: Totalsegmentator: robust segmen- tation of 104 anatomic structures in ct images. Radiology: Artificial Intelligence 5(5), e230024 (2023)

    2023

  31. [31]

    Nature4, 1182– 1183 (2024)

    Wen, J.: Biological age shows that no organ system is an island. Nature4, 1182– 1183 (2024)

    2024

  32. [32]

    Nature Communications16(1), 7866 (2025)

    Wu, C., Zhang, X., Zhang, Y., Hui, H., Wang, Y., Xie, W.: Towards generalist foun- dation model for radiology by leveraging web-scale 2d&3d medical data. Nature Communications16(1), 7866 (2025)

    2025

  33. [33]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Wu, C., Zhang, X., Zhang, Y., Wang, Y., Xie, W.: Medklip: Medical knowledge enhanced language-image pre-training for x-ray diagnosis. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 21372–21383 (2023)

    2023

  34. [34]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Wu,L.,Zhuang,J.,Chen,H.:Voco:Asimple-yet-effectivevolumecontrastivelearn- ing framework for 3d medical image analysis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 22873–22882 (2024)

    2024

  35. [35]

    In: European Conference on Com- puter Vision

    Xie, Y., Zhang, J., Xia, Y., Wu, Q.: Unimiss: Universal medical self-supervised learning via breaking dimensionality barrier. In: European Conference on Com- puter Vision. pp. 558–575. Springer (2022)

    2022

  36. [36]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Xie, Z., Zhang, Z., Cao, Y., Lin, Y., Bao, J., Yao, Z., Dai, Q., Hu, H.: Simmim: A simple framework for masked image modeling. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 9653–9663 (2022)

    2022

  37. [37]

    npj Digital Medicine8(1), 639 (2025)

    Xu, T., Hosseini, S., Anderson, C., Rinaldi, A., Krishnan, R.G., Martel, A.L., Goubran, M.: A generalizable 3d framework and model for self-supervised learning in medical imaging. npj Digital Medicine8(1), 639 (2025)

    2025

  38. [38]

    arXiv preprint arXiv:2505.09388 (2025)

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al.: Qwen3 technical report. arXiv preprint arXiv:2505.09388 (2025)

    Pith/arXiv arXiv 2025

  39. [39]

    arXiv preprint arXiv:2605.13544 (2026)

    Zhang, H., Liu, Y., Dai, D., Yang, J., Liu, Q., Xie, Y., Wang, P.: Ca-gcl: Cross- anatomy global-local contrastive learning for robust 3d medical image understand- ing. arXiv preprint arXiv:2605.13544 (2026)

    Pith/arXiv arXiv 2026

  40. [40]

    Yes" or

    Zhou, Z., Sodha, V., Pang, J., Gotway, M.B., Liang, J.: Models genesis. Medical image analysis67, 101840 (2021) 18 R. Kenia et al. A Preprocessing Table 3:Mapping of TotalSegmentator labels to grouped anatomical structures for fine-grained modeling. Idx TotalSegmentator Name Grouping Name Grp 1 spleen spleen 1 2 kidney_right kidney 2 3 kidney_left kidney ...

    2021

  41. [41]

    Precise extraction: Extract the descriptive information directly related to {anatomy} from the report

  42. [42]

    Specify anatomical details: If the report mentions specific areas, parts, or anatomical details of {anatomy}, make sure to include this information in the description

  43. [43]

    Concise and clear: Directly extract the report content, avoiding unnecessary explanations or background

  44. [44]

    {anatomy}: descriptive information

    Format requirement: Return the information in the format "{anatomy}: descriptive information" ensuring {anatomy} is used as the unified prefix. Even if the anatomy has multiple independent parts or lateral characteristics, treat it as a single anatomy and return one comprehensive description. Abnormality Classification Prompt {description} You are a profe...

    2048