Pith. sign in

REVIEW 3 major objections 5 minor 31 references

SCALPEL: Semantic Cross-modal Alignment via LLM-Powered Encoder Learning for Medical Vision-Language Representation

T0 review · 3 major / 5 minor · reviewed 2026-07-30 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SCALPEL converts medical LLMs into isotropic encoders and aligns them to chest X-rays with anatomy- and negation-aware penalties, reaching state-of-the-art retrieval, zero-shot classification, and VQA.

desk verdict Competent medical VLP systems paper: real failure modes, known pieces assembled cleanly, SOTA tables that partly rest on unvalidated NER labels. read the letter →

arxiv 2607.26885 v1 pith:ZH5SYXRW submitted 2026-07-29 cs.CV

Yunzhan Fu , Enyu Bao , Xiangyu Shen , Yihao Wu , Chunbo Jiang , Fangli Guan , Liqi Yan This is my paper · ORCID
classification cs.CV
keywords vision-languagepre-trainingmedicalLLMscontrastivelearningcross-modalretrievalanatomy-negationawareobjectiveradiologyreportszero-shotclassificationVQA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Medical vision-language models usually rely on short-context BERT-style text encoders that cannot fully absorb long, terminology-heavy radiology reports. Plugging in large medical language models should help, but generative LLMs collapse into anisotropic embeddings under ordinary contrastive loss, joint training is memory-prohibitive, and vanilla InfoNCE ignores laterality and negation, producing clinical hallucinations. SCALPEL first contrastively fine-tunes a medical LLM on reports so its embeddings become isotropic and bidirectional, then freezes it and caches text features offline so only the vision side and a light projector train under large batches. An Anatomy-Negation Aware Objective adds explicit penalties whenever high similarity is assigned to pairs that disagree on laterality or on whether a finding is present or absent. On MIMIC-CXR, CheXpert, IU X-Ray, SLAKE and VQA-RAD the resulting system leads reported baselines in cross-modal retrieval, zero-shot disease classification and medical visual question answering.

What carries the argument

Anatomy-Negation Aware Objective (ANAO): standard InfoNCE plus additive penalties that drive down similarity for batch pairs whose extracted anatomical laterality labels differ or whose negation vectors disagree on radiological findings.

What would settle it

Replace the external NER-derived laterality and negation labels with deliberately flipped or random labels and check whether retrieval R@1 and zero-shot F1 on MIMIC-CXR / CheXpert drop sharply; if performance is unchanged, ANAO is not doing the claimed work.

Watch

Extended reading notes

Core claim

A two-stage LLM-powered pipeline—Clinical Report Contrastive fine-tuning that turns a generative medical LLM into an isotropic bidirectional encoder, followed by asymmetric offline-cached alignment under an Anatomy-Negation Aware Objective—resolves anisotropy, memory cost and clinical hallucination and yields state-of-the-art medical vision-language representations.

Load-bearing premise

The anatomy and negation labels used by the penalty terms are assumed correct; they come from external clinical NER pipelines whose errors would push the embedding space the wrong way.

Editorial extensions

If this is right

  • Existing medical VLP pipelines can swap in a CRC-tuned LLM encoder plus ANAO and obtain measurable gains in I2T/T2I recall without redesigning the vision backbone.
  • Zero-shot disease classification and medical VQA improve when the shared space is forced to respect laterality and negation, reducing clinically dangerous mismatches.
  • Offline text-feature caching makes billion-parameter medical LLMs practical for large-batch contrastive training on standard hardware.
  • The same modular recipe can be dropped into other medical image-report collections beyond chest X-ray.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If NER quality is the binding constraint, replacing the external pipeline with an end-to-end LLM that jointly extracts laterality/negation and produces the embedding could remove the cascading-error failure mode the authors flag.
  • The same anisotropy-plus-negation problem likely appears in other long-document medical modalities (pathology, discharge summaries); CRC+ANAO-style adaptation may transfer there with little change.
  • Freezing the LLM after CRC leaves early cross-modal attention unexplored; parameter-efficient unfreezing under the same ANAO loss is a natural next measurement.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. SCALPEL is a two-stage medical vision-language pre-training framework that converts a generative medical LLM (PMC-LLaMA-13B) into a discriminative isotropic text encoder via Clinical Report Contrastive (CRC) fine-tuning (bidirectional MNTP + SimCSE with LoRA), then performs asymmetric cross-modal alignment by freezing the LLM, caching text embeddings offline, and training only the vision encoder (DINOv2 ViT-B/16) plus a lightweight projector under an Anatomy-Negation Aware Objective (ANAO). ANAO augments InfoNCE with hinge penalties on pairs whose externally extracted anatomical laterality labels or negation vectors disagree (Eqs. 6–8). Experiments on MIMIC-CXR retrieval, zero-shot classification (IU X-Ray, CheXpert), and medical VQA (SLAKE, VQA-RAD) report state-of-the-art or consistently improved numbers versus MedCLIP, BioViL-T, BiomedCLIP, MedProbCLIP, MMedPO and related baselines, with ablations (Fig. 3) and hyperparameter sweeps (Table 3) supporting the contribution of CRC, caching, and ANAO.

Significance. The work targets three genuine, well-motivated bottlenecks in medical VLP: limited context of BERT-scale encoders on long radiology reports, anisotropic collapse of generative LLM embeddings under contrastive losses, and clinical hallucinations around laterality and negation. The asymmetric caching design yields a clear efficiency win (reported ~76% FLOPs reduction vs. a CXR-BERT baseline while retaining a large LLM text tower), and the modular “+SCALPEL” results show the components can be dropped into existing pipelines. If the gains are robust, the paper offers a practical recipe for bringing billion-parameter medical LLMs into contrastive VLP without full end-to-end joint training. Strengths include multi-benchmark evaluation, component ablations, efficiency numbers, and an explicit hyperparameter sensitivity table.

major comments (3)
  1. [§3.3, Eqs. (6)–(8); Fig. 3; §5] The central attribution of gains to ANAO (Eqs. 6–8, §3.3) rests on anatomical labels a_i and negation vectors n_i produced by external clinical NER/negation pipelines [23,21] that are treated as ground truth. No precision/recall, error analysis, or inter-annotator comparison of laterality or negation extraction is reported on MIMIC-CXR (or any held-out sample), and Fig. 3 only contrasts InfoNCE vs. ANAO under the same uncharacterized labels. Section 5 itself flags cascading errors from this step. Because L_anat and L_neg directly reshape the embedding space, systematic NER mistakes (common on free-text reports) would push representations in the wrong direction; the SOTA margins in Tables 1–2 cannot be confidently credited to clinically correct supervision without a noise study, clean-vs-noisy ablation, or at least reported extraction accuracy on a labeled subset.
  2. [Tables 1–2; Fig. 3; Table 3] Tables 1–2 and Fig. 3 report point estimates only; there are no error bars, multi-seed means/stds, or statistical significance tests. Several absolute gains are modest (e.g., +1–2 points R@1 on already competitive baselines; occasional metric drops such as BioViL-T T2I R@5 and IU X-Ray F1 under +SCALPEL). Without variance estimates it is difficult to judge whether the claimed SOTA ranking of SCALPEL* is stable or sensitive to initialization and data order, especially given that α, λ_a, λ_n are selected for peak table numbers (Table 3).
  3. [Fig. 3(a); §4.4] The ablation in Fig. 3a jumps across simultaneous changes of text encoder (Transformer / CXR-BERT / PubMedBERT / PMC-LLaMA), vision backbone (ViT-B/16 vs DeiT3-L), and loss (InfoNCE vs ANAO). A fuller factorial or staged isolation (CRC alone, caching alone, ANAO alone, then combinations) on a fixed backbone would make the contribution of each claimed ingredient clearer and would better support the narrative that CRC resolves anisotropy and ANAO resolves hallucinations.
minor comments (5)
  1. [Fig. 1] Fig. 1’s similarity-contribution bar chart is hard to read in grayscale and the exact embedding-component decomposition method is not specified in the caption or text.
  2. [§3.1–3.2] Notation: S_ij is defined with temperature τ in Eq. 1, yet τ_s appears only in the SimCSE term; a single temperature table or explicit statement of which temperature is used at inference would help reproducibility.
  3. [§4.2] Implementation details give LoRA rank/α and training steps for CRC but omit the exact mean-pooling layer, whether [EOS]/last-token or mean-over-tokens is used for h_i, and the projector hidden sizes—small but necessary for exact reimplementation.
  4. [§2] Related work could more clearly position against concurrent LLM2CLIP-style caption-contrastive LLM encoders and against other medical negation-aware losses; a short paragraph on how ANAO differs from soft-target or ontology-based methods (MedCLIP, MedKLIP) would strengthen novelty framing.
  5. [§3.1; §4.3] Minor typos / formatting: “F ramework” spacing in §3.1 heading; “V alidation” in §4.3; arXiv-style “Title Suppressed Due to Excessive Length” headers should be cleaned for camera-ready.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: standard empirical VLP pipeline; losses and metrics are independent of each other by construction.

full rationale

SCALPEL’s claimed SOTA numbers are ordinary held-out empirical results (Recall@k, ACC/F1, VQA open/closed) on public benchmarks (MIMIC-CXR, CheXpert, IU X-Ray, SLAKE, VQA-RAD). The training objectives—MNTP (Eq. 2), SimCSE (Eq. 3), InfoNCE (Eq. 1), and ANAO penalties L_anat/L_neg (Eqs. 6–8)—are optimization losses, not algebraic restatements of those metrics. Anatomy/negation labels a_i and n_i come from external NER pipelines and act as fixed supervision side-information; they do not define the reported retrieval or classification scores. Hyperparameter selection of α, λ_a, λ_n (Table 3) is ordinary tuning, not a fitted identity renamed as a prediction. There is no self-definitional loop, no uniqueness theorem imported from the same authors, no load-bearing self-citation chain, and no renaming of a known closed-form result. The paper is a self-contained empirical ML contribution; circularity score is 0.

Assumptions & free parameters 6 free parameters · 5 assumptions · 2 invented entities

The central empirical claim rests on standard contrastive VLP assumptions, the validity of external clinical NLP labels for ANAO, the sufficiency of frozen LLM text features plus a small projector, and several hand-chosen loss weights and architecture choices. No new physical entities are postulated; the invented pieces are methodological objectives and training stages.

free parameters (6)
  • α (CRC loss mix of MNTP and SimCSE) = 0.5
    Balances masked-token and SimCSE terms in L_CRC; swept in Table 3 and set to 0.5 for best retrieval.
  • λ_a (anatomy penalty weight) = 0.1
    Scales L_anat inside ANAO; chosen by sensitivity sweep on MIMIC-CXR retrieval.
  • λ_n (negation penalty weight) = 0.15
    Scales L_neg inside ANAO; chosen by the same sweep.
  • contrastive temperatures τ / τ_s
    Temperature-scaled cosine similarities in InfoNCE and SimCSE; standard free knobs affecting gradient scale.
  • LoRA rank r and α_LoRA = r=8, α=16
    Controls capacity of CRC adaptation on PMC-LLaMA; fixed at r=8, α=16 without theoretical derivation.
  • ANAO hinge via max(0, S_ij) = hinge at 0
    Only high-similarity mismatched pairs are penalized; threshold-at-zero is a design choice, not derived.
assumptions (5)
  • domain assumption Symmetric InfoNCE on batch cosine similarities is a valid objective for medical image-report alignment.
    Eq. 1 is taken as the base loss throughout Stage 2; standard VLP assumption imported without proof of optimality for clinical reports.
  • domain assumption External clinical NER can supply reliable discrete anatomy labels a_i and negation vectors n_i for each report.
    §3.3 builds L_anat and L_neg directly on these labels; correctness of ANAO supervision depends on this pipeline ([23,21]).
  • domain assumption Replacing causal masks with bidirectional attention plus MNTP+SimCSE yields isotropic, discriminative report embeddings from a generative LLM.
    Stage 1 motivation and L_CRC (Eqs. 2–4); supported by SimCSE literature but assumed adequate for radiology anisotropy collapse.
  • ad hoc to paper Freezing the CRC-tuned LLM and aligning only E_V and A_proj to cached text features preserves enough cross-modal signal.
    Asymmetric Stage 2 design (§3.3, Eq. 5) chosen for memory; conclusion admits this blocks deep early fusion.
  • domain assumption Mean pooling of LLM token states is a sufficient report representation for contrastive alignment.
    Used for h_i in SimCSE and for cached t_i; not compared to alternatives like EOS or attention pooling in the main claim path.
invented entities (2)
  • Anatomy-Negation Aware Objective (ANAO)
    purpose: Add explicit penalties for high similarity when anatomy laterality or finding negation status disagrees.
    Defined in Eqs. 6–8 as L_InfoNCE + λ_a L_anat + λ_n L_neg; central methodological invention of the paper.
  • Clinical Report Contrastive (CRC) fine-tuning stage
    purpose: Convert generative PMC-LLaMA into a bidirectional isotropic report encoder before vision alignment.
    Stage 1 recipe combining bidirectional attention, MNTP, SimCSE, and LoRA on MIMIC reports; procedural construct rather than a physical entity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SCALPEL: Semantic Cross-modal Alignment via LLM-Powered Encoder Learning for Medical Vision-Language Representation." pith.science (2026). https://pith.science/paper/ZH5SYXRW

@misc{pith2026260726885,
  author       = {Pith},
  title        = {Pith review of: SCALPEL: Semantic Cross-modal Alignment via LLM-Powered Encoder Learning for Medical Vision-Language Representation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZH5SYXRW}},
  note         = {Machine review of arXiv:2607.26885}
}
read the original abstract

Vision-language pre-training (VLP) serves as a cornerstone for medical multimodal representation learning. However, existing medical VLP frameworks are often constrained by the limited context windows and shallow representational capacities of lightweight text encoders when processing lengthy, terminology-dense clinical reports. While integrating medical large language models (LLMs) offers unprecedented clinical reasoning capabilities, it introduces three major bottlenecks: (i) the anisotropic representational collapse of generative LLMs under standard contrastive objectives, (ii) the prohibitive memory overhead of joint end-to-end training with large batch sizes, and (iii) the medical hallucinations induced by vanilla contrastive losses that ignore fine-grained anatomical laterality and negation modifiers. To address these challenges, we propose \textbf{SCALPEL}, a \textbf{S}emantic \textbf{C}ross-modal \textbf{A}lignment framework via \textbf{L}LM-\textbf{P}owered \textbf{E}ncoder \textbf{L}earning. First, Clinical Report Contrastive fine-tuning converts a generative LLM into an isotropic encoder via domain-specific clinical text adaptation. Second, an asymmetric alignment strategy leverages offline feature caching to enable efficient training. Critically, we formulate an Anatomy-Negation Aware Objective that explicitly penalizes mismatched image-text pairs involving laterality confusion or false negations. Extensive experiments across MIMIC-CXR, CheXpert, and IU X-Ray benchmarks demonstrate that SCALPEL achieves state-of-the-art performance in cross-modal retrieval, zero-shot disease classification and medical visual question answering.

Figures

Figures reproduced from arXiv: 2607.26885 by the authors.

Figure 1
Figure 1. Vanilla medical contrastive learning models fail to assign differentiated scores to reports with opposite conclusions on MIMIC-CXR [13]. Integrating generative medical large models with massive parameters into the visual feature alignment training pipeline introduces several bottlenecks. (i) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the SCALPEL framework. Stage 1 (upper): CRC fine-tuning converts a generative medical LLM into a bidirectional text encoder via masked token prediction on radiology reports. Stage 2 (lower): the asymmetric alignment stage freezes the CRC-tuned LLM, pre-computes text features offline, and trains only the vision encoder and a lightweight cross-modal adapter under the ANAO loss [PITH_FULL_IMAGE:figures/ful… view at source ↗
Figure 3
Figure 3. Ablation study and efficiency analysis on MIMIC-CXR [13] and CheXpert[12]. Efficiency tests in [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 1 linked inside Pith

  1. [1]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Bannur, S., Hyland, S., Liu, Q., P´ erez-Garc ´ ıa, F., Ilse, M., Castro, D.C., Boeck- ing, B., Sharma, H., Bouzid, K., Thieme, A., Schwaighofer, A., Wetscherek, M., Lungren, M.P., Nori, A., Alvarez-Valle, J., Oktay, O.: Learning to exploit tem- poral structure for biomedical vision-language processing. In: Proceedings of the IEEE/CVF Conference on Comput...

    2023

  2. [2]

    In: European conference on computer vision

    Boecking, B., Usuyama, N., Bannur, S., Castro, D.C., Schwaighofer, A., Hyland, S., Wetscherek, M., Naumann, T., Nori, A., Alvarez-Valle, J., et al.: Making the most of text semantics to improve biomedical vision–language processing. In: European conference on computer vision. pp. 1–21. Springer (2022)

    2022

  3. [3]

    In: International Con- ference on Learning Representations

    Chun, S.: Improved probabilistic image-text representations. In: International Con- ference on Learning Representations. vol. 2024, pp. 39787–39815 (2024)

    2024

  4. [4]

    Journal of the American Medical Informatics Association23(2), 304–310 (2016)

    Demner-Fushman, D., Kohli, M.D., Rosenman, M.B., Shooshan, S.E., Rodriguez, L., Antani, S., Thoma, G.R., McDonald, C.J.: Preparing a collection of radiol- ogy examinations for distribution and retrieval. Journal of the American Medical Informatics Association23(2), 304–310 (2016)

    2016

  5. [5]

    In: Burstein, J., Doran, C., Solorio, T

    Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North Ameri- can Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)...

    2019

  6. [6]

    In: Proceedings of the IEEE/CVF Winter Con- ference on Applications of Computer Vision

    Elallaf, A., Zhang, Y., Masupalli, Y., Yang, J., Lee, Y., Cao, Z., Liang, G.: Med- ProbCLIP: Probabilistic adaptation of vision-language foundation model for reli- able radiograph-report retrieval. In: Proceedings of the IEEE/CVF Winter Con- ference on Applications of Computer Vision. pp. 1–10 (2026) Title Suppressed Due to Excessive Length 13

    2026

  7. [7]

    In: Proceedings of the 2021 conference on empirical methods in natural language processing

    Gao, T., Yao, X., Chen, D.: SimCSE: Simple contrastive learning of sentence em- beddings. In: Proceedings of the 2021 conference on empirical methods in natural language processing. pp. 6894–6910 (2021)

    2021

  8. [8]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 9729–9738 (2020)

    2020

Show all 31 references
  1. [9]

    International Conference on Learning Representations1(2), 3 (2022)

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: LoRA: Low-rank adaptation of large language models. International Conference on Learning Representations1(2), 3 (2022)

    2022

  2. [10]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    Huang, S.C., Shen, L., Lungren, M.P., Yeung, S.: GLoRIA: A multimodal global- local representation learning framework for label-efficient medical image recogni- tion. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 3942–3951 (October 2021)

    2021

  3. [11]

    Proceedings of the AAAI Conference on Artificial Intelligence40(7), 5131–5139 (Mar 2026)

    Huang, W., Wu, A., Yang, Y., Luo, X., Yang, Y., Naseem, U., Wang, C., Dai, Q., Dai, X., Chen, D., et al.: LLM2CLIP: Powerful language model unlocks richer cross-modality representation. Proceedings of the AAAI Conference on Artificial Intelligence40(7), 5131–5139 (Mar 2026)

    2026

  4. [12]

    In: Proceedings of the AAAI conference on artificial intelligence

    Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al.: CheXpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI conference on artificial in...

    2019

  5. [13]

    Scientific data6(1), 317 (2019)

    Johnson, A.E., Pollard, T.J., Berkowitz, S.J., Greenbaum, N.R., Lungren, M.P., Deng, C.y., Mark, R.G., Horng, S.: MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data6(1), 317 (2019)

    2019

  6. [14]

    Scientific data 5(1), 180251 (2018)

    Lau, J.J., Gayen, S., Ben Abacha, A., Demner-Fushman, D.: A dataset of clinically generated visual questions and answers about radiology images. Scientific data 5(1), 180251 (2018)

    2018

  7. [15]

    In: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S

    Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., Gao, J.: LLaV A-Med: Training a large language-and-vision assistant for biomedicine in one day. In: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S. (eds.) Advances in Neur...

    2023

  8. [16]

    In: Greenspan, H., Madabhushi, A., Mousavi, P., Salcudean, S., Duncan, J., Syeda- Mahmood, T., Taylor, R

    Lin, W., Zhao, Z., Zhang, X., Wu, C., Zhang, Y., Wang, Y., Xie, W.: PMC- CLIP: Contrastive Language-Image Pre-training Using Biomedical Documents. In: Greenspan, H., Madabhushi, A., Mousavi, P., Salcudean, S., Duncan, J., Syeda- Mahmood, T., Taylor, R. (eds.) Medical Image Com...

    2023

  9. [17]

    In: 2021 IEEE 18th international symposium on biomedical imaging (ISBI)

    Liu, B., Zhan, L.M., Xu, L., Ma, L., Yang, Y., Wu, X.M.: SLAKE: A semantically- labeled knowledge-enhanced dataset for medical visual question answering. In: 2021 IEEE 18th international symposium on biomedical imaging (ISBI). pp. 1650–1654. IEEE (2021)

    2021

  10. [18]

    Na- ture616(7956), 259–265 (2023)

    Moor, M., Banerjee, O., Abad, Z.S.H., Krumholz, H.M., Leskovec, J., Topol, E.J., Rajpurkar, P.: Foundation models for generalist medical artificial intelligence. Na- ture616(7956), 259–265 (2023)

    2023

  11. [19]

    In: Hegselmann, S., Parziale, A., Shanmugam, D., Tang, S., Asiedu, M.N., Chang, S., Hartvigsen, T., Singh, H

    Moor, M., Huang, Q., Wu, S., Yasunaga, M., Dalmia, Y., Leskovec, J., Zakka, C., Reis, E.P., Rajpurkar, P.: Med-Flamingo: a multimodal medical few-shot learner. In: Hegselmann, S., Parziale, A., Shanmugam, D., Tang, S., Asiedu, M.N., Chang, S., Hartvigsen, T., Singh, H. (eds.) ...

  12. [20]

    arXiv preprint arXiv:2304.07193 (2023)

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: DINOv2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023)

    2023 arXiv

  13. [21]

    Journal of imaging informatics in medicine38(3), 1297–1303 (2025)

    Su, Y., Babore, Y.B., Kahn Jr, C.E.: A large language model to detect negated expressions in radiology reports. Journal of imaging informatics in medicine38(3), 1297–1303 (2025)

    2025

  14. [22]

    NEJM AI 1(3), AIoa2300138 (2024)

    Tu, T., Azizi, S., Driess, D., et al.: Towards generalist biomedical AI. NEJM AI 1(3), AIoa2300138 (2024)

    2024

  15. [23]

    JAMIA Open8(4), ooaf094 (2025)

    Velamala, B., Sagheb Hossein Pour, E., Lin, M., Fan, J.W.: medspacyV: a graphical user interface for the open source medspaCy natural language processing package. JAMIA Open8(4), ooaf094 (2025)

    2025

  16. [24]

    In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing

    Wang, Z., Wu, Z., Agarwal, D., Sun, J.: MedCLIP: Contrastive learning from unpaired medical images and text. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. pp. 3876–3887 (2022)

    2022

  17. [25]

    Journal of the American Medical Informatics Association31(9), 1833–1843 (2024)

    Wu, C., Lin, W., Zhang, X., Zhang, Y., Xie, W., Wang, Y.: PMC-LLaMA: to- ward building open-source language models for medicine. Journal of the American Medical Informatics Association31(9), 1833–1843 (2024)

    2024

  18. [26]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    Wu, C., Zhang, X., Zhang, Y., Wang, Y., Xie, W.: MedKLIP: Medical knowledge enhanced language-image pre-training for X-ray diagnosis. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 21372– 21383 (October 2023)

    2023

  19. [27]

    In: Inter- national Conference on Medical Image Computing and Computer-Assisted Inter- vention

    You, K., Gu, J., Ham, J., Park, B., Kim, J., Hong, E.K., Baek, W., Roh, B.: CXR-CLIP: Toward large scale chest X-ray language-image pre-training. In: Inter- national Conference on Medical Image Computing and Computer-Assisted Inter- vention. pp. 101–111. Springer (2023)

    2023

  20. [28]

    NEJM AI2(1), AIoa2400640 (2025)

    Zhang, S., Xu, Y., Usuyama, N., Xu, H., Bagga, J., Tinn, R., et al.: A multimodal biomedical foundation model trained from fifteen million image–text pairs. NEJM AI2(1), AIoa2400640 (2025)

    2025

  21. [29]

    In: Machine learning for healthcare conference

    Zhang, Y., Jiang, H., Miura, Y., Manning, C.D., Langlotz, C.P.: Contrastive learn- ing of medical visual representations from paired images and text. In: Machine learning for healthcare conference. pp. 2–25. PMLR (2022)

    2022

  22. [30]

    Medical Image Analysis102, 103551 (2025)

    Zhao, Z., Liu, Y., Wu, H., Wang, M., Li, Y., Wang, S., Teng, L., Liu, D., Cui, Z., Wang, Q., et al.: CLIP in medical imaging: A survey. Medical Image Analysis102, 103551 (2025)

    2025

  23. [31]

    In: Proceedings of the 42nd International Conference on Machine Learning (ICML)

    Zhu, K., Xia, P., Li, Y., Zhu, H., Wang, S., Yao, H.: MMedPO: aligning medical vision-language models with clinical-aware multimodal preference optimization. In: Proceedings of the 42nd International Conference on Machine Learning (ICML). ICML’25, JMLR.org (2025)

    2025

Pith tools

Reviewed July 30, 2026 · model on record in the stance chip above.