Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Exploring the Capabilities of Large Language Model Encoders for Image-Text Retrieval in Chest X-rays

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM text encoders make chest X-ray retrieval reliable across messy report styles, outperforming BERT-based models and keeping performance when noisy hospital notes are added.

desk verdict A solid, believable medical-CLIP application of LLM encoders; the abstract overclaims bidirectional retrieval and the generated training variants lack clinician verification. read the letter →

arxiv 2509.15234 v2 pith:TZ5I2NXV submitted 2025-09-17 cs.CV

Hanbin Ko , Gihun Cho , Inhyeok Baek , Donguk Kim , Joonbeom Koo , Changi Kim , Dongheon Lee , Chang Min Park This is my paper
classification cs.CV
keywords vision-languagepretrainingchestX-rayLLMtextencoderimage-textretrievalclinicalreportheterogeneitysupervisedcontrastivelearningdomainadaptationCLIP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether large language model text encoders, after domain adaptation, can replace BERT-style encoders in chest X-ray image–text retrieval. The authors claim that an LLM text tower trained on stylistically diverse but clinically equivalent report variants produces representations that handle abbreviations, impression-only notes, and stylistic heterogeneity better than BERT, and that these representations transfer to a dual-tower vision–language model. They report that the resulting system improves retrieval accuracy and clinically oriented scores on both an internal test set and an external dataset, and that it degrades less than BERT-based models when noisy, abbreviation-rich hospital reports are added to training. The central message is that robustness to report style, not dataset scale alone, is the key to scaling medical vision–language pretraining.

What carries the argument

The central object is LLM2VEC4CXR, a domain-adapted LLM encoder. It removes the causal mask of a decoder-only LLM to allow bidirectional context, adds latent attention pooling to form the global sentence embedding, and is trained with masked next token prediction and supervised contrastive learning over report variants produced by two generative LLMs. The variants include paraphrases, sentence splits, prior-omitted versions, anatomical partitions, and Findings–Impression summarization pairs, making the embedding space invariant to style and abbreviation. In the multimodal stage, LLM2CLIP4CXR keeps this encoder mostly frozen and adapts it with low-rank adapters while training a vision tower a

What would settle it

A clinical audit of a random sample of generated variants (e.g., 200 paraphrases and splits) that finds a material factual error rate, such as reversed laterality or wrong severity, would undermine the equivalence assumption; a controlled retrieval experiment where a deliberately wrong variant is inserted as a positive pair and the model's error-detection accuracy drops would directly test the mechanism.

Watch

Extended reading notes

Core claim

The paper introduces LLM2VEC4CXR, an encoder for chest X-ray reports built by converting a decoder-only large language model into a bidirectional encoder—removing the causal attention mask, pooling with latent attention, and training with masked token prediction plus supervised contrastive learning. Positive pairs for contrastive learning are generated by using two general-purpose LLMs to produce clinically equivalent variants of each report: paraphrases, sentence splits, removal of temporal references, anatomical partitioning, and Findings-to-Impression summaries. The authors then couple this text encoder with a vision backbone using low-rank adapters and a projection head to form LLM2CLIP4

Load-bearing premise

The training pipeline assumes the LLM-generated report variants are clinically equivalent to the original reports, so that every positive pair in supervised contrastive learning is factually faithful; unverified hallucinations in those variants would reinforce false clinical content.

Editorial extensions

If this is right

  • Scaling medical vision-language models to large, heterogeneous hospital report corpora becomes viable without performance collapse.
  • Clinically oriented retrieval metrics such as CheXbert F1, RadGraph F1, and GREEN can serve as primary evaluation, capturing semantic equivalence that exact-match recall misses.
  • Abbreviation-heavy and impression-only reports, common in real hospitals, can be used as training data rather than filtered out, provided the text encoder captures style invariance.
  • Section-aware prompting and placeholder tokens help when mixing full findings with impression-only sources, mitigating information-density mismatches.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the variant-generation LLMs occasionally hallucinate a clinical detail, the supervised contrastive objective would treat that false detail as a positive anchor, quietly teaching the encoder to accept the error; a clinical audit of the generated variants would test this risk.
  • The same recipe—domain-adapted LLM encoder plus contrastive alignment—could transfer to other medical imaging modalities, such as MRI or pathology, where reports vary in style and abbreviation conventions.
  • The paper's distinction between text-only gains and multimodal gains suggests the contrastive projection is a bottleneck; better cross-modal alignment, rather than a stronger text model alone, may be where future gains lie.
  • If robust style-invariant text embeddings lower the cost of noisy data, data curation and architecture capacity become substitutes; the paper does not explore this trade-off.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes LLM2VEC4CXR, a bidirectional LLM encoder for chest X-ray reports, trained with masked next-token prediction and supervised contrastive learning on LLM-generated report variants (rephrasings, sentence splits, prior-omitted versions, anatomical recombinations, and findings–impression summaries), plus abbreviation-expanded Indication fields. This encoder is then integrated into a dual-tower CLIP-style framework, LLM2CLIP4CXR, using a frozen LLM backbone with LoRA and a fully trained vision tower. The authors evaluate on text-only tasks and multimodal retrieval on MIMIC-CXR (internal) and Open-I (external), reporting top-k retrieval and clinically oriented metrics (CheXbert F1, RadGraph F1, SembScore, RaTEScore, GREEN), and also present qualitative evaluations by medical students and LLM judges. The central claim is that domain-adapted LLM encoders provide robust clinical text representations that improve image–text alignment and external generalization, and that they resist the degradation seen when noisy abbreviation-rich, impression-only private reports are added to training.

Significance. If the results hold, the paper makes a useful contribution: it demonstrates a concrete recipe for adapting large language models as text towers in medical CLIP, shows gains on an external dataset (Open-I) with clinically oriented metrics, and releases models. The inclusion of retrained baselines under the same data conditions, the use of multiple clinical metrics beyond top-k retrieval, and the qualitative evaluation with both human and LLM raters are strengths. The central idea — that text-encoder robustness, rather than raw scale, is what matters when aggregating heterogeneous radiology reports — is plausible and worth testing. However, the current evidence is weakened by evaluation design issues that need to be addressed before the claims can be accepted.

major comments (4)
  1. [§3 Data generation; Appendix A.2] The text-only evaluation tasks partially reuse the exact pair types used in supervised contrastive training. Task 1 (prior-omitted → original), Task 2 (Findings → Impression), and Task 4 (acronym expansion) are all generated by the same pipeline that defines the training positives. The near-perfect scores for LLM2VEC4CXR on these tasks (e.g., Task 1 @1 = 0.933) therefore reflect memorization of the training task family rather than generalization to new style variations. The genuinely external text-only evidence is Task 3 (error discrimination) and Task 5 (cross-dataset clinical similarity); there the gains over the already-strong general LLM2VEC are modest (Task 5 GREEN 0.676 vs. 0.667; RadGraph 0.402 vs. 0.404). Please re-analyze the results excluding or reinterpreting the in-distribution tasks, and consider adding held-out tasks with variant types not used in training.
  2. [Appendix A.2] The central training pipeline assumes that variants produced by Gemini 2.0-Flash and Deepseek-R1-Distill-Qwen-14B are 'clinically equivalent' to the original reports. These variants define positive pairs in supervised contrastive learning, and the resulting encoder initializes the text tower of LLM2CLIP4CXR. No radiologist audit, no automated consistency check, and no error-rate analysis for the generated variants is reported. If a paraphrase changes a finding, an abbreviation expansion is incorrect, or an anatomical recombination drops a finding, the model is explicitly trained to embed incorrect clinical content close to the correct report. Task 3 shows the model can detect some injected errors, but it does not establish that the training variants themselves are clean. Please add a verification step: a sampled expert audit, an automated clinical-consistency check against CheXpert label
  3. [Table 5] All results are reported as point estimates without error bars, confidence intervals, or significance tests. Several of the key comparisons are small in magnitude — for example, in Table 5, on MIMIC the difference between LLM2CLIP4CXRsection and LLM2CLIP4CXRbase for GREEN is 0.308 vs. 0.299, and on Open-I the corresponding difference is 0.618 vs. 0.600. Without repeated-seed variance or bootstrap confidence intervals, the claims of improvement for the section-aware and private-data variants are not statistically supported. Please provide uncertainty estimates for the three central comparisons: text-encoder adaptation, robustness to added noisy data, and external generalization.
  4. [§4.4] The clinically oriented retrieval metrics for LLM2CLIP4CXR are computed by retrieving from the MIMIC train/validation pool — reports the model has seen during training. This may inflate the absolute clinical-metric values, although comparisons across models trained with the same pool remain fair. The claim of 'strong clinical alignment' would be more convincing if at least one evaluation used a held-out report pool (e.g., MIMIC test reports or a curated external pool). Please clarify the potential impact of this training-pool retrieval on the reported GREEN and related scores.
minor comments (5)
  1. [Table 5] The caption lists 'US-Mix' as a training dataset for MAIRA2 but the term is not defined or used in the text. Please define all dataset abbreviations or remove unused ones.
  2. [Appendix A.3] The study describes human raters as 'medical students' with 3, 8, and 44 months of training, but the evaluation prompt in Appendix A.3 addresses the rater as 'an expert chest X-ray radiologist.' This discrepancy should be acknowledged in the limitations, or the prompt should be adapted for the actual rater population.
  3. [Appendix A.1] Typo: 'Indicationfields' should be 'Indication fields'.
  4. [Table 3] The header abbreviates SembScore as 'Semb' in Task 5 while the text uses 'SembScore'; keep the spelling consistent.
  5. [Appendix B] The text says 'LLM2VEC4CXR is pretrained with MNTP for one epoch' but Appendix B says 'MNTP Epochs: 1'; consistent, but the section also says 'supervised contrastive learning' after MNTP. Consider clarifying the number of contrastive epochs (not stated in the table).

Circularity Check

3 steps flagged · score 6.0 of 10

Text-only robustness Tasks 1, 2, and 4 are the same transformation pair types used as supervised-contrastive training positives, so part of the reported gain is in-distribution; multimodal external results remain independent.

  1. fitted input called prediction [Section 3, Data generation and Supervised contrastive learning; Section 4.4, Task 1; Table 3]
    "Omitting temporal references (ro): Removal of temporal expressions and change descriptors. ... Task 1: Prior-omitted-Original matching. Given a prior-omitted report (ro), the model must retrieve the original report. ... Segments from the same report or describing the same clinical findings are treated as positive pairs."

    The prior-omitted variant ro is explicitly one of the positive-pair types used in supervised contrastive training: the loss directly optimizes the embedding so that ro is close to the original report. Task 1 then evaluates exactly this mapping on held-out reports. The near-perfect Top-k scores therefore measure fit to the training objective, not an independent test of robustness to temporal-reference omission.

  2. fitted input called prediction [Section 3, Data generation and Supervised contrastive learning; Section 4.4, Task 2; Table 3]
    "Summarization pairs: Linking Findings (rf) with Impression (ri) sections. ... Task 2: Report summarization. Given the Findings section, the model retrieves the corresponding Impression."

    The Findings–Impression pairs are created as training positives: 'Summarization pairs: Linking Findings (rf) with Impression (ri) sections.' Task 2 measures Findings→Impression retrieval, which is the same mapping the supervised-contrastive objective trains. The reported improvement over BERT is therefore a direct consequence of the training signal rather than an independent demonstration of clinical summarization ability.

1 more flagged steps
  1. fitted input called prediction [Section 3, Data generation and Supervised contrastive learning; Section 4.4, Task 4; Table 3]
    "To capture clinical shorthand, we additionally include MIMIC's Indication fields, which contain frequent acronyms. ... Task 4: Understanding medical acronyms. We manually curate reports from real hospital data containing acronyms (e.g., 'BLLF', 'PTX') and create expert-refined versions consistent with MIMIC style (e.g., 'bilateral lower lung field'). Performance is measured with Top-k retrieval."

    Training includes abbreviated MIMIC Indication fields paired with expanded variants through the report-variation pipeline. Task 4 evaluates the same acronym-expansion mapping, albeit on manually curated inputs. The model was optimized to embed abbreviated forms close to their expanded forms, making this task in-distribution; the manual curation changes the sample but not the learned mapping.

full rationale

Three of the five text-only evaluation tasks (1, 2, and 4) reuse the exact pair types created in Section 3 and used as supervised-contrastive training positives. Their reported scores therefore reflect the training objective, making the corresponding 'robustness to abbreviations and style variation' claims partially circular. However, the central multimodal claims have independent grounding: MIMIC and Open-I test sets are evaluated with clinically oriented metrics (GREEN, RadGraph F1, CheXbert F1), Task 3 uses synthesized errors not used in training, and Task 5 is cross-dataset (Open-I to MIMIC). The Appendix A.2 self-citation to Ko et al. [39] for splitting/omission is an implementation pointer, not a load-bearing uniqueness premise, so it does not add circularity. The absence of clinician verification of LLM-generated variants is a validity/robustness risk, not a circularity in the derivation. Overall the paper is partially circular in its text-only evaluation but not in its externally validated multimodal results, warranting a score of 6.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

No new physical or ontological entities are introduced; the contributions are new model checkpoints and training recipes. The load-bearing assumptions are about data semantics: paired reports describe the images, synthetic variants preserve clinical truth, automated labelers are trustworthy, and resized frontal images are sufficient. The free parameters are standard CLIP and LoRA hyperparameters plus the specific hand-chosen synthetic-data generation recipe.

free parameters (3)
  • CLIP temperature tau = not specified; follows LLM2CLIP default
    Scales logits in Eq. (1) and controls the sharpness of the contrastive alignment; it is optimized during training but no initial or final value is reported.
  • LoRA rank, alpha, dropout = r=16, alpha=32, dropout=0.1
    Table B.7; these values control how much of the frozen LLM backbone is adapted and follow prior LLM2CLIP defaults rather than being derived from principles.
  • MNTP masking probability and max sequence length = 0.2, 512
    Table B.7; chosen by hand and directly affects the masked token prediction pretraining objective.
assumptions (5)
  • domain assumption Paired CXR images and reports are clinically aligned
    The CLIP objective in Eq. (1) treats each paired image and report as a positive pair; if the pairing is noisy, the alignment target is wrong. Introduced in Section 4.1.
  • domain assumption LLM-generated report variants preserve clinical equivalence
    Supervised contrastive positives are built from Gemini and DeepSeek paraphrases, splits, omissions, and anatomical partitions (Section 3). No clinician verification of these synthetic variants is reported, so hallucinations would inject false positives.
  • domain assumption CheXGPT labels are accurate enough for classification-based positives
    Section 3 uses CheXGPT to label findings and includes classification-based pairs as positives; incorrect labels would reinforce wrong clinical semantics.
  • domain assumption Clinical metrics (CheXbert F1, RadGraph F1, SembScore, RaTEScore, GREEN) measure clinically correct report similarity
    The paper's main evidence for clinical improvement uses these imperfect automated labelers and scoring functions, which may favor certain phrasings or miss others. Used throughout Tables 3, 5, and 6.
  • domain assumption Frontal X-rays resized to 448x448 retain enough diagnostic information for alignment
    Only frontal views are used and all images are resized to 448x448 (Section 4.1); lateral views and technical details are discarded, which could limit clinically relevant alignment.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring the Capabilities of Large Language Model Encoders for Image-Text Retrieval in Chest X-rays." pith.science (2026). https://pith.science/paper/TZ5I2NXV

@misc{pith2026250915234,
  author       = {Pith},
  title        = {Pith review of: Exploring the Capabilities of Large Language Model Encoders for Image-Text Retrieval in Chest X-rays},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TZ5I2NXV}},
  note         = {Machine review of arXiv:2509.15234}
}
read the original abstract

Multimodal learning from paired medical images and clinical text is a central challenge in medical data-driven informatics, where effective cross-modal alignment is critical for scalable analysis and retrieval. In chest radiography, vision-language pretraining is constrained by heterogeneous radiology reports that contain abbreviations, impression-only notes, and institution-specific writing styles. Unlike general-domain settings, naively aggregating large collections of noisy reports can plateau or even degrade multimodal learning when reporting styles differ substantially. We propose a domain-adapted bidirectional large language model text encoder for chest radiograph reports, trained with masked token prediction and supervised contrastive learning on stylistically diverse but clinically equivalent report variants to produce robust, generalizable text embeddings. We then integrate this encoder into a dual-tower contrastive vision-language framework using parameter-efficient adaptation to improve image-text alignment. Across 1.6 million paired studies from public datasets and a de-identified hospital cohort, the proposed models improve bidirectional retrieval accuracy and external generalization, achieving GREEN scores of 0.308 on MIMIC-CXR and 0.618 on Open-I, while reducing the degradation observed when abbreviation-rich, impression-only hospital reports are added to training.

Figures

Figures reproduced from arXiv: 2509.15234 by the authors.

Figure 1
Figure 1. Overview of the proposed framework. (Left) LLM2VEC4CXR adapts a large language model into a clinical embedding model using Masked Next Token Prediction and supervised contrastive learning with diverse report variants. (Right) Compared to BERT, our LLM-based embeddings group clinically similar phrases closer in space, yielding richer and more robust representations for downstream multimodal learning. Positioning of t… view at source ↗
Figure 2
Figure 2. Examples of report variations used for training. Variants include rephrasings ( [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Temporal Inversion for Learning Interval Change in Chest X-Rays

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    TILA uses temporal inversion of image pairs as a supervisory signal to make existing temporal vision-language models more sensitive to directional interval changes in chest X-rays.

Reference graph

Works this paper leans on

47 extracted references · 21 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Mishkin, J

    A.Radford, J.W.Kim, C.Hallacy, A.Ramesh, G.Goh, S.Agarwal, G.Sastry, A.Askell, P. Mishkin, J. Clark, et al., Learning transferable visual models from natural language supervision, in: International conference on machine learning, PMLR, 2021, pp. 8748– 8763

    2021

  2. [2]

    Z. Zhao, Y. Liu, H. Wu, M. Wang, Y. Li, S. Wang, L. Teng, D. Liu, Z. Cui, Q. Wang, et al., Clip in medical imaging: A survey, Medical Image Analysis (2025) 103551

    2025

  3. [3]

    Çallı, E

    E. Çallı, E. Sogancioglu, B. Van Ginneken, K. G. van Leeuwen, K. Murphy, Deep learning for chest x-ray analysis: A survey, Medical image analysis 72 (2021) 102125

    2021

  4. [4]

    X. Liu, H. Liu, G. Yang, Z. Jiang, S. Cui, Z. Zhang, H. Wang, L. Tao, Y. Sun, Z. Song, et al., A generalist medical language model for disease diagnosis assistance, Nature medicine 31 (3) (2025) 932–942

    2025

  5. [5]

    E.Alsentzer, J.R.Murphy, W.Boag, W.-H.Weng, D.Jin, T.Naumann, M.McDermott, Publicly available clinical bert embeddings, arXiv preprint arXiv:1904.03323 (2019)

    arXiv 1904

  6. [6]

    Boecking, N

    B. Boecking, N. Usuyama, S. Bannur, D. C. Castro, A. Schwaighofer, S. Hyland, M. Wetscherek, T. Naumann, A. Nori, J. Alvarez-Valle, et al., Making the most of text semantics to improve biomedical vision–language processing, in: European conference on computer vision, Springer, 2022, pp. 1–21

    2022

  7. [7]

    Bannur, S

    S. Bannur, S. Hyland, Q. Liu, F. Perez-Garcia, M. Ilse, D. C. Castro, B. Boecking, H. Sharma, K. Bouzid, A. Thieme, et al., Learning to exploit temporal structure for biomedical vision-language processing, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 15016–15027

    2023

  8. [8]

    A. E. Johnson, T. J. Pollard, L. Shen, L.-w. H. Lehman, M. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. Anthony Celi, R. G. Mark, Mimic-iii, a freely accessible critical care database, Scientific data 3 (1) (2016) 1–9

    2016

Show all 47 references
  1. [9]

    Johnson, L

    A. Johnson, L. Bulgarelli, T. Pollard, S. Horng, L. A. Celi, R. Mark, Mimic-iv, Phy- sioNet. Available online at: https://physionet. org/content/mimiciv/1.0/(accessed Au- gust 23, 2021) (2020) 49–55

    2021

  2. [10]

    Chambon, J.-B

    P. Chambon, J.-B. Delbrouck, T. Sounack, S.-C. Huang, Z. Chen, M. Varma, S. Q. Truong, C. T. Chuong, C. P. Langlotz, Chexpert plus: Hundreds of thousands of aligned radiology texts, images and patients, arXiv preprint arXiv:2405.19538 (2024)

    2024 arXiv

  3. [11]

    Bustos, A

    A. Bustos, A. Pertusa, J.-M. Salinas, M. De La Iglesia-Vaya, Padchest: A large chest x- ray image dataset with multi-label annotated reports, Medical image analysis 66 (2020) 101797

    2020

  4. [12]

    Demner-Fushman, M

    D. Demner-Fushman, M. D. Kohli, M. B. Rosenman, S. E. Shooshan, L. Rodriguez, S. Antani, G. R. Thoma, C. J. McDonald, Preparing a collection of radiology exam- inations for distribution and retrieval, Journal of the American Medical Informatics Association 23 (2) (2016) 304–310. 16

    2016

  5. [13]

    BehnamGhader, V

    P. BehnamGhader, V. Adlakha, M. Mosbach, D. Bahdanau, N. Chapados, S. Reddy, Llm2vec: Large language models are secretly powerful text encoders, arXiv preprint arXiv:2404.05961 (2024)

    2024 arXiv

  6. [14]

    Huang, A

    W. Huang, A. Wu, Y. Yang, X. Luo, Y. Yang, L. Hu, Q. Dai, X. Dai, D. Chen, C. Luo, et al., Llm2clip: Powerful language model unlock richer visual representation, arXiv preprint arXiv:2411.04997 (2024)

    2024

  7. [15]

    Zhang, H

    Y. Zhang, H. Jiang, Y. Miura, C. D. Manning, C. P. Langlotz, Contrastive learning of medical visual representations from paired images and text, in: Machine Learning for Healthcare Conference, PMLR, 2022, pp. 2–25

    2022

  8. [16]

    E. Tiu, E. Talius, P. Patel, C. P. Langlotz, A. Y. Ng, P. Rajpurkar, Expert-level de- tection of pathologies from unannotated chest x-ray images via self-supervised learning, Nature Biomedical Engineering 6 (12) (2022) 1399–1406

    2022

  9. [17]

    Z. Wang, Z. Wu, D. Agarwal, J. Sun, Medclip: Contrastive learning from unpaired medical images and text, in: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing, Vol. 2022, 2022, p. 3876

    2022

  10. [18]

    Zhang, Y

    S. Zhang, Y. Xu, N. Usuyama, H. Xu, J. Bagga, R. Tinn, S. Preston, R. Rao, M. Wei, N. Valluri, et al., Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs, arXiv preprint arXiv:2303.00915 (2023)

    2023 arXiv

  11. [19]

    K. You, J. Gu, J. Ham, B. Park, J. Kim, E. K. Hong, W. Baek, B. Roh, Cxr-clip: Towardlargescalechestx-raylanguage-imagepre-training, in: InternationalConference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2023, pp. 101–111

    2023

  12. [20]

    Zhang, H.-Y

    X. Zhang, H.-Y. Zhou, X. Yang, O. Banerjee, J. N. Acosta, J. Miller, O. Huang, P. Ra- jpurkar, Rexrank: A public leaderboard for ai-powered radiology report generation, arXiv preprint arXiv:2411.15122 (2024)

    2024 arXiv

  13. [21]

    Bannur, K

    S. Bannur, K. Bouzid, D. C. Castro, A. Schwaighofer, S. Bond-Taylor, M. Ilse, F. Pérez- García, V. Salvatelli, H. Sharma, F. Meissen, et al., Maira-2: Grounded radiology report generation, arXiv preprint arXiv:2406.04449 (2024)

    2024 arXiv

  14. [22]

    Z. Chen, M. Varma, J.-B. Delbrouck, M. Paschali, L. Blankemeier, D. Van Veen, J. M. J. Valanarasu, A. Youssef, J. P. Cohen, E. P. Reis, et al., Chexagent: Towards a foundation model for chest x-ray interpretation, arXiv preprint arXiv:2401.12208 (2024)

    2024 arXiv

  15. [23]

    A. Smit, S. Jain, P. Rajpurkar, A. Pareek, A. Y. Ng, M. P. Lungren, Chexbert: com- bining automatic labelers and expert annotations for accurate radiology report labeling using bert, arXiv preprint arXiv:2004.09167 (2020)

    2004 arXiv

  16. [24]

    S. Jain, A. Agrawal, A. Saporta, S. Q. Truong, D. N. Duong, T. Bui, P. Chambon, Y. Zhang, M. P. Lungren, A. Y. Ng, et al., Radgraph: Extracting clinical entities and relations from radiology reports, arXiv preprint arXiv:2106.14463 (2021). 17

    2021 arXiv

  17. [25]

    Ostmeier, J

    S. Ostmeier, J. Xu, Z. Chen, M. Varma, L. Blankemeier, C. Bluethgen, A. E. Michalson, M. Moseley, C. Langlotz, A. S. Chaudhari, et al., Green: Generative radiology report evaluation and error notation, arXiv preprint arXiv:2405.03595 (2024)

    2024 arXiv

  18. [26]

    C. Lee, R. Roy, M. Xu, J. Raiman, M. Shoeybi, B. Catanzaro, W. Ping, Nv-embed: Improved techniques for training llms as generalist embedding models, arXiv preprint arXiv:2405.17428 (2024)

    2024 arXiv

  19. [27]

    Koloski, A

    B. Koloski, A. Margeloiu, X. Jiang, B. Škrlj, N. Simidjievski, M. Jamnik, Llm embed- dings for deep learning on tabular data, arXiv preprint arXiv:2502.11596 (2025)

    2025 arXiv

  20. [28]

    Zhang, H

    C. Zhang, H. Zhang, S. Wu, D. Wu, T. Xu, X. Zhao, Y. Gao, Y. Hu, E. Chen, Notellm-2: multimodal large representation models for recommendation, arXiv preprint arXiv:2405.16789 (2024)

    2024 arXiv

  21. [29]

    R. Xu, W. Shi, Y. Yu, Y. Zhuang, Y. Zhu, M. D. Wang, J. C. Ho, C. Zhang, C. Yang, Bmretriever: Tuning large language models as better biomedical text retrievers, arXiv preprint arXiv:2404.18443 (2024)

    2024 arXiv

  22. [30]

    G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican, et al., Gemini: a family of highly capable multimodal models, arXiv preprint arXiv:2312.11805 (2023)

    2023 arXiv

  23. [31]

    D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al., Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, arXiv preprint arXiv:2501.12948 (2025)

    2025 arXiv

  24. [32]

    Gu, H.-C

    J. Gu, H.-C. Cho, J. Kim, K. You, E. K. Hong, B. Roh, Chex-gpt: Harnessing large lan- guage models for enhanced chest x-ray report labeling, arXiv preprint arXiv:2401.11505 (2024)

    2024 arXiv

  25. [33]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, K. N. Toutanova, Bert: Pre-training of deep bidirec- tional transformers for language understanding, 2018. URLhttps://arxiv.org/abs/1810.04805

    2018 arXiv

  26. [34]

    Dubey, A

    A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan, et al., The llama 3 herd of models, arXiv preprint arXiv:2407.21783 (2024)

    2024 arXiv

  27. [35]

    V. M. Rao, S. Zhang, J. N. Acosta, S. Adithan, P. Rajpurkar, Rexerr: Synthesizing clinically meaningful errors in diagnostic radiology reports, in: Biocomputing 2025: Proceedings of the Pacific Symposium, World Scientific, 2024, pp. 70–81

    2025

  28. [36]

    W. Zhao, C. Wu, X. Zhang, Y. Zhang, Y. Wang, W. Xie, Ratescore: A metric for radiology report generation, arXiv preprint arXiv:2406.16845 (2024)

    2024 arXiv

  29. [37]

    Z. Wang, X. Luo, X. Jiang, D. Li, L. Qiu, Llm-radjudge: Achieving radiologist-level evaluation for x-ray report generation, arXiv preprint arXiv:2404.00998 (2024). 18

    2024 arXiv

  30. [38]

    S. Lee, J. Youn, H. Kim, M. Kim, S. H. Yoon, Cxr-llava: a multimodal large language model for interpreting chest x-ray images, European Radiology (2025) 1–13

    2025

  31. [39]

    Ko, C.-M

    H. Ko, C.-M. Park, Bringing clip to the clinic: Dynamic soft labels and negation-aware learning for medical analysis, in: Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 25897–25906

    2025

  32. [40]

    Huang, L

    S.-C. Huang, L. Shen, M. P. Lungren, S. Yeung, Gloria: A multimodal global-local representation learning framework for label-efficient medical image recognition, in: Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 3942–3951

    2021

  33. [41]

    year old woman with spontaneous PTX // Assess for PTX or interval change s/p CT placed to WS

    Google Research, Radextract: Radiology text extraction toolkit, available athttps: //huggingface.co/spaces/google/radextract(2024). 19 Appendix A. Datasets We summarize dataset usage and provide additional details of the preprocessing steps. Appendix A.1. Use of Indication fie...

    2024

  34. [42]

    mild”→“moderate

    **Change Severity** Change the severity of a finding (e.g., “mild”→“moderate”). 2. **Change Location** Change the anatomical location of a finding (e.g., “right”→“left”) while keeping it clinically plausible. 3. **False Prediction** Insert a new finding not in the original rep...

  35. [43]

    **Interpret Abbreviations and Shorthand** * Expand abbreviations into full medical terms, except for widely recognized ones such as “COPD.” * Preserve the original sentence’s clinical meaning and structure

  36. [44]

    Reticulonodular pattern

    **Refine Wording** * If no abbreviations are present, still rewrite or refine the sentence using standard radiology reporting style. * Substitute phrases with their common radiological equivalents where appropriate, while ensuring accuracy. Examples include: * “Reticulonodular...

  37. [45]

    left), distribution (upper lobes, lower lobes, basilar, etc.), severity (mild, moderate, severe), and comparisons to prior images

    **Preserve Clinical Details** * Maintain accuracy in laterality (right vs. left), distribution (upper lobes, lower lobes, basilar, etc.), severity (mild, moderate, severe), and comparisons to prior images. * If substitutions do not perfectly reflect the original context, adapt...

  38. [46]

    Reticular opacity in BLLF

    **Output Format** * Provide only the rewritten medical report text. * Do not add extra explanations or commentary. –- **Examples** * Original: “Reticular opacity in BLLF.” Rewritten: “Bilateral lung fields show reticular opacity.” * Original: “Lung nodules in LULF, suspecting ...

  39. [47]

    ‘json "gt

    **False prediction of a finding**: The report states a finding not present in the GT. 2. **Omission of a finding**: The report fails to mention a finding present in the GT. 3. **Incorrect location of a finding**: The report describes the correct finding but in the wrong anatom...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.