Pith. sign in

REVIEW 3 major objections 7 minor 37 references

Medical phrase grounding should return a set of zero, one, or multiple scored image regions per sentence, and the new MedGrounder model shows this is achievable with far fewer human box annotations than end-to-end grounded report generators

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 19:17 UTC pith:2WTW4VSA

load-bearing objection GMPG is a real step forward, but the abstract overstates the non-groundable advantage; MAIRA-2 GR is at parity on N-Acc. the 3 major comments →

arxiv 2512.01085 v3 pith:2WTW4VSA submitted 2025-11-30 cs.CV cs.CL

Generalised Medical Phrase Grounding

Wenjun Zhang , Shekhar S. Chandra , Aaron Nicolson This is my paper
classification cs.CV cs.CL
keywords medical phrase groundinggeneralised referring expression comprehensionchest X-rayradiology report generationDETRweak supervisionzero-shot transfergrounded report generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Real radiology reports routinely violate the one-box-per-phrase assumption that most medical phrase grounding systems inherit from referring-expression comprehension: a sentence can describe findings in several regions, or be a negation or non-diagnostic statement that should ground to nothing. The paper redefines the task as generalised medical phrase grounding (GMPG), in which each sentence maps to a set of zero, one, or more scored bounding boxes, and introduces MedGrounder, a detection-transformer model trained in two stages. It first pretrains on a large-scale, weakly labelled sentence–anatomy alignment dataset (Chest ImaGenome, cleaned with an LLM that keeps only the most specific regions and removes spurious boxes) and then fine-tunes on small expert-annotated sets. The paper's central claim is that this recipe yields strong zero-shot transfer and state-of-the-art results on multi-region and non-groundable phrases, beating REC-style and grounded-report-generation baselines while using roughly one tenth of the human-annotated boxes of the strongest competitor. A sympathetic reader cares because, if correct, GMPG is a more faithful and data-efficient formulation of grounding, and it lets existing report generators be turned into grounded, verifiable systems without retraining them.

Core claim

On PadChest-GR and MS-CXR, the pretrained-only MedGrounder outperforms all baselines in zero-shot transfer, and after fine-tuning it sets the best results on multi-box phrases—for example 28.6% P@F1=1 on PadChest-GR multi-box cases, nearly double the strongest competitor MAIRA-2, while abstaining correctly on non-groundable sentences. When composed with two different report generators (MAIRA-2 in finding-generation mode and CXRMate-RRG24), MedGrounder produces grounded reports whose RadFact spatial F1 is on par with or better than the end-to-end MAIRA-2 GR, despite being trained on one tenth of the human annotations. The authors attribute the transfer to the anatomical inductive bias learned

What carries the argument

The load-bearing object is the GMPG task definition itself: a variable-size set of scored boxes, including the empty set, replaces the fixed single-box regression of REC. MedGrounder realises it with a DETR-style architecture—ResNet-101 and BioClinical ModernBERT encoders, a cross-modal transformer encoder-decoder, bipartite matching between predictions and ground-truth sets, and a confidence head whose threshold permits abstention. The two-stage training regime is the other piece of machinery: an LLM-filtered version of Chest ImaGenome supplies 426,749 weak sentence–box pairs that teach the model which anatomical regions a sentence is about, and a short fine-tuning stage on human boxes refi

Load-bearing premise

The whole pretraining benefit rests on the assumption that the LLM-cleaned Chest ImaGenome annotations are a faithful 'most specific regions' decomposition of each sentence; the paper reports no manual audit or agreement study of this cleaned data, and if the cleanup removes legitimate regions or keeps hallucinated ones, the model's inductive bias—and the zero-shot and fine-tuning gains built on it—are unvalidated.

What would settle it

A human audit of a random sample of GMPG-ImaGenome would settle it: if annotators often find that the retained region is not entailed by the sentence, or that a sensible region was discarded, the pretraining signal is biased. A cheaper quantitative check is an ablation training MedGrounder identically on raw Chest ImaGenome versus the cleaned version; if raw-pretrained performance matches or beats the cleaned version on multi-box and diffuse findings, the LLM filter is not the source of the reported gains.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • GMPG becomes the natural evaluation setting for medical phrase grounding; future datasets and models should report metrics that separate non-groundable, single-box, and multi-box cases, not only mIoU.
  • Weakly supervised pretraining on sentence–anatomy alignments can substitute for a large fraction of human box annotation, lowering the annotation cost of building grounding systems.
  • Grounded report generation becomes modular: any existing report generator can be upgraded to a grounded generator by appending MedGrounder, avoiding expensive image–report–box training triplets.
  • Confidence scores and explicit abstention make AI-generated reports more checkable by radiologists, since every grounded sentence carries spatial evidence.
  • The released GMPG-ImaGenome cleanup provides a reusable weak-supervision resource for pretraining future grounding models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the weak-label assumption holds, the same LLM-filtering recipe could be ported to other weakly aligned medical corpora (CT, MRI, pathology) to create pretraining data for grounding in those modalities.
  • The performance gap between anatomically fixed findings (cardiomegaly) and diffuse or spatially variable ones (lung opacity, rib fractures) suggests an intrinsic ceiling to what fixed anatomical-region priors can teach; further gains will require finer supervision such as segmentation masks or disease-specific box sets.
  • The modular generation-plus-grounding result implies that progress in text-only report generation can be directly converted into progress in verifiable grounded reporting, decoupling the two improvement curves.
  • A concrete test of the cleaning pipeline's value would be to compare MedGrounder pretrained on the raw versus the LLM-cleaned Chest ImaGenome; if raw pretraining matches or beats it on diffuse findings, the 89.6% discard is removing signal, not just noise.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper reformulates medical phrase grounding as generalised medical phrase grounding (GMPG), in which a phrase is mapped to zero, one, or multiple scored bounding boxes. To solve GMPG the authors propose MedGrounder, an MDETR-style detector pretrained on an LLM-cleaned version of Chest ImaGenome and fine-tuned on MS-CXR and PadChest-GR. They report strong zero-shot transfer, gains over REC-style and GRG baselines on multi-box and non-groundable phrases, and a modular pipeline in which MedGrounder grounds the output of existing report generators. Ablations indicate pretraining is essential, WBF post-processing helps, and a clinical language encoder improves results.

Significance. If the claims hold, GMPG is a well-motivated task reformulation and MedGrounder demonstrates a data-efficient route to multi-box grounding and negative suppression. The paper's strengths are its explicit zero/one/multi formulation, the weak-to-expert training recipe, the inclusion of non-grounding examples, and the planned release of code and data. The ablations are informative: from-scratch training collapses, and WBF and BioClinical ModernBERT give consistent gains. However, the headline non-groundable claim is not supported by the fine-tuned comparison in Table III, and the pretraining signal from the 89.6% filtered weak labels lacks validation. These issues are fixable but require additional experiments or a softened claim.

major comments (3)
  1. [Abstract; Table III] The abstract states that MedGrounder 'outperforms ... non-groundable phrases', but in the fine-tuned comparison on PadChest-GR, MAIRA-2 GR has N-Acc 91.0 while MedGrounder (M,P) has 90.1. Since N-Acc is the only non-groundable metric, the claimed advantage over GRG baselines on this requirement is unsupported. The 0.9-point gap is reported without confidence intervals or significance tests. N-Acc is also gameable: Table VII shows from-scratch models reach 99.8-99.9 N-Acc with P@F1 near zero. Please add interval estimates or significance tests, report N-Acc together with P@F1, and adjust the abstract to 'comparable' unless superiority is statistically supported.
  2. [§III-D.1; Fig. 3] The central pretraining step is built on LLaMA-3-70B filtering that discards 89.6% of Chest ImaGenome sentence-box pairs, yet there is no manual audit, agreement study, or comparison against pretraining on the unfiltered dataset. Since Table VII establishes that pretraining is essential, a biased filter could be the source of the zero-shot and fine-tuning gains. I request a sample audit and a raw-vs-cleaned pretraining ablation, or at minimum a per-region error analysis showing the retained labels are not systematically wrong.
  3. [Table VI; §V-B] The modular-GRG conclusion rests heavily on a 38-case non-overlapping subset. The reported differences (e.g., Spatial F1 53.52 vs 52.03) are within plausible noise for n=38. Please provide confidence intervals or a larger non-overlapping evaluation set before claiming the pipeline matches or exceeds MAIRA-2 GR.
minor comments (7)
  1. [Fig. 2] Caption typo: 'MobernBERT' should be 'ModernBERT'.
  2. [Eq. (3)] The notation 'αtj =0 = 0.1' is confusing; use α_{t_j=0}.
  3. [Table VII] The PadChest-GR from-scratch row shows P@F1=41.3 with CH-F1=1.5; please verify the column alignment.
  4. [Tables III/IV] Several numbers are merged in the text (e.g., '60.775.1', '086.5'); fix the spacing.
  5. [§III-E] CH-F1 and Mask IoU accuracy are described only informally; provide formal definitions or equations.
  6. [Fig. 7] Figure contains typos: 'sihouette', 'Enlongated', 'efusion'.
  7. [§V-B] The 38-case subset is not described; report how it was selected and its disease distribution. Also 'subest' is a typo.

Circularity Check

0 steps flagged

No significant circularity: MedGrounder's results are empirical comparisons on independent human-annotated benchmarks; no reported metric reduces to a fitted parameter, a self-citation chain, or a by-construction definition.

full rationale

MedGrounder's derivation chain is empirical and externally benchmarked. The GMPG objective (Eqs. 2–4) is the standard MDETR set-prediction loss; no reported number (P@F1=1, CH-F1, N-Acc, Mask IoU, RadFact) is defined in terms of a fitted parameter of the same experiment. Pretraining on LLM-cleaned Chest ImaGenome is a label-generation step: LLaMA-3-70B filters candidate regions and negative sentences before training, and the fine-tuning/evaluation sets (MS-CXR, PadChest-GR) are independent expert annotations. Thus the model's ability to suppress non-groundable phrases or emit multiple boxes is learned from weak labels but tested against human labels; it is not forced by construction. Self-citations [1], [24], [25], [29], [37] are either baselines (AGPT) or unrelated report-generation references; none supplies an unverified premise required for the central claim. The closest issue is not circularity: the abstract says MedGrounder 'outperforms ... baselines on ... non-groundable phrases', while Table III shows MAIRA-2 GR N-Acc 91.0 vs MedGrounder (M,P) 90.1 on PadChest-GR. That is a factual/statistical overclaim (no error bars), not an equation reducing the prediction to its input. The LLM cleanup's 89.6% reduction without a manual audit is a data-quality limitation, not a circular step.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The paper introduces a task formulation and dataset, not physical entities. The central claim rests on a small number of hand-set hyperparameters and, more importantly, on the unvalidated LLM-cleaned weak supervision and the trustworthiness of the benchmark annotations.

free parameters (3)
  • confidence_threshold = 0.8
    Selected by maximizing center-hit precision–recall F1 on the validation sets of MS-CXR and PadChest-GR (§V-C.2); applied to all reported inference results.
  • wbf_iou_threshold = 0.1
    Weighted box fusion merge threshold used at inference (§V-C.2); ablation in Table VIII shows it alters reported P@F1.
  • loss_weights = α_tj=0=0.1, λ_L1=5, λ_giou=2
    Hand-set loss weights following DETR/MDETR conventions (§III-B); not fitted in this paper but chosen before experiments.
axioms (4)
  • domain assumption Chest ImaGenome anatomical boxes, after LLM filtering, are valid spatial supervision for radiology sentences
    Section III-C assumes the model-generated boxes are meaningful targets; no manual validation of the 426,749 cleaned pairs is provided.
  • domain assumption LLaMA-3-70B region filtering is accurate and unbiased
    Section III-D.1 relies entirely on LLM judgments to remove redundant regions and negative-sentence boxes; the 89.6% reduction from >4M to 426,749 pairs is drastic and unverified.
  • domain assumption MS-CXR and PadChest-GR human boxes are trustworthy ground truth
    Used for fine-tuning and evaluation (§III-C); MS-CXR was originally an evaluation benchmark, not a training set, and annotation noise is not quantified.
  • domain assumption RadFact LLM evaluation measures clinical and spatial entailment faithfully
    Section III-F adopts RadFact without showing its agreement with radiologist judgments in this setting; the abstract claim of GRG quality depends on this metric.

pith-pipeline@v1.3.0-alltime-deepseek · 15354 in / 19445 out tokens · 171311 ms · 2026-08-03T19:17:44.263106+00:00 · methodology

0 comments
read the original abstract

Medical phrase grounding (MPG) maps textual descriptions of radiological findings to corresponding image regions. These grounded reports are easier to interpret, especially for non-experts. Existing MPG systems mostly follow the referring expression comprehension (REC) paradigm and return exactly one bounding box per phrase. Real reports often violate this assumption. They contain multi-region findings, non-diagnostic text, and non-groundable phrases, such as negations or descriptions of normal anatomy. Motivated by this, we reformulate the task as generalised medical phrase grounding (GMPG), where each sentence is mapped to zero, one, or multiple scored regions. To realise this formulation, we introduce the first GMPG model: MedGrounder. We adopted a two-stage training regime: pre-training on report sentence--anatomy box alignment datasets and fine-tuning on report sentence--human annotated box datasets. Experiments on PadChest-GR and MS-CXR show that MedGrounder achieves strong zero-shot transfer and outperforms REC-style and grounded report generation baselines on multi-region and non-groundable phrases, while using far fewer human box annotations. Finally, we show that MedGrounder can be composed with existing report generators to produce grounded reports without retraining the generator.

Figures

Figures reproduced from arXiv: 2512.01085 by Aaron Nicolson, Shekhar S. Chandra, Wenjun Zhang.

Figure 1
Figure 1. Figure 1: Top: MPG predicts a single bounding box per sentence, while GMPG additionally supports (i) multiple boxes per sentence, (ii) sup￾pression of boxes for non-groundable phrases (e.g., negations), and (iii) confidence scoring for all predictions. Bottom: Applications of GMPG: (A) grounding a radiologist-written report for patient comprehension, and (B) grounding an AI-generated report for radiologist verificat… view at source ↗
Figure 2
Figure 2. Figure 2: MedGrounder architecture overview. ResNet-101 and BioClinical MobernBERT encode image and phrase features, respectively, which are concatenated and fed into a cross-encoder. From the features of the cross-encoder, a Transformer decoder then predicts a set of scored bounding boxes corresponding to the phrase, with an example shown on the right. TABLE II STATISTICS OF DATASETS FOR PRETRAINING AND FINE-TUNING… view at source ↗
Figure 3
Figure 3. Figure 3: LLM prompt for filtering redundant anatomical regions. The model selects the most specific regions from candidates, discarding broader parent regions. and discard redundant parent regions. The prompt used for this filtering step is shown in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison of grounding results from TransVG, MedRPG, MAIRA-2 GR, and the proposed MedGrounder on three PadChest-GR examples. Ground-truth (GT) is shown in the first column, model predictions in the remaining columns. Numbers in brackets after each sentence indicate the number of bounding boxes in the GT. TABLE III PHRASE GROUNDING PERFORMANCE ON THE TEST SET OF PADCHEST-GR. MODELS ARE GROUPED … view at source ↗
Figure 5
Figure 5. Figure 5: Correlation heatmap on MS-CXR. Performance vs. data properties; cell text shows Pearson r and p-values. Asterisks indicate statistical significance levels, ** p < 0.01, and * p < 0.05. 3) Disease specific analysis on MS-CXR: Table V shows that MedGrounder’s performance depends strongly on disease characteristics. Anatomically constrained findings such as car￾diomegaly are localised with near-perfect accura… view at source ↗
Figure 6
Figure 6. Figure 6: Visualisation result of our model on MS-CXR across eight findings. Ground truth boxes are green, model predictions are red (dashed) with IoU shown. Top: correct cases. Bottom: error cases for the same disease; no cardiomegaly error example appears. C. Ablation Studies We conducted a series of ablation studies to validate the im￾pact of our key design choices: the two-stage training strategy, post-processin… view at source ↗
Figure 7
Figure 7. Figure 7: Visualisation result of our model on PadChest-GR across sixteen findings. The top two rows show eight disease categories with strong phrase grounding results. The bottom two rows show eight categories with weaker performance. Ground-truth regions are green; predicted boxes are red (dashed) with IoU indicated. Fusion (WBF) with a 0.1 IoU threshold to merge redundant detections further improves performance, … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 4 linked inside Pith

  1. [1]

    Anatomical grounding pre- training for medical phrase grounding,

    W. Zhang, S. S. Chandra, and A. Nicolson, “Anatomical grounding pre- training for medical phrase grounding,” inISBI, 2025, pp. 1–5

    2025

  2. [2]

    Medical phrase grounding with region-phrase context contrastive alignment,

    Z. Chen, Y . Zhou, A. Tran, J. Zhao, L. Wan, G. S. K. Ooi, L. T.-E. Cheng, C. H. Thng, X. Xu, Y . Liu, and H. Fu, “Medical phrase grounding with region-phrase context contrastive alignment,” inMICCAI, 2023, pp. 371–381

    2023

  3. [3]

    Interactive and explainable region-guided radiology report generation,

    T. Tanida, P. M ¨uller, G. Kaissis, and D. Rueckert, “Interactive and explainable region-guided radiology report generation,” inCVPR, 2023, pp. 7433–7442

    2023

  4. [4]

    GREC: Generalized referring expression comprehension,

    S. He, H. Ding, C. Liu, and X. Jiang, “GREC: Generalized referring expression comprehension,”arXiv:2308.16182 [cs.LG], 2023

    Pith/arXiv arXiv 2023

  5. [5]

    Towards visual grounding: A survey,

    L. Xiao, X. Yang, X. Lan, Y . Wang, and C. Xu, “Towards visual grounding: A survey,”IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–20, 2025

    2025

  6. [6]

    MDETR – modulated detection for end-to-end multi-modal understand- ing,

    A. Kamath, M. Singh, Y . LeCun, G. Synnaeve, I. Misra, and N. Carion, “MDETR – modulated detection for end-to-end multi-modal understand- ing,” inICCV, 2021, pp. 1760–1770

    2021

  7. [7]

    Chest ImaGenome dataset for clinical reasoning,

    J. T. Wu, N. N. Agu, I. Lourentzou, A. Sharma, J. A. Paguio, J. S. Yao, E. C. Dee, W. Mitchell, S. Kashyap, A. Giovannini, L. A. Celi, and M. Moradi, “Chest ImaGenome dataset for clinical reasoning,” in NeurIPS, 2021, pp. 1–14

    2021

  8. [8]

    Making the most of text semantics to improve biomedical vision–language processing,

    B. Boecking, N. Usuyama, S. Bannur, D. C. Castro, A. Schwaighofer, S. Hyland, M. Wetscherek, T. Naumann, A. Nori, J. Alvarez-Valle, H. Poon, and O. Oktay, “Making the most of text semantics to improve biomedical vision–language processing,” inECCV, 2022, pp. 1–21

    2022

  9. [9]

    PadChest-GR: A bilingual chest X-ray dataset for grounded radiology report generation,

    D. C. de Castro, A. Bustos, S. Bannur, S. L. Hyland, K. Bouzid, M. T. Wetscherek, M. D. S ´anchez-Valverde, L. Jaques-P´erez, L. P ´erez- Rodr´ıguez, K. Takeda, J. M. Salinas-Serrano, J. Alvarez-Valle, J. Galant- Herrero, and A. Pertusa, “PadChest-GR: A bilingual chest X-ray dataset for grounded radiology report generation,”NEJM AI, vol. 2, no. 7, 6 2025

    2025

  10. [10]

    Multimodal health- care AI: Identifying and designing clinically relevant vision-language applications for radiology,

    N. Yildirim, H. Richardson, and e. a. Wetscherek, “Multimodal health- care AI: Identifying and designing clinically relevant vision-language applications for radiology,”Conference on Human Factors in Computing Systems - Proceedings, vol. 22, 2 2024

    2024

  11. [11]

    MAIRA- 2: Grounded radiology report generation,

    S. Bannur, K. Bouzid, D. Coelho de Castro, A. Schwaighofer, S. Bond- Taylor, M. Ilse, F. P ´erez-Garc´ıa, V . Salvatelli, H. Sharma, F. Meissen, M. Ranjit, S. Srivastav, J. Gong, F. Falck, O. Oktay, A. Thieme, M. P. Lungren, M. T. Wetscherek, J. Alvarez-Valle, and S. Hyland, “MAIRA- 2: Grounded radiology report generation,” Microsoft, Tech. Rep., 2024

    2024

  12. [12]

    GLoRIA: A multimodal global-local representation learning framework for label- efficient medical image recognition,

    S.-C. Huang, L. Shen, M. P. Lungren, and S. Yeung, “GLoRIA: A multimodal global-local representation learning framework for label- efficient medical image recognition,” inICCV, 2021, pp. 3942–3951

    2021

  13. [13]

    Learning to exploit temporal structure for biomedical vision-language processing,

    S. Bannur, S. Hyland, Q. Liu, F. Perez-Garcia, M. Ilse, D. C. Castro, B. Boecking, H. Sharma, K. Bouzid, A. Thieme, A. Schwaighofer, M. Wetscherek, M. P. Lungren, A. Nori, J. Alvarez-Valle, and O. Oktay, “Learning to exploit temporal structure for biomedical vision-language processing,” inCVPR, 2023, pp. 15 016–15 027

    2023

  14. [14]

    Zero-shot medical phrase grounding with off-the-shelf diffusion models,

    K. Vilouras, P. Sanchez, A. Q. O’neil, and S. A. Tsaftaris, “Zero-shot medical phrase grounding with off-the-shelf diffusion models,”IEEE Journal of Biomedical and Health Informatics, 2024

    2024

  15. [15]

    Generate to ground: Mul- timodal text conditioning boosts phrase grounding in medical vision- language models,

    F. N ¨utzel, M. Dombrowski, and B. Kainz, “Generate to ground: Mul- timodal text conditioning boosts phrase grounding in medical vision- language models,” inMedical Imaging with Deep Learning, 2025

    2025

  16. [16]

    MedRG: Medical report grounding with multi- modal large language model,

    K. Zou, Y . Bai, Z. Chen, Y . Zhou, Y . Chen, K. Ren, M. Wang, X. Yuan, X. Shen, and H. Fu, “MedRG: Medical report grounding with multi- modal large language model,”arXiv:2404.06798 [cs.CV], 2024

    Pith/arXiv arXiv 2024

  17. [17]

    ChEX: Interactive localization and region description in chest X-rays,

    P. M ¨uller, G. Kaissis, and D. Rueckert, “ChEX: Interactive localization and region description in chest X-rays,” inECCV, 2024, pp. 92–111

    2024

  18. [18]

    End-to-end object detection with transformers,

    N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in ECCV, 2020, pp. 213–229

    2020

  19. [19]

    Visual grounding of whole radiology reports for 3D CT images,

    A. Ichinose, T. Hatsutani, K. Nakamura, Y . Kitamura, S. Iizuka, E. Simo- Serra, S. Kido, and N. Tomiyama, “Visual grounding of whole radiology reports for 3D CT images,” inMICCAI, 2023, pp. 611–621

    2023

  20. [20]

    ReferItGame: Referring to objects in photographs of natural scenes,

    S. Kazemzadeh, V . Ordonez, M. Matten, and T. Berg, “ReferItGame: Referring to objects in photographs of natural scenes,” inEMNLP, 2014, pp. 787–798

    2014

  21. [21]

    Visual Genome: Connecting language and vision using crowdsourced dense image annotations,

    R. Krishna, Y . Zhu, O. Groth, J. Johnson, K. Hata, J. Kravitz, S. Chen, Y . Kalantidis, L.-J. Li, D. A. Shamma, M. S. Bernstein, and L. Fei-Fei, “Visual Genome: Connecting language and vision using crowdsourced dense image annotations,”International Journal of Computer Vision, vol. 123, no. 1, pp. 32–73, 2017

    2017

  22. [22]

    Modeling context in referring expressions,

    L. Yu, P. Poirson, S. Yang, A. C. Berg, and T. L. Berg, “Modeling context in referring expressions,” inECCV, 2016, pp. 69–85

    2016

  23. [23]

    RECANTFormer: Referring expression comprehension with varying numbers of targets,

    B. Hemanthage, H. Bilen, P. Bartie, C. Dondrup, and O. Lemon, “RECANTFormer: Referring expression comprehension with varying numbers of targets,” inEMNLP, 2024, pp. 21 784–21 798

    2024

  24. [24]

    Improving chest X-ray report generation by leveraging warm starting,

    A. Nicolson, J. Dowling, and B. Koopman, “Improving chest X-ray report generation by leveraging warm starting,”Artificial Intelligence in Medicine, vol. 144, p. 102633, 2023

    2023

  25. [25]

    The impact of auxiliary patient data on automated chest X-ray report generation and how to incorporate it,

    A. Nicolson, S. Zhuang, J. Dowling, and B. Koopman, “The impact of auxiliary patient data on automated chest X-ray report generation and how to incorporate it,” inACL, 2025, pp. 177–203

    2025

  26. [26]

    Exploring and distilling posterior and prior knowledge for radiology report generation,

    F. Liu, X. Wu, S. Ge, W. Fan, and Y . Zou, “Exploring and distilling posterior and prior knowledge for radiology report generation,” inCVPR, 2021, pp. 13 753–13 762

    2021

  27. [27]

    Towards gener- alist foundation model for radiology by leveraging web-scale 2D&3D medical data,

    C. Wu, X. Zhang, Y . Zhang, W. Xie, and Y . Wang, “Towards gener- alist foundation model for radiology by leveraging web-scale 2D&3D medical data,”Nature Communications, vol. 16, no. 1, p. 7866, 2025

    2025

  28. [28]

    MedVersa: A generalist foundation model for medical image interpretation,

    H.-Y . Zhou, J. N. Acosta, S. Adithan, S. Datta, E. J. Topol, and P. Rajpurkar, “MedVersa: A generalist foundation model for medical image interpretation,”arXiv:2405.07988 [cs.CV], 2024

    Pith/arXiv arXiv 2024

  29. [29]

    Longitudinal data and a semantic similarity reward for chest X-ray report generation,

    A. Nicolson, J. Dowling, D. Anderson, and B. Koopman, “Longitudinal data and a semantic similarity reward for chest X-ray report generation,” Informatics in Medicine Unlocked, vol. 50, p. 101585, 2024

    2024

  30. [30]

    Development of a large-scale grounded vision language dataset for chest CT analysis,

    X. Zhang, C. Wu, Z. Zhao, J. Lei, W. Tian, Y . Zhang, W. Xie, and Y . Wang, “Development of a large-scale grounded vision language dataset for chest CT analysis,”Scientific Data, vol. 12, no. 1, p. 1636, 2025

    2025

  31. [31]

    VividMed: Vision lan- guage model with versatile visual grounding for medicine,

    L. Luo, B. Tang, X. Chen, R. Han, and T. Chen, “VividMed: Vision lan- guage model with versatile visual grounding for medicine,” inNAACL, 2025, pp. 1800–1821

    2025

  32. [32]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inCVPR, 2016, pp. 770–778

    2016

  33. [33]

    BioClinical ModernBERT: A state-of-the-art long-context encoder for biomedical and clinical NLP,

    T. Sounack, J. Davis, B. Durieux, A. Chaffin, T. J. Pollard, E. Lehman, A. E. W. Johnson, M. McDermott, T. Naumann, and C. Lindvall, “BioClinical ModernBERT: A state-of-the-art long-context encoder for biomedical and clinical NLP,”arXiv:2506.10896 [cs.CL], 2025

    Pith/arXiv arXiv 2025

  34. [34]

    MIMIC-CXR, a de- identified publicly available database of chest radiographs with free-text reports,

    A. E. Johnson, T. J. Pollard, S. J. Berkowitz, N. R. Greenbaum, M. P. Lungren, C.-y. Deng, R. G. Mark, and S. Horng, “MIMIC-CXR, a de- identified publicly available database of chest radiographs with free-text reports,”Scientific Data, vol. 6, no. 1, p. 317, 2019

    2019

  35. [35]

    Introducing Meta Llama 3,

    Meta AI, “Introducing Meta Llama 3,” https://ai.meta.com/blog/ meta-llama-3/, 2024

    2024

  36. [36]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” inICLR, 2017

    2017

  37. [37]

    e-Health CSIRO at RRG24: Entropy-augmented self-critical sequence training for radiology report generation,

    A. Nicolson, J. Liu, J. Dowling, A. Nguyen, and B. Koopman, “e-Health CSIRO at RRG24: Entropy-augmented self-critical sequence training for radiology report generation,” inProceedings of the 23rd Workshop on Biomedical Natural Language Processing, 2024, pp. 99–104

    2024