Pith. sign in

REVIEW 4 major objections 5 minor 32 references

Reinforcement learning that rewards clinical accuracy, not just word overlap, produces chest X-ray reports that generalize across institutions and outperform prior state of the art.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 08:24 UTC pith:F4Y3C32C

load-bearing objection Solid RL-for-RRG benchmark result with released weights, but the clinical-error headline rests on a self-referential metric and needs human validation. the 4 major comments →

arxiv 2601.17151 v2 pith:F4Y3C32C submitted 2026-01-23 cs.CV cs.CL

Scaling medical imaging report generation with multimodal reinforcement learning

Qianchu Liu , Sheng Zhang , Guanghui Qin , Yu Gu , Ying Jin , Sam Preston , Yanbo Xu , Sid Kiblawi
show 5 more authors
Wen-wai Yim Timothy Ossowski Tristan Naumann Mu Wei Hoifung Poon
This is my paper
classification cs.CV cs.CL
keywords radiology report generationreinforcement learningGRPOchest X-rayclinical error metricmultimodal healthcare AIzero-shot generalizationlongitudinal report generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that supervised fine-tuning alone makes report generators overfit to dataset-specific boilerplate, and that reinforcement learning over a composite of lexical, semantic, and clinical-error rewards fixes this. It introduces UniRG-CXR, an 8B vision-language model trained with GRPO to directly optimize BLEU, BERTScore, SembScore, RadGraph-F1, and an LLM-judged clinical error count. On the ReXrank benchmark the model reports the best 1/RadCliQ-v1 score on all four datasets in both findings-only and findings+impression settings, and best zero-shot performance on a held-out public dataset and a proprietary one. The authors' central claim is that optimizing the full evaluation stack, including error awareness, yields durable, universal generalization rather than metric overfitting.

Core claim

UniRG-CXR is a single Qwen3-VL-8B model fine-tuned first by SFT on 560k+ studies from MIMIC-CXR, CheXpert Plus, ReXGradient, and IU-Xray, then by GRPO reinforcement learning in two steps: first optimizing a RadCliQ-weighted composite of BLEU, BERTScore, SembScore, and RadGraph-F1; then adding 1/(#CheXprompt errors + 1) as an error-reduction reward with weight 0.5. The paper reports SOTA 1/RadCliQ-v1 on ReXrank across ReXGradient, MIMIC-CXR, IU-Xray, and CheXpert Plus in both settings, with gains exceeding 50% over prior best on ReXGradient and IU. It also reports 21.3% of reports with at most one clinical error vs 16.1% for MedVersa and 3.1% for MedGemma, robust condition-level F1, better lo

What carries the argument

The load-bearing mechanism is the two-stage, multi-metric reward design under GRPO. Step 1 applies RadCliQ coefficients (0 for BLEU-2, 0.370 BERTScore, 0.253 SembScore, 0.377 RadGraph-F1) to align with the benchmark aggregate; Step 2 adds the CheXprompt error-count reward, 1/(errors+1), at weight 0.5 with KL regularization 0.03, to drive factual correctness. The longitudinal setup additionally feeds the most recent prior image and report so the model can reason over temporal change instead of copying prior text.

Load-bearing premise

The central claim that UniRG-CXR produces more clinically faithful reports rests on CheXprompt, a GPT-4-based error counter used both as the Step-2 training reward and as the evaluation metric, with no radiologist validation that its judgments are unbiased and non-gameable for RL-optimized output.

What would settle it

A blinded reading by board-certified radiologists comparing UniRG-CXR, MedVersa, and MedGemma reports on a random sample of MIMIC-CXR and proprietary studies, scoring clinically significant errors; if the radiologist error ranking does not match CheXprompt's ranking, or if CheXprompt errors shift under trivial stylistic rewrites of the same clinical content, the clinical-fidelity claim collapses.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the claims hold, the same SFT+RL recipe should transfer to other imaging domains (e.g., CT, MRI, ultrasound) where reference reports exist.
  • Error-aware RL could push clinical-error rates far below current SOTA, making automated draft reports more plausible for radiologist review.
  • The zero-shot results suggest models can be trained on large public corpora and deployed at new institutions without site-specific fine-tuning.
  • Longitudinal conditioning with RL may become the standard for tracking disease progression and regression, replacing copy-prior shortcuts.
  • Because the model improves metrics it was not explicitly trained on (e.g., RaTEScore), combined-reward RL appears to produce genuinely more clinically informative output.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • CheXprompt is simultaneously the training reward and the headline evaluation metric, and its authors overlap with this paper's authors; if GPT-4's error judgments are sensitive to output style or RL-induced phrasing, the 'clinical fidelity' improvement could be partially metric-specific, and a radiologist-grounded reading study would settle this.
  • The 'universal across institutions' claim rests on benchmark test sets and a proprietary dataset; a prospective deployment study in real clinical workflow, including ambiguous images and varied hardware, would be needed to confirm generalization in practice.
  • The model uses only frontal 512x512 images; higher-resolution or lateral views might change the observed error profile, so the reported performance ceiling is not necessarily the ceiling of the approach.
  • The paper's 'durable generalization' thesis implies SFT is the bottleneck in prior models; a direct test would be applying the same two-step RL to a much weaker base model and seeing whether the gap persists or narrows.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces UniRG, a reinforcement-learning framework for medical imaging report generation. UniRG-CXR is built by supervised fine-tuning and GRPO-based RL on Qwen3-VL-8B-Instruct, optimizing a composite reward of BLEU, BERTScore, SembScore, RadGraph-F1, and CheXprompt (an LLM-based clinical-error counter). The model is trained on MIMIC-CXR, CheXpert Plus, ReXGradient, and IU-Xray (with a zero-shot variant excluding IU), and evaluated on the ReXrank benchmark across four datasets in two settings, plus longitudinal, demographic-subgroup, and condition-level analyses. The paper claims new overall SOTA on ReXrank, substantially improved clinical-error counts, and strong generalization to unseen data. The central technical claims are that combined-reward RL outperforms single-metric RL and that adding CheXprompt to the reward reduces clinical errors.

Significance. If the results hold, the paper provides a scalable recipe for optimizing radiology report generation and releases a competitive open-weight model. The evaluation is unusually broad: ReXrank leaderboard results, ablations, zero-shot generalization, longitudinal reasoning, demographic stratification, and disease-level classification. The ablations in Figure 2(b)-(d) are informative and support the value of combining multiple reward signals. The release of UniRG-CXR on HuggingFace is a contribution. However, the most consequential non-benchmark claim—that RL improves clinical fidelity—is measured exclusively with CheXprompt, which is simultaneously the Step-2 training reward and whose authors overlap with this paper's authors. The manuscript also lacks confidence intervals for all headline comparisons. These concerns are substantial but addressable with additional analysis and independent evaluation.

major comments (4)
  1. [Methods: Optimization (Step 2) and Results (Figure 2f)] The claim that UniRG-CXR is more 'clinically faithful' (Abstract, Discussion) rests entirely on CheXprompt, an LLM-based error count. CheXprompt is not an independent evaluation: Step 2 of the RL training uses 1/(#CheXprompt errors + 1) as a reward with weight 0.5, and Figure 2(f) reports improvements on the same metric. Optimizing a metric on the training set will typically improve it on the test set even if the underlying clinical construct is unchanged. The paper provides no radiologist or expert-reader validation, no measurement of GPT-4's agreement with human judgment on these RL-optimized outputs, and no calibration of CheXprompt on the model's report style. This is load-bearing because the paper's central clinical-fidelity claim depends on it. Please add an independent expert evaluation (e.g., radiologist error ratings on a sample) or explicitly re-frame the claim as 'improvement
  2. [Methods: Optimization (Step 1) and Evaluation Metrics] The Step-1 RL reward is the RadCliQ-v1 composite itself (BLEU weight 0, BERTScore 0.370, SembScore 0.253, RadGraph 0.377), and the headline ReXrank metric is 1/RadCliQ-v1. Thus the main SOTA result is the outcome of direct optimization of the evaluation metric. This is not data leakage, but it means the ReXrank gains cannot, by themselves, support the 'durable generalization' and 'universal improvement' claims. The paper should acknowledge this explicitly and place more weight on metrics not in the reward (RaTEScore, disease-level F1, zero-shot datasets), which are the only evidence that the gains are not metric overfitting. A control that optimizes RadCliQ with a different or weaker optimizer would also help substantiate the framework's value.
  3. [Results (Figures 2-4)] All headline numbers are point estimates. No confidence intervals, bootstrap, or significance tests are reported. On CheXpert Plus the test set has only 200 studies, and several subgroup and longitudinal categories have n < 300 (e.g., n=81 for pneumothorax, n=221 for the 4th encounter). The claimed 'wide margin' (e.g., IU 4.80 vs 1.92, a 150% difference) needs to be accompanied by uncertainty quantification to establish that the margins are not due to small-sample variability.
  4. [Methods: Optimization] The description of the RL hyperparameters is internally inconsistent. The main text says 'we incorporate two key improvements: (1) a higher clipping threshold ... (2) remove KL penalty,' but Step 2 states 'We also apply a KL regularization term with coefficient 0.03.' Please clarify whether a KL penalty is used in Step 1, Step 2, or both, and specify the clipping threshold. This is necessary for reproducibility.
minor comments (5)
  1. [References] References [26] and [27] are identical; renumber to avoid duplicate citation.
  2. [Methods: Dataset Details] In the SFT hyperparameter grid, 'batch sizes [128, 256, 521]' — the last value should likely be 512.
  3. [Figure 1 caption] 'every steup' should be 'every setup.'
  4. [Figure 4 caption] 'PD' is used in the figure but defined only in the text; add a definition in the caption.
  5. [Longitudinal evaluation] The temporal-change categories are labeled by GPT-5. Please provide a validation of this labeler or report agreement with human annotations, since the categories are used to draw conclusions about longitudinal reasoning.

Circularity Check

3 steps flagged

Clinical-fidelity and diagnostic-accuracy claims are measured with the same instruments used as RL rewards (CheXprompt and CheXbert/SembScore); the ReXrank SOTA itself is a transparent held-out optimization result, so circularity is partial.

specific steps
  1. fitted input called prediction [Methods – Optimization, Step 2; Main – Figure 2(f) and surrounding text]
    "Step 2: Error-reduction optimization. Starting from the best checkpoint from Step 1’s RadCliQ optimization, we perform an additional epoch in which we incorporate the CheXprompt error metric into the reward. Specifically, we use 1 /(# CheXprompt errors + 1) as the CheXprompt reward to incentivize reducing factual reporting errors. ... UniRG-CXR produces substantially more error-free or low-error reports (21.3% ≤ 1 error) compared with prior state-of-the-art systems (Medversa 16.1% and MedGemma 3.1%) ... These results indicate that UniRG-CXR achieves more clinically faithful and accurate report"

    The clinical-error evidence for 'clinically faithful' reporting is the identical CheXprompt metric that is directly optimized as the Step-2 training reward (up to the monotone 1/(#errors+1) transform). The reported 21.3% ≤1-error rate is therefore the outcome of optimizing that exact objective, not an independent measure of clinical correctness. The inference that this means 'more clinically accurate' assumes CheXprompt is a valid and non-gameable clinical error counter; the paper provides no radiologist validation or calibration of GPT-4 on RL-optimized report styles. Thus the central clinical-fidelity claim reduces to 'the optimized reward decreased' unless CheXprompt is independently trusted.

  2. fitted input called prediction [Methods – Inference and Evaluation Metrics (SembScore); Results – Generalization and Robustness / Figure 4(b,c)]
    "SembScore [20] is a radiology-specific metric that computes cosine similarity between 14-pathology indicator vectors produced by the CheXbert labeler for generated and groundtruth reports. ... We evaluate the condition diagnosis capabilities of the output from UniRG-CXR by applying the CheXbert model [20] to detect diseases from its generated reports."

    SembScore is a Step-1 RL reward component, and it is defined by cosine similarity between CheXbert's 14-pathology indicator vectors for generated and reference reports. The later 'disease-level diagnostic accuracy' evaluation uses the same CheXbert model to label generated reports and compute F1. The model is therefore directly trained to align with CheXbert's label vectors, and the reported diagnostic gains are measured with that same labeler. The disease-level F1 result is thus at least partly forced by construction and is not an independent assessment of clinical diagnostic ability.

  3. self citation load bearing [References [26]/[27]; Methods – Inference and Evaluation Metrics (CheXprompt)]
    "CheXprompt [26] In addition to the ReXrank leaderboard metrics, we also evaluate report quality with CheXprompt, an LLM-based error detection metric. ... [26] Juan Manuel Zambrano Chaves, Shih-Cheng Huang, Yanbo Xu, Hanwen Xu, Naoto Usuyama, Sheng Zhang, ... [27] same citation."

    The only instrument supporting the paper's headline clinical-fidelity claim is CheXprompt, a GPT-4-based error metric introduced in a prior paper whose authors overlap substantially with the present paper (e.g., Sheng Zhang, Yanbo Xu, Hoifung Poon). The metric is cited as an authoritative external evaluator, but it is simultaneously the Step-2 training reward and is not independently re-validated on RL-optimized outputs in this work. The clinical-fidelity conclusion therefore rests on a self-referential loop: the model is shaped by the metric, evaluated with the metric, and the metric's authority is imported from the authors' own prior publication.

full rationale

The ReXrank SOTA claim is not itself circular: the Step-1 RL reward is the RadCliQ weighted composite, so the paper explicitly trains toward the headline metric, but the ReXrank test sets are held out and the leaderboard is externally checkable. The circularity is concentrated in the interpretive claims built on top of that optimization. The clinical-error-reduction claim uses CheXprompt both as the Step-2 training reward and as the evaluation metric, so the reported error reduction is the direct product of optimizing the evaluation instrument rather than independent evidence of clinical fidelity. Similarly, the disease-level F1 evaluation uses CheXbert, the same labeler that defines the SembScore reward, making the diagnostic-accuracy gain partly self-measured. No radiologist or expert-reader validation of the RL-optimized report styles is provided, and the CheXprompt citation is the authors' own prior work. These factors warrant a partial-circularity score of 6; the benchmark rankings and zero-shot comparisons retain independent, checkable content.

Axiom & Free-Parameter Ledger

4 free parameters · 7 axioms · 0 invented entities

The central claims rest on seven domain assumptions, all standard for this literature but unverified within the paper: the validity of the GPT-4-based CheXprompt judge (also the Step-2 reward and self-cited), the validity of RadCliQ-v1 as a human-judgment proxy, the soundness of GRPO/DAPO, the base-model choice, CheXbert's labeling accuracy, leakage-free ReXrank splits, and the accuracy of ground-truth reports. The four free parameters are the reward-mix constants (CheXprompt weight 0.5, KL 0.03) and the borrowed RadCliQ coefficients plus the roughly specified DAPO hyperparameters. No invented entities (particles, forces, dimensions) are introduced.

free parameters (4)
  • CheXprompt reward weight (Step 2) = 0.5
    Hand-chosen weight balancing the error-reduction reward against RadCliQ components in the second RL epoch (Methods, Optimization).
  • KL regularization coefficient (Step 2) = 0.03
    Hand-chosen coefficient to keep the Step-2 policy near the Step-1 policy (Methods, Optimization).
  • RadCliQ composite reward weights = BLEU 0; BERTScore 0.370; SembScore 0.253; RadGraph-F1 0.377
    Borrowed from the fitted RadCliQ-v1 metric (ref [24]); used as the Step-1 RL objective, making the training reward equal in form to the headline evaluation metric. These weights were themselves fit to human-judgment data in prior work.
  • RL rollout count and clipping threshold = 16 rollouts/query; 'higher' clipping threshold (unspecified)
    Engineering choices following DAPO (ref [25]) that affect exploration and entropy but are not stated precisely enough to reproduce exactly.
axioms (7)
  • domain assumption CheXprompt (GPT-4) clinical-error counts are accurate and sensitive to clinically meaningful errors in generated reports.
    Used as Step-2 reward and as the headline error metric (Figure 2f). Self-cited (refs 26/27); not validated against radiologists in this paper.
  • domain assumption RadCliQ-v1 is a valid proxy for human judgment of radiology report quality.
    The paper states it 'correlates more strongly with human judgment than individual metrics' (cited to ref [24]) and uses 1/RadCliQ-v1 as primary metric and as the Step-1 reward basis.
  • standard math GRPO with DAPO modifications (higher clipping, no KL in Step 1) is a valid and stable RL trainer for this setting.
    Adopted from refs [19]/[25]; no derivation in this paper.
  • domain assumption Qwen3-VL-8B-Instruct is a capable base for medical VLM fine-tuning.
    Foundation model chosen without a comparative ablation of base models.
  • domain assumption CheXbert (ref [20]) accurately labels 14 thoracic conditions in generated reports.
    Used for SembScore reward and for disease-level F1 evaluation (Figure 4b,c).
  • domain assumption ReXrank test splits (MIMIC 2,347; CheXpert Plus 200; ReXGradient 10,000 private; IU 590) are leakage-free relative to the public training splits used.
    Relies on the ReXrank protocol; an explicit leakage check is not described.
  • domain assumption Ground-truth radiology reports in the four training datasets are accurate clinical references.
    Standard assumption in report-generation benchmarking; errors in ground truth would propagate to both reward and evaluation.

pith-pipeline@v1.3.0-alltime-deepseek · 15393 in / 25774 out tokens · 242674 ms · 2026-08-03T08:24:57.002576+00:00 · methodology

0 comments
read the original abstract

Frontier models have demonstrated remarkable capabilities in understanding and reasoning with natural-language text, but they still exhibit major competency gaps in multimodal understanding and reasoning especially in high-value verticals such as biomedicine. Medical imaging report generation is a prominent example. Supervised fine-tuning can substantially improve performance, but they are prone to overfitting to superficial boilerplate patterns. In this paper, we introduce Universal Report Generation (UniRG) as a general framework for medical imaging report generation. By leveraging reinforcement learning as a unifying mechanism to directly optimize for evaluation metrics designed for end applications, UniRG can significantly improve upon supervised fine-tuning and attain durable generalization across diverse institutions and clinical practices. We trained UniRG-CXR on publicly available chest X-ray (CXR) data and conducted a thorough evaluation in CXR report generation with rigorous evaluation scenarios. On the authoritative ReXrank benchmark, UniRG-CXR sets new overall SOTA, outperforming prior state of the art by a wide margin. We release our model at https://huggingface.co/microsoft/UniRG-CXR.

Figures

Figures reproduced from arXiv: 2601.17151 by Guanghui Qin, Hoifung Poon, Mu Wei, Qianchu Liu, Sam Preston, Sheng Zhang, Sid Kiblawi, Timothy Ossowski, Tristan Naumann, Wen-wai Yim, Yanbo Xu, Ying Jin, Yu Gu.

Figure 1
Figure 1. Figure 1: Overview of UniRG-CXR. (a) Training Data: UniRG-CXR is trained on the training splits of MIMIC￾CXR [11], CheXpert Plus [4], ReXGradient-160k [30] and IU [6] covering diverse institutions and patient demographics. (b) Training and Rewards: Taking input from the current image, clinical context (e.g., indication), and optionally prior studies, UniRG-CXR uses GRPO reinforcement learning to optimize composite r… view at source ↗
Figure 2
Figure 2. Figure 2: UniRG-CXR achieves state-of-the-art performance, delivering consistent and comprehensive performance gains across metrics. (a) On the ReXrank leaderboard, UniRG-CXR (green) shows robust, universal improvement across all evaluation metrics. (b). Starting from the same SFT checkpoint, RL with our combined reward achieves more balanced gains across metrics and the highest RadCliQ-v1 score compared to RL on si… view at source ↗
Figure 3
Figure 3. Figure 3: UniRG-CXR enhances longitudinal report generation. (a). Comparing UniRG-CXR and its non￾longitudinal ablation with prior models on longitudinal report generation, we show UniRG-CXR exhibits the best performance and the longitudinal information is beneficial to the performance. (b). UniRG-CXR achieves the best performance across different longitudinal encounter points ranging from the first encounter to the… view at source ↗
Figure 4
Figure 4. Figure 4: Generalization and robustness of UniRG-CXR. (a). We held out two datasets sources (IU-Xray and PD (proprietary data) from the training data and evaluate UniRG-CXR in a zero-shot setting on these datasets . UniRG-CXR consistently outperforms prior models, maintaining substantial performance gains in this challenging setup. (b) and (c) present condition-level F1 scores on MIMIC-CXR and PD and highlight that … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

32 extracted references · 11 linked inside Pith

  1. [1]

    Qwen3-vl technical report, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan ...

    Pith/arXiv arXiv 2025

  2. [2]

    Learning to exploit temporal structure for biomedical vision-language processing

    Shruthi Bannur, Stephanie Hyland, Qianchu Liu, Fernando Perez-Garcia, Maximilian Ilse, Daniel C Castro, Benedikt Boecking, Harshita Sharma, Kenza Bouzid, Anja Thieme, et al. Learning to exploit temporal structure for biomedical vision-language processing. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15016–15027, 2023

    2023

  3. [3]

    Shruthi Bannur, Kenza Bouzid, Daniel C. Castro, Anton Schwaighofer, Anja Thieme, Sam Bond-Taylor, Maximilian Ilse, Fernando P´ erez-Garc ´ ıa, Valentina Salvatelli, Harshita Sharma, Felix Meissen, Mercy Ranjit, Shaury Srivastav, Julia Gong, Noel C. F. Codella, Fabian Falck, Ozan Oktay, Matthew P. Lungren, Maria Teodora Wetscherek, Javier Alvarez-Valle, an...

    Pith/arXiv arXiv 2024

  4. [4]

    Langlotz

    Pierre Chambon, Jean-Benoit Delbrouck, Thomas Sounack, Shih-Cheng Huang, Zhihong Chen, Maya Varma, Steven QH Truong, Chu The Chuong, and Curtis P. Langlotz. Chexpert plus: Augmenting a large chest x-ray dataset with text radiology reports, patient demographics and additional image formats,

  5. [5]

    Automated structured radiology report generation

    Jean-Benoit Delbrouck, Justin Xu, Johannes Moll, Alois Thomas, Zhihong Chen, Sophie Ostmeier, Asfandyar Azhar, Kelvin Zhenghao Li, Andrew Johnston, Christian Bluethgen, et al. Automated structured radiology report generation. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 26813–26829, 2025

    2025

  6. [6]

    Preparing a collection of radiology examinations for distribution and retrieval, 2015

    Dina Demner-Fushman, Marc D Kohli, Marc B Rosenman, Sonya E Shooshan, Laritza Rodriguez, Sameer Antani, George R Thoma, and Clement J McDonald. Preparing a collection of radiology examinations for distribution and retrieval, 2015

    2015

  7. [7]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pp. 4171–4186, 2019

    2019

  8. [8]

    How to create a great radiology report.Radiographics, 40(6):1658–1670, 2020

    Michael P Hartung, Ian C Bickle, Frank Gaillard, and Jeffrey P Kanne. How to create a great radiology report.Radiographics, 40(6):1658–1670, 2020

    2020

  9. [9]

    Generative artificial intelligence for chest radiograph interpretation in the emergency department.JAMA network open, 6(10):e2336100–e2336100, 2023

    Jonathan Huang, Luke Neill, Matthew Wittbrodt, David Melnick, Matthew Klug, Michael Thompson, John Bailitz, Timothy Loftus, Sanjeev Malik, Amit Phull, et al. Generative artificial intelligence for chest radiograph interpretation in the emergency department.JAMA network open, 6(10):e2336100–e2336100, 2023

    2023

  10. [10]

    Radgraph: Extracting clinical entities and relations from radiology reports

    Saahil Jain, Ashwin Agrawal, Adriel Saporta, Steven Truong, Du Nguyen Duong, Tan Bui, Pierre Chambon, Yuhao Zhang, Matthew P Lungren, Andrew Y Ng, et al. Radgraph: Extracting clinical entities and relations from radiology reports. InThirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021

    2021

  11. [11]

    Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports.Scientific data, 6(1):317, 2019

    Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng. Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports.Scientific data, 6(1):317, 2019

    2019

  12. [12]

    Toward best practices in radiology reporting.Radiology, 252(3):852–856, 2009

    Charles E Kahn Jr, Curtis P Langlotz, Elizabeth S Burnside, John A Carrino, David S Channin, David M Hovsepian, and Daniel L Rubin. Toward best practices in radiology reporting.Radiology, 252(3):852–856, 2009

    2009

  13. [13]

    Clinically accurate chest x-ray report generation

    Guanxiong Liu, Tzu-Ming Harry Hsu, Matthew McDermott, Willie Boag, Wei-Hung Weng, Peter Szolovits, and Marzyeh Ghassemi. Clinically accurate chest x-ray report generation. InMachine Learning for Healthcare Conference, pp. 249–269. PMLR, 2019

    2019

  14. [14]

    The current status and future of fda-approved artificial intelligence tools in chest radiology in the united states.Clinical Radiology, 78(2):115–122, 2023

    ME Milam and CW Koo. The current status and future of fda-approved artificial intelligence tools in chest radiology in the united states.Clinical Radiology, 78(2):115–122, 2023

    2023

  15. [15]

    Improving chest X-ray report generation by leveraging warm starting.Artificial Intelligence in Medicine, 144:102633, 2023

    Aaron Nicolson, Jason Dowling, and Bevan Koopman. Improving chest X-ray report generation by leveraging warm starting.Artificial Intelligence in Medicine, 144:102633, 2023. ISSN 0933-3657. doi: https://doi.org/10.1016/j.artmed.2023.102633. URL https://www.sciencedirect.com/science/ article/pii/S0933365723001471

    arXiv 2023

  16. [16]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. InProceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311–318, 2002

    2002

  17. [17]

    Steiner, Can Kirmizibayrak, Rory Pilgrim, Daniel Golden, and Lin Yang

    Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, C ´ ıan Hughes, Charles Lau, Justin Chen, Fereshteh Mahvar, Liron Yatziv, Tiffany Chen, Bram Sterling, Stefanie Anna Baby, Susanna Maria Baby, Jeremy Lai, Samuel 12 Schmidgall, Lu Yang, Kejia Chen, Per Bjornsson, Shashir Reddy, R...

    Pith/arXiv arXiv 2025

  18. [18]

    Controllable chest x-ray report generation from longitudinal representations.arXiv preprint arXiv:2310.05881, 2023

    Francesco Dalla Serra, Chaoyang Wang, Fani Deligianni, Jeffrey Dalton, and Alison Q O’Neil. Controllable chest x-ray report generation from longitudinal representations.arXiv preprint arXiv:2310.05881, 2023

    Pith/arXiv arXiv 2023

  19. [19]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  20. [20]

    Ng, and Matthew P

    Akshay Smit, Saahil Jain, Pranav Rajpurkar, Anuj Pareek, Andrew Y. Ng, and Matthew P. Lungren. Chexbert: Combining automatic labelers and expert annotations for accurate radiology report labeling using bert, 2020. URLhttps://arxiv.org/abs/2004.09167

    Pith/arXiv arXiv 2020

  21. [21]

    Interactive and explainable region- guided radiology report generation

    Tim Tanida, Philip M¨ uller, Georgios Kaissis, and Daniel Rueckert. Interactive and explainable region- guided radiology report generation. InCVPR, 2023

    2023

  22. [22]

    External validation of deep learning algorithms for radiologic diagnosis: a systematic review.Radiology: Artificial Intelligence, 4(3):e210064, 2022

    Alice C Yu, Bahram Mohajer, and John Eng. External validation of deep learning algorithms for radiologic diagnosis: a systematic review.Radiology: Artificial Intelligence, 4(3):e210064, 2022

    2022

  23. [24]

    Evaluating progress in automatic chest x-ray radiology report generation.Patterns, 4(9), 2023

    Feiyang Yu, Mark Endo, Rayan Krishnan, Ian Pan, Andy Tsai, Eduardo Pontes Reis, Eduardo Kaiser Ururahy Nunes Fonseca, Henrique Min Ho Lee, Zahra Shakeri Hossein Abad, Andrew Y Ng, et al. Evaluating progress in automatic chest x-ray radiology report generation.Patterns, 4(9), 2023

    2023

  24. [25]

    Dapo: An open-source llm reinforcement learning system at scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025

    Pith/arXiv arXiv 2025

  25. [27]

    A clinically accessible small multimodal radiology model and evaluation metric for chest x-ray findings.Nature Communications, 16(1):3108, 2025

    Juan Manuel Zambrano Chaves, Shih-Cheng Huang, Yanbo Xu, Hanwen Xu, Naoto Usuyama, Sheng Zhang, Fei Wang, Yujia Xie, Mahmoud Khademi, Ziyi Yang, et al. A clinically accessible small multimodal radiology model and evaluation metric for chest x-ray findings.Nature Communications, 16(1):3108, 2025

    2025

  26. [28]

    Bertscore: Evaluating text generation with bert

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. InInternational Conference on Learning Representations, 2019

    2019

  27. [29]

    Rexrank: A public leaderboard for ai-powered radiology report generation

    Xiaoman Zhang, Hong-Yu Zhou, Xiaoli Yang, Oishi Banerjee, Juli´ an N Acosta, Josh Miller, Ouwen Huang, and Pranav Rajpurkar. Rexrank: A public leaderboard for ai-powered radiology report generation. arXiv preprint arXiv:2411.15122, 2024. 13

    Pith/arXiv arXiv 2024

  28. [30]

    Acosta, Josh Miller, Ouwen Huang, and Pranav Rajpurkar

    Xiaoman Zhang, Juli´ an N. Acosta, Josh Miller, Ouwen Huang, and Pranav Rajpurkar. Rexgradient- 160k: A large-scale publicly available dataset of chest radiographs with free-text reports, 2025. URL https://arxiv.org/abs/2505.00228

    Pith/arXiv arXiv 2025

  29. [31]

    Ratescore: A metric for radiology report generation

    Weike Zhao, Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. Ratescore: A metric for radiology report generation. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 15004–15019, 2024

    2024

  30. [32]

    Topol, and Pranav Rajpurkar

    Hong-Yu Zhou, Juli´ an Nicol´ as Acosta, Subathra Adithan, Suvrankar Datta, Eric J. Topol, and Pranav Rajpurkar. Medversa: A generalist foundation model for medical image interpretation, 2025. URL https://arxiv.org/abs/2405.07988

    Pith/arXiv arXiv 2025

  31. [33]

    Utilizing longitudinal chest x-rays and reports to pre-fill radiology reports

    Qingqing Zhu, Tejas Sudharshan Mathai, Pritam Mukherjee, Yifan Peng, Ronald M Summers, and Zhiyong Lu. Utilizing longitudinal chest x-rays and reports to pre-fill radiology reports. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 189–198. Springer, 2023. Supplementary Information Comparing SFT and RL To identi...

    2023

  32. [2024]

    URLhttps://arxiv.org/abs/2405.19538

    Pith/arXiv arXiv