Pith. sign in

REVIEW 3 major objections 6 minor 67 references

Stabilizing reinforcement learning with response-weighted KL penalties and validation resets produces higher-quality automated chest X-ray reports.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 13:55 UTC pith:3I3PANVV

load-bearing objection Solid GRPO stabilizer for CXR report gen with real ablations and code; SOTA CE gains partly ride on BLEU reward plus validation that already mixes RadGraph. the 3 major comments →

arxiv 2607.10147 v1 pith:3I3PANVV submitted 2026-07-11 cs.CV

REVA-PO: Stabilizing Reinforcement Learning for Chest X-ray Report Generation

Li Guo , Anas M. Tahir , Z. Jane Wang This is my paper
classification cs.CV
keywords reinforcement learningKL regularizationchest X-ray report generationvision-language modelspolicy optimizationmedical imagingGRPO
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Automated chest X-ray reports still lag professional writing in fluency and clinical precision, and reinforcement learning often makes that worse by either choking exploration or collapsing after long training. This paper argues that the usual fixed KL penalty and static reference policy are the culprits: good new behaviors get over-penalized, bad ones under-penalized, and pressure piles up until learning fails. REVA-PO fixes both problems. Response-Weighted Regularization scales the KL weight for each generated report from its advantage and the reference policy’s entropy, loosening the leash on promising answers and tightening it on poor ones. Validation-Anchored Policy Reset periodically rewinds the reference and current policies to the best validation checkpoint, clearing that pressure so exploration can continue. A three-stage pipeline—warm-up of a pretrained vision-language model, classifier-guided supervised fine-tuning, then this RL stage—puts the model in a strong starting place. On MIMIC-CXR and IU-Xray the method reports new peaks in both language metrics and clinical scores, including clear gains in BLEU-4, CheXpert F1, and RadGraph F1 over prior leaders.

Core claim

The authors claim that fixed KL regularization and a static reference policy are the main reasons RL for chest X-ray report generation is unstable or under-exploratory, and that pairing response-level adaptive KL weights (RER) with periodic resets of both reference and current policies to the validation-best checkpoint (VAPR) stabilizes long training, expands usable exploration, and yields state-of-the-art linguistic and clinical report quality on standard public benchmarks when started from a three-stage LVLM pipeline.

What carries the argument

Response-Weighted Regularization (RER): for each sampled report it sets a KL weight β_i from standardized advantage and length-normalized reference entropy—lowering β for high-advantage uncertain (novel good) responses and raising β for low-advantage uncertain (bad) ones—combined with Validation-Anchored Policy Reset (VAPR), which every M epochs sets reference, current, and old policies to the best validation checkpoint so accumulated KL pressure is zeroed and exploration restarts from a validated anchor.

Load-bearing premise

The method treats BLEU-4 overlap with the reference report as a good enough reward signal that improving it will also improve clinical accuracy.

What would settle it

Run the same REVA-PO pipeline with BLEU-4 as the only reward, then check whether CheXpert F1 and RadGraph F1 still rise on a held-out test split or an external hospital dataset; if n-gram scores climb while clinical metrics stall or fall, the proxy-reward claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Long-horizon RL for medical report generation can continue without the late collapse that fixed-reference KL often produces.
  • Pretrained LVLMs plus light warm-up and frozen-classifier prompts can replace heavy contrastive image-encoder pretraining, especially on smaller sets like IU-Xray.
  • Response-level KL scaling improves sample efficiency relative to a constant KL coefficient under the same GRPO-style objective.
  • Simultaneous gains in BLEU-style overlap and clinical entity/relation consistency become achievable when exploration is protected rather than throttled.
  • Periodic validation-anchored resets give a practical alternative to EMA reference tracking when reward hacking is a concern.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same RER-plus-VAPR pattern is a natural candidate for other sequence RL settings where fixed KL either freezes learning or ends in collapse, such as clinical note generation or dense image captioning.
  • Replacing BLEU-4 with a direct clinical reward (RadGraph F1 or labeler F1) inside the same machinery could further close remaining gaps between fluency and factual consistency.
  • Validation resets effectively turn the reference into a moving trust region whose center is chosen by empirical validation rather than by blind tracking of the online policy.
  • Keeping a frozen disease classifier only in the SFT stage suggests that external clinical cues remain useful scaffolding even when the generator itself is a strong LVLM.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript proposes REVA-PO, a GRPO-style RL framework for LVLM-based chest X-ray report generation that combines Response-Weighted Regularization (RER) and Validation-Anchored Policy Reset (VAPR). RER sets a per-response KL coefficient β_i from the group-relative advantage A_i and normalized reference-policy entropy (Eqs. 7–9), relaxing the penalty on high-advantage, high-entropy responses and tightening it on low-advantage ones. VAPR periodically resets both the reference and current policies to the validation-best checkpoint (Algorithm 1; Eq. 10), clearing accumulated KL pressure. Training uses a three-stage pipeline (merger warm-up, classifier-guided SFT, then REVA-PO RL) with BLEU-4 as the reward. On MIMIC-CXR and IU-Xray the method reports new SOTA NLG and clinical-efficacy numbers, including relative BLEU-4 gains of 5.1% and 3.6% and CheXpert F1 / RadGraph F1 gains of 4.5% and 12.8% over prior leading methods (Tables 1–2), with ablations of the stages, RER, and VAPR (Tables 3–4, Fig. 3).

Significance. Stabilizing long-horizon RL for medical report generation is a genuine practical problem; fixed-β KL and static references are known failure modes. RER is a clean, preference-free, critic-free response-level schedule that drops into GRPO/PPO-style objectives, and VAPR is a simple alternative to EMA reference tracking that is grounded in validation rather than online tracking. The three-stage LVLM pipeline (no separate image-encoder contrastive pretraining) is a useful engineering contribution for smaller datasets such as IU-Xray. Public code, multi-baseline comparisons, and collapse curves for the no-KL / no-VAPR settings strengthen the empirical case. If the stability and SOTA claims hold under stronger statistical and reward-alignment checks, the work is a solid incremental advance for RL-based CXR-RG.

major comments (3)
  1. Central CE claim vs. BLEU-4 reward (Abstract; Sec. 3.3; Eqs. 2–3; Table 2). The paper optimizes pure BLEU-4 rewards while claiming new SOTA CheXpert F1 (+4.5%) and RadGraph F1 (+12.8%). Attribution of those CE lifts to RER-weighted policy gradients load-bears on directional alignment between per-response BLEU advantages A_i and clinical quality (asserted in Supp. A). Table 3 shows Stage-2 F1 0.474 → full 0.506, but VAPR’s validation score already mixes RadGraph F1 (Eq. 10), so model selection can inject CE signal independent of the BLEU reward. Table 4 / Fig. 3 show that RER and VAPR raise the BLEU reward curve and final CE numbers relative to CKL / no-VAPR, yet they do not report whether high-A_i group members are systematically better on CheXpert/RadGraph. Please add (i) a correlation or rank analysis of A_i vs. CE metrics of the sampled responses, and/or (ii) a controlled run that fre
  2. Statistical support for SOTA deltas (Tables 1–2; Abstract). Relative gains of 5.1% BLEU-4, 4.5% CheXpert F1, and 12.8% RadGraph F1 are reported as point estimates from a single training run with no error bars, multi-seed means, or significance tests. Given the free parameters (β_min/β_base/β_max, G, M, N, λ, clip range) and the known variance of group-sampled RL, these margins are not yet established as robust. At minimum, report 3-seed means±std for the main REVA-PO row on MIMIC-CXR (NLG + CE) and note whether the ranking vs. AM-MRG / STREAM is stable.
  3. VAPR semantics and exploration claim (Sec. 3.3; Algorithm 1; Figs. 2c–d, 3c–d). Every M epochs the method sets π_ref ← π_θbest and also overwrites π_θ and π_θold with θ_best, forcing D_KL=0 and discarding any non-best current policy. The ablation (Table 4 row 4; Fig. 3c) shows collapse without this reset, which supports a stability benefit, but the narrative of “expanding the viable exploration space” is only partially accurate: the reachable set is re-centered on the validation-best point, and intermediate exploratory mass is thrown away. Please clarify in the text that VAPR is a hard restart from the best validated anchor (not a soft expansion around the current policy), and discuss whether performance is sensitive to M (e.g., M∈{1,2,4}) so readers can judge the stability–progress trade-off.
minor comments (6)
  1. Title / running header inconsistency: the PDF header shows “REV A-PO” (space) while the title is “REVA-PO”; unify.
  2. Eq. (3) and the surrounding text mix J_REVA−PO and J_REV A−PO notation; keep a single spelling.
  3. Fig. 1 legend and Stage-3 description: “Classifier omitted in Stage 3” is clear, but the prompt construction q=[I,V,D] vs q=[I,V] across stages should be stated once in the main text for reproducibility.
  4. Table 1: several baselines have missing B-2/B-3 or METEOR entries; a short note that those numbers were not reported by the original papers would avoid the appearance of selective comparison.
  5. Hyperparameters β_min, β_base, β_max, G, M, N, λ, ε_low/ε_high are only deferred to the supplement; a one-line default table in the main paper (or caption of Algorithm 1) would help readers reproduce the RER schedule without the supplement.
  6. Related Work: SCST is correctly identified as high-variance; a brief quantitative comparison (e.g., one SCST run under the same LVLM backbone) would make the GRPO choice more concrete, but is optional.

Circularity Check

0 steps flagged

No circularity: empirical RL method with held-out test SOTA; validation selection and BLEU reward are standard, not definitional of the claims.

full rationale

REVA-PO is an engineering contribution (RER response-level βi from Ai and Href_i in Eqs. 7–9; VAPR resets of πref/πθ to θbest every M epochs via Algorithm 1 and Eq. 10) evaluated by direct comparison of NLG/CE metrics against external published baselines on the official MIMIC-CXR and IU-Xray test splits (Tables 1–2). The three-stage pipeline, GRPO-style objective (Eqs. 2–6), and ablations (Tables 3–4, Fig. 3) are self-contained; no quantity is algebraically forced by its own definition, no parameter is fitted on the reported test numbers and then re-presented as a prediction, and no uniqueness theorem or ansatz is imported via self-citation. VAPR’s use of a validation composite that includes RadGraph F1 is ordinary model selection, not a construction that makes the test-set CheXpert/RadGraph numbers true by definition. The BLEU-4 reward’s claimed directional alignment with CE (Supp. A) is an empirical assumption whose validity can be questioned on correctness grounds, but it does not render the reported lifts circular. Score 0 is therefore required.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

Central claims rest on standard RL/LM math, domain choices (BLEU-4 reward, CheXpert/RadGraph evaluation, frozen classifier cues), and several hand-set training knobs (β bounds, reset periods, group size, validation mix λ). No new physical entities; RER/VAPR are algorithmic constructs defined in the paper.

free parameters (5)
  • β_min, β_base, β_max
    Bounds and base for per-response KL weight in Eq. (9); chosen by design to control exploration vs stability and not derived from first principles.
  • group size G
    Number of sampled reports per prompt for GRPO-style advantages; affects variance and compute, set experimentally.
  • reference reset period M (epochs) and sampling sync period N (steps)
    VAPR and old-policy sync schedules in Algorithm 1; free schedule hyperparameters that control when KL pressure is cleared.
  • validation score mix λ (RadGraph weight in sval)
    Balances NLG metrics vs RadGraph F1 when selecting θ_best (Eq. 10); hand-chosen trade-off for checkpointing.
  • PPO-style clip range (ε_low, ε_high) and learning rate η
    Standard RL optimization knobs that affect stability of J_REVA-PO; fitted/tuned rather than predicted.
axioms (5)
  • domain assumption Group-relative standardized advantages from G samples (Eq. 2) are a valid baseline without a learned critic.
    Inherited from GRPO; assumed adequate for CXR report rewards in Sec. 3.3.
  • domain assumption BLEU-4 vs reference reports is a low-variance reward that directionally tracks clinical efficacy metrics.
    Stated in Sec. 3.3 and Supplementary A; load-bearing for interpreting CE gains as RL success.
  • domain assumption A frozen CheXpert-style classifier’s positive labels appended to the prompt improve lesion-relevant generation without backpropagating into the generator.
    Stage 2 design (Sec. 3.2); ablation Table 3 supports gains but the cue quality depends on classifier accuracy.
  • standard math Unbiased token-level KL estimator (Eq. 6) and token-averaged objective (Eq. 3) stabilize long-sequence report training.
    Standard policy-gradient / KL practice cited via prior RL work; used as given.
  • ad hoc to paper Periodic hard reset of π_ref and π_θ to validation-best θ_best expands exploration without inducing harmful validation overfitting on the final test metrics.
    Core VAPR premise (Sec. 3.3, Figs. 2c–d); not proven, supported by collapse ablations.
invented entities (2)
  • Response-Weighted Regularization (RER) β_i(A_i, Ĥ_ref_i) no independent evidence
    purpose: Per-response KL coefficient that relaxes constraints on high-advantage high-entropy responses and tightens them on low-advantage high-entropy responses.
    Defined in Eqs. (7)–(9); algorithmic invention of this paper, not independently measured outside the training loop.
  • Validation-Anchored Policy Reset (VAPR) no independent evidence
    purpose: Periodic synchronization of reference and current policies to the best validation checkpoint to clear accumulated KL pressure.
    Defined in Sec. 3.3 and Algorithm 1; mechanism introduced here as alternative to fixed reference or EMA.

pith-pipeline@v1.1.0-grok45 · 21740 in / 3499 out tokens · 38144 ms · 2026-07-14T13:55:58.069796+00:00 · methodology

0 comments
read the original abstract

Automated chest X-ray report generation has recently benefited from reinforcement learning (RL) and large language models. However, RL training often suffers from instability or limited exploration due to fixed Kullback-Leibler (KL) regularization and a static reference policy that accumulates KL pressure over time. We propose Response-Weighted and Validation-Anchored Policy Optimization (REVA-PO), a RL framework that stabilizes long-term training via Response-Weighted Regularization (RER) and Validation-Anchored Policy Reset (VAPR). RER dynamically adjusts per-response KL weights based on advantage and reference-policy entropy, relaxing constraints for high-quality responses while tightening them for low-quality ones. Complementarily, VAPR periodically synchronizes the reference and current policies to the best validation checkpoint, resetting accumulated regularization pressure to expand the viable exploration space. To ensure a robust starting point, we employ a three-stage pipeline consisting of warm-up training, classifier-guided supervised fine-tuning, and RL. Extensive evaluations on MIMIC-CXR and IU-Xray demonstrate that REVA-PO sets new state-of-the-art benchmarks in both linguistic quality and clinical accuracy. Notably, BLEU-4 improves by 5.1% on MIMIC-CXR and 3.6% on IU-Xray, while CheXpert F1 and RadGraph F1 scores increase by 4.5% and 12.8%, respectively, over prior leading methods. The code is publicly available at https://github.com/LiGuo12/REVA_PO/.

Figures

Figures reproduced from arXiv: 2607.10147 by Anas M. Tahir, Li Guo, Z. Jane Wang.

Figure 1
Figure 1. Figure 1: The three-stage training pipeline. Stage 1: Warm-up. Freeze the image encoder and LLM; train only the merger with LLM. Stage 2: Classifier-guided SFT. Build prompt q = [I, V, D] by appending predicted labels D; freeze the classifier; train the merger and LoRA-tune the encoder and LLM with LLM. Stage 3: RL with REVA-PO. πθold samples G reports, computes advantages Ai, and maximizes JREVA−PO with entropy bon… view at source ↗
Figure 2
Figure 2. Figure 2: (Left) Mapping from the normalized reference-policy entropy Hˆref i to the per￾response KL weight βi. For advantage Ai > 0, the line has slope k = −(βbase − βmin); for Ai < 0, the line has slope k = βmax − βbase. (Right) Fixed Reference vs. Validation￾Anchored Policy Reset (VAPR). The reference policy πref is the anchor (gray), the current policy πθ is the ball (blue), and the KL penalty DKL(πθ ∥ πref) is … view at source ↗
Figure 3
Figure 3. Figure 3: Ablation study of RL-stage modifications on MIMIC-CXR. Mean reward of responses sampled from πθold vs. training steps. (a) Response-Weighted Regularization (RER) outperforms constant KL (CKL). (b) Omitting the KL term causes early collapse. (c) Validation-Anchored Policy Reset (VAPR) prevents collapse and boosts rewards; green lines mark πref updates. (d) KL loss vs. training steps, plotted on a symmetric … view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison examples on MIMIC-CXR: baseline (Qwen2.5-VL-7B￾Instruct) and Stage 1 to Stage 3. Columns: input image; reference with ground-truth (GT) categories; and outputs with classifier-predicted categories. Sentences referring to the same anatomical region are color-coded and key phrases are underlined. Case 2. The baseline detects the pacemaker but misses heart enlargement. Stage 1 and Stage… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

67 extracted references · 17 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2502.13923 (2025) 2, 4, 10

    Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., et al.: Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923 (2025) 2, 4, 10

    Pith/arXiv arXiv 2025

  2. [2]

    arXiv preprint arXiv:2204.05862 (2022) 2, 4

    Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al.: Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 (2022) 2, 4

    Pith/arXiv arXiv 2022

  3. [3]

    In: Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization

    Banerjee, S., Lavie, A.: Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In: Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization. pp. 65–72 (2005) 9

    2005

  4. [4]

    American Journal of Roentgenology175(1), 17–22 (2000) 1

    Berlin, L.: Liability of interpreting too many radiographs. American Journal of Roentgenology175(1), 17–22 (2000) 1

    2000

  5. [5]

    IEEE Transactions on Image Processing (2025) 4, 10, 11, 12

    Chen, W., Liu, Y., Wang, C., Zhu, J., Li, G., Liu, C.L., Lin, L.: Cross-modal causal representation learning for radiology report generation. IEEE Transactions on Image Processing (2025) 4, 10, 11, 12

    2025

  6. [6]

    arXiv preprint arXiv:2506.16141 (2025) 2, 4

    Chen, Y., Ge, Y., Wang, R., Ge, Y., Cheng, J., Shan, Y., Liu, X.: Grpo-care: Consistency-aware reinforcement learning for multimodal reasoning. arXiv preprint arXiv:2506.16141 (2025) 2, 4

    Pith/arXiv arXiv 2025

  7. [7]

    Chen, Z., Shen, Y., Song, Y., Wan, X.: Generating radiology reports via memory- driven transformer. In: Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Aug 2021) 9

    2021

  8. [8]

    arXiv preprint arXiv:2502.01456 (2025) 12

    Cui, G., Yuan, L., Wang, Z., Wang, H., Li, W., He, B., Fan, Y., Yu, T., Xu, Q., Chen, W., et al.: Process reinforcement through implicit rewards. arXiv preprint arXiv:2502.01456 (2025) 12

    Pith/arXiv arXiv 2025

  9. [9]

    In: Findings of the Association for Computational Linguistics: ACL 2024

    Delbrouck, J.B., Chambon, P., Chen, Z., Varma, M., Johnston, A., Blankemeier, L., Van Veen, D., Bui, T., Truong, S., Langlotz, C.: Radgraph-xl: A large-scale expert-annotated dataset for entity and relation extraction from radiology reports. In: Findings of the Association for Computational Linguistics: ACL 2024. pp. 12902–12915 (2024) 8, 10

    2024

  10. [10]

    Journal of the American Medical Informatics Association23(2), 304–310 (2015) 9 16 L

    Demner-Fushman, D., Kohli, M.D., Rosenman, M.B., Shooshan, S.E., Rodriguez, L., Antani, S., Thoma, G.R., McDonald, C.J.: Preparing a collection of radiology examinations for distribution and retrieval. Journal of the American Medical Informatics Association23(2), 304–310 (2015) 9 16 L. Guo et al

    2015

  11. [11]

    arXiv preprint arXiv:2407.21783 (2024) 2, 10

    Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al.: The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024) 2, 10

    Pith/arXiv arXiv 2024

  12. [12]

    Nature645(8081), 633–638 (2025) 2, 4

    Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al.: Deepseek-r1 incentivizes reasoning in llms through reinforcement learning. Nature645(8081), 633–638 (2025) 2, 4

    2025

  13. [13]

    APSIPA Transactions on Signal and Information Processing13(1) (2024) 1

    Guo, L., Tahir, A.M., Zhang, D., Wang, Z.J., Ward, R.K., et al.: Automatic medical report generation: Methods and applications. APSIPA Transactions on Signal and Information Processing13(1) (2024) 1

    2024

  14. [14]

    ICLR1(2), 3 (2022) 6

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: Lora: Low-rank adaptation of large language models. ICLR1(2), 3 (2022) 6

    2022

  15. [15]

    arXiv preprint arXiv:2503.24290 (2025) 12

    Hu, J., Zhang, Y., Han, Q., Jiang, D., Zhang, X., Shum, H.Y.: Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model. arXiv preprint arXiv:2503.24290 (2025) 12

    Pith/arXiv arXiv 2025

  16. [16]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Huang, Z., Zhang, X., Zhang, S.: Kiut: Knowledge-injected u-transformer for radiology report generation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 19809–19818 (2023) 4

    2023

  17. [17]

    In: Proceedings of the AAAI conference on artificial intelligence

    Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al.: Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI conference on artificial intelligence. vol. 33, pp. 590–597 (2019) 6

    2019

  18. [18]

    IEEE Transactions on Medical Imaging (2025) 2, 4, 10, 11, 12

    Jin, H., Che, H., He, S., Chen, H.: A chain of diagnosis framework for accurate and explainable radiology report generation. IEEE Transactions on Medical Imaging (2025) 2, 4, 10, 11, 12

    2025

  19. [19]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Jin, H., Che, H., Lin, Y., Chen, H.: Promptmrg: Diagnosis-driven prompts for medical report generation. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 2607–2615 (2024) 3, 6, 11, 12

    2024

  20. [20]

    In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

    Jing, B., Wang, Z., Xing, E.: Show, describe and conclude: On exploiting the structure information of chest X-ray reports. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. pp. 6570–6580 (2019) 2

    2019

  21. [21]

    arXiv preprint arXiv:1901.07042 (2019) 9

    Johnson, A.E., Pollard, T.J., Greenbaum, N.R., Lungren, M.P., Deng, C.y., Peng, Y., Lu, Z., Mark, R.G., Berkowitz, S.J., Horng, S.: Mimic-cxr-jpg, a large publicly available database of labeled chest radiographs. arXiv preprint arXiv:1901.07042 (2019) 9

    Pith/arXiv arXiv 1901

  22. [22]

    Journal of the American College of Radiology7(9), 698–704 (2010) 1

    Krupinski, E.A., Berbaum, K.S., Caldwell, R.T., Schartz, K.M., Kim, J.: Long radiology workdays reduce detection and accommodation accuracy. Journal of the American College of Radiology7(9), 698–704 (2010) 1

    2010

  23. [23]

    In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025) 2, 4

    Lee, S., Han, J., Song, H., Choi, S.J., Lee, H., Yu, Y.: KL penalty control via pertur- bation for direct preference optimization. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025) 2, 4

    2025

  24. [24]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Li, M., Lin, B., Chen, Z., Lin, H., Liang, X., Chang, X.: Dynamic graph en- hanced contrastive learning for chest x-ray report generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3334–3343 (2023) 4, 9, 11, 12

    2023

  25. [25]

    In: Text summarization branches out

    Lin, C.Y.: Rouge: A package for automatic evaluation of summaries. In: Text summarization branches out. pp. 74–81 (2004) 9

    2004

  26. [26]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Liu, C., Tian, Y., Chen, W., Song, Y., Zhang, Y.: Bootstrapping large language models for radiology report generation. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 18635–18643 (2024) 2, 4, 9, 10, 11, 12 REVA-PO for CXR Report Generation 17

    2024

  27. [27]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Liu, F., Wu, X., Ge, S., Fan, W., Zou, Y.: Exploring and distilling posterior and prior knowledge for radiology report generation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 13753–13762 (2021) 4

    2021

  28. [28]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Liu, H., Li, C., Li, Y., Lee, Y.J.: Improved baselines with visual instruction tuning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 26296–26306 (2024) 5

    2024

  29. [29]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention

    Liu, H., Wei, D., Xu, Z., Wu, X., Zheng, Y., Wang, L.: Rrg-dpo: Direct preference optimization for clinically accurate radiology report generation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 552–562. Springer (2025) 11, 12

    2025

  30. [30]

    Advances in neural information processing systems35, 27730–27744 (2022) 2, 4

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al.: Training language models to follow instructions with human feedback. Advances in neural information processing systems35, 27730–27744 (2022) 2, 4

    2022

  31. [31]

    In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics

    Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics. pp. 311–318 (2002) 9

    2002

  32. [32]

    In: Findings of the Association for Computational Linguistics: ACL 2022

    Qin, H., Song, Y.: Reinforced cross-modal alignment for radiology report generation. In: Findings of the Association for Computational Linguistics: ACL 2022. pp. 448–458 (2022) 2, 4, 11, 12

    2022

  33. [33]

    Advances in neural information processing systems36, 53728–53741 (2023) 4

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C.D., Ermon, S., Finn, C.: Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems36, 53728–53741 (2023) 4

    2023

  34. [34]

    arXiv preprint arXiv:2406.16768 (2024) 2, 4

    Ramé, A., Ferret, J., Vieillard, N., Dadashi, R., Hussenot, L., Cedoz, P.L., Sessa, P.G., Girgin, S., Douillard, A., Bachem, O.: Warp: On the benefits of weight averaged rewarded policies. arXiv preprint arXiv:2406.16768 (2024) 2, 4

    Pith/arXiv arXiv 2024

  35. [35]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Rennie, S.J., Marcheret, E., Mroueh, Y., Ross, J., Goel, V.: Self-critical sequence training for image captioning. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 7008–7024 (2017) 4

    2017

  36. [36]

    BMJ: British Medical Journal (Online)359(2017) 1

    Rimmer, A.: Radiologist shortage leaves patient care at risk, warns royal college. BMJ: British Medical Journal (Online)359(2017) 1

    2017

  37. [37]

    arXiv preprint arXiv:1707.06347 (2017) 2, 4

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017) 2, 4

    Pith/arXiv arXiv 2017

  38. [38]

    arXiv preprint arXiv:2402.03300 (2024) 2, 4

    Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y., Wu, Y., et al.: Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 (2024) 2, 4

    Pith/arXiv arXiv 2024

  39. [39]

    arXiv preprint arXiv:2504.07615 (2025) 4

    Shen, H., Liu, P., Li, J., Fang, C., Ma, Y., Liao, J., Shen, Q., Zhang, Z., Zhao, K., Zhang, Q., et al.: Vlm-r1: A stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615 (2025) 4

    Pith/arXiv arXiv 2025

  40. [40]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Shen, H., Pei, M., Liu, J., Tian, Z.: Automatic radiology reports generation via memory alignment network. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 4776–4783 (2024) 9, 11, 12

    2024

  41. [41]

    IEEE Reviews in Biomedical Engineering 18, 368–387 (2024) 1

    Sloan, P., Clatworthy, P., Simpson, E., Mirmehdi, M.: Automated radiology report generation: A review of recent advances. IEEE Reviews in Biomedical Engineering 18, 368–387 (2024) 1

    2024

  42. [42]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Tanida, T., Müller, P., Kaissis, G., Rueckert, D.: Interactive and explainable region- guided radiology report generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7433–7442 (2023) 11, 12

    2023

  43. [43]

    In: European Conference on Computer Vision

    Wang, F., Du, S., Yu, L.: Hergen: Elevating radiology report generation with longitudinal data. In: European Conference on Computer Vision. pp. 183–200. Springer (2024) 2, 4, 11, 12 18 L. Guo et al

    2024

  44. [44]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention

    Wang, L., Ning, M., Lu, D., Wei, D., Zheng, Y., Chen, J.: An inclusive task-aware framework for radiology report generation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 568–577. Springer (2022) 3

    2022

  45. [45]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Wang, X., Wang, F., Li, Y., Ma, Q., Wang, S., Jiang, B., Tang, J.: Cxpmrg-bench: Pre-training and benchmarking for x-ray medical report generation on chexpert plus dataset. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 5123–5133 (2025) 2, 4, 9, 10, 11, 12

    2025

  46. [46]

    IEEE Transactions on Medical Imaging45, 583–595 (2026) 4, 10, 11, 12

    Wang, X., Wang, F., Wang, H., Jiang, B., Li, C., Wang, Y., Tian, Y., Tang, J.: Activating associative disease-aware vision token memory for llm-based x-ray report generation. IEEE Transactions on Medical Imaging45, 583–595 (2026) 4, 10, 11, 12

    2026

  47. [47]

    IEEE Transactions on Medical Imaging41(10), 2803–2813 (2022) 3

    Wang, Z., Han, H., Wang, L., Li, X., Zhou, L.: Automated radiographic report generation purely on transformer: A multicriteria supervised approach. IEEE Transactions on Medical Imaging41(10), 2803–2813 (2022) 3

    2022

  48. [48]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Wang, Z., Liu, L., Wang, L., Zhou, L.: Metransformer: Radiology report genera- tion by transformer with multiple learnable expert tokens. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 11558–11567 (2023) 9, 11, 12

    2023

  49. [49]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention

    Wang, Z., Tang, M., Wang, L., Li, X., Zhou, L.: A medical semantic-assisted trans- former for radiographic report generation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 655–664. Springer (2022) 3, 4

    2022

  50. [50]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wang, Z., Zhou, L., Wang, L., Li, X.: A self-boosting framework for automated radiographic report generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 2433–2442 (2021) 4

    2021

  51. [51]

    Advances in Neural Information Processing Systems pp

    Wu, J., Xie, Y., Yang, Z., Wu, J., Gao, J., Ding, B., Wang, X., He, X.:β-dpo: Direct preference optimization with dynamicβ. Advances in Neural Information Processing Systems pp. 129944–129966 (2024) 2, 4

    2024

  52. [52]

    In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision

    Xiao, J., Bai, Y., Yuille, A., Zhou, Z.: Delving into masked autoencoders for multi-label thorax disease classification. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 3588–3600 (2023) 10

    2023

  53. [53]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Xu, G., Jin, P., Wu, Z., Li, H., Song, Y., Sun, L., Yuan, L.: Llava-cot: Let vision lan- guage models reason step-by-step. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 2087–2098 (2025) 2

    2087

  54. [54]

    In: Proceedings of the AAAI conference on artificial intelligence

    Yan, B., Pei, M.: Clinical-bert: Vision-language pre-training for radiograph diag- nosis and reports generation. In: Proceedings of the AAAI conference on artificial intelligence. vol. 36, pp. 2982–2990 (2022) 11, 12

    2022

  55. [55]

    arXiv preprint arXiv:2504.14945 (2025) 12

    Yan, J., Li, Y., Hu, Z., Wang, Z., Cui, G., Qu, X., Cheng, Y., Zhang, Y.: Learning to reason under off-policy guidance. arXiv preprint arXiv:2504.14945 (2025) 12

    Pith/arXiv arXiv 2025

  56. [56]

    Medical Image Analysis 86, 102798 (2023) 3

    Yang,S.,Wu,X.,Ge,S.,Zheng,Z.,Zhou,S.K.,Xiao,L.:Radiologyreportgeneration with a learned knowledge base and multi-modal alignment. Medical Image Analysis 86, 102798 (2023) 3

    2023

  57. [57]

    Medical image analysis80, 102510 (2022) 4

    Yang, S., Wu, X., Ge, S., Zhou, S.K., Xiao, L.: Knowledge matters: Chest radiology report generation with general and specific knowledge. Medical image analysis80, 102510 (2022) 4

    2022

  58. [58]

    IEEE Transactions on Medical Imaging (2025) 2, 4, 10, 11, 12 REVA-PO for CXR Report Generation 19

    Yang, Y., You, X., Zhang, K., Fu, Z., Wang, X., Ding, J., Sun, J., Yu, Z., Huang, Q., Han, W., et al.: Spatio-temporal and retrieval-augmented modelling for chest x-ray report generation. IEEE Transactions on Medical Imaging (2025) 2, 4, 10, 11, 12 REVA-PO for CXR Report Generation 19

    2025

  59. [59]

    arXiv preprint arXiv:2503.14476 (2025) 7, 12

    Yu, Q., Zhang, Z., Zhu, R., Yuan, Y., Zuo, X., Yue, Y., Dai, W., Fan, T., Liu, G., Liu, L., et al.: Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476 (2025) 7, 12

    Pith/arXiv arXiv 2025

  60. [60]

    arXiv preprint arXiv:2501.12368 (2025) 4

    Zang, Y., Dong, X., Zhang, P., Cao, Y., Liu, Z., Ding, S., Wu, S., Ma, Y., Duan, H., Zhang, W., et al.: Internlm-xcomposer2.5-reward: A simple yet effective multi-modal reward model. arXiv preprint arXiv:2501.12368 (2025) 4

    Pith/arXiv arXiv 2025

  61. [61]

    arXiv preprint arXiv:2503.18013 (2025) 4

    Zhan, Y., Zhu, Y., Zheng, S., Zhao, H., Yang, F., Tang, M., Wang, J.: Vision-r1: Evolving human-free alignment in large vision-language models via vision-guided reinforcement learning. arXiv preprint arXiv:2503.18013 (2025) 4

    Pith/arXiv arXiv 2025

  62. [62]

    In: Forty-second International Conference on Machine Learning (2025) 4

    Zhang, Y., Yu, T., Tian, H., Fu, C., Li, P., Zeng, J., Xie, W., Shi, Y., Zhang, H., Wu, J., Wang, X., Hu, Y., Wen, B., Gao, T., Zhang, Z., Yang, F., ZHANG, D., Wang, L., Jin, R.: MM-RLHF: The next step forward in multimodal LLM alignment. In: Forty-second International Conference on Machine Learning (2025) 4

    2025

  63. [63]

    In: Proceedings of the AAAI conference on artificial intelligence

    Zhang, Y., Wang, X., Xu, Z., Yu, Q., Yuille, A., Xu, D.: When radiology report generation meets knowledge graph. In: Proceedings of the AAAI conference on artificial intelligence. vol. 34, pp. 12910–12917 (2020) 4

    2020

  64. [64]

    In: Findings of the Association for Computational Linguistics: EMNLP 2023

    Zhao, G., Yan, Y., Zhao, Z.: Normal-abnormal decoupling memory for medical report generation. In: Findings of the Association for Computational Linguistics: EMNLP 2023. pp. 1962–1977 (2023) 11, 12

    2023

  65. [65]

    In: International Conference on Learning Representations

    Zheng, R., Shen, W., Hua, Y., Lai, W., Dou, S., Zhou, Y., Xi, Z., Wang, X., Huang, H., Gui, T., Zhang, Q., Huang, X.: Improving generalization of alignment with human preferences through group invariant learning. In: International Conference on Learning Representations. pp. 44714–44736 (2024) 2, 4

    2024

  66. [66]

    arXiv preprint arXiv:2304.10592 (2023) 5

    Zhu, D., Chen, J., Shen, X., Li, X., Elhoseiny, M.: Minigpt-4: Enhancing vision- language understanding with advanced large language models. arXiv preprint arXiv:2304.10592 (2023) 5

    Pith/arXiv arXiv 2023

  67. [67]

    arXiv preprint arXiv:1909.08593 (2019) 2, 4

    Ziegler, D.M., Stiennon, N., Wu, J., Brown, T.B., Radford, A., Amodei, D., Chris- tiano, P., Irving, G.: Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593 (2019) 2, 4

    Pith/arXiv arXiv 1909