Pith. sign in

REVIEW 4 major objections 5 minor 51 references

A three-stage training recipe teaches a general vision-language model to reason through chest X-rays step by step before writing a diagnostic report.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 23:38 UTC pith:6HPLA5BL

load-bearing objection A plausible three-stage training recipe for medical VLMs, but the report-generation results are largely an artifact of optimizing the evaluation metric, and the classification/grounding comparisons look unfair. the 4 major comments →

arxiv 2509.06409 v1 pith:6HPLA5BL submitted 2025-09-08 cs.AI

Teaching AI Stepwise Diagnostic Reasoning with Report-Guided Chain-of-Thought Learning

Yihong Luo , Wenwu He , Zhuo-Xu Cui , Dong Liang This is my paper
classification cs.AI
keywords DiagCoTchest X-ray report generationchain-of-thoughtvision-language modelreinforcement fine-tuningGRPOdisease classificationpathology grounding
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes DiagCoT, a three-stage training recipe that claims to convert a general-purpose vision-language model into a stepwise chest X-ray diagnostician using only free-text radiology reports. The first stage aligns images with reports; the second stage fine-tunes the model on radiologist-style chains of thought distilled from a larger teacher model and filtered by a larger judge model; the third stage applies GRPO reinforcement learning with format and lexical-similarity rewards. The authors report that this raises zero-shot classification AUC from 0.52 to 0.76, pathology-grounding mIoU from 0.08 to 0.31, and report-generation BLEU from 0.11 to 0.33, beating specialized baselines on long-tailed diseases and on the external IU-Xray set. The broader claim is that unstructured clinical narratives are enough to teach a model not just what to say but how to look and reason, so interpretable, report-guided training could be a scalable path to clinically usable radiology AI.

Core claim

DiagCoT is the claim that stepwise diagnostic reasoning can be instilled into a general VLM without any manual reasoning annotations. Starting from Qwen2-VL-7B, the paper trains in three stages: Stage 1 aligns chest X-rays with MIMIC-CXR reports by training the projector; Stage 2 performs supervised fine-tuning on a curated CoT dataset, where a Qwen2.5-VL-32B teacher generates think chains using backtracking, alternative-path, verification, and correction prompts, and a Qwen2.5-VL-72B judge keeps only chains consistent with the reference report; Stage 3 applies GRPO fine-tuning with a reward that combines output format with a weighted BLEU1-4/ROUGE-L/METEOR/CIDEr accuracy score. The authors

What carries the argument

The load-bearing object is the CoT-formatted training instance: an X-ray image paired with a <think> chain, modeled on a radiologist's six-perspective scan (image quality, mediastinum, lungs, cardiomediastinum, pleura, bones), and an <answer> report. The chain is produced by a teacher VLM under four correction strategies, filtered by a judge VLM, and then generated and reinforced by the student. The think-chain carries the argument: it injects an explicit intermediate reasoning step between image and report, which the paper claims ordinary token-by-token supervised fine-tuning lacks.

Load-bearing premise

The load-bearing premise is that a weighted blend of BLEU1-4, ROUGE-L, METEOR, and CIDEr measures genuine diagnostic report quality; if these word-overlap metrics do not track clinical correctness, the reinforcement stage is being rewarded with the same yardstick used to grade it.

What would settle it

Have radiologists blindly rate Stage-3 versus Stage-2 reports for diagnostic accuracy, or extract the 14 CheXpert labels from DiagCoT's generated reports and compare that label-AUC with the model's direct classification AUC. If the independent clinical score does not improve alongside BLEU, the report-generation gains reflect metric overfitting rather than better diagnosis.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Free-text radiology reports, without hand-written reasoning annotations, can serve as supervision for chain-of-thought reasoning in vision-language models.
  • The three-stage recipe is modality-agnostic and may transfer to other imaging-report domains such as CT, MRI, or ultrasound if paired reports exist.
  • GRPO-style reinforcement learning with rule-based rewards can be applied to open-ended medical text generation, not only to math and coding.
  • Explicit reasoning supervision improves performance on long-tailed diseases and on spatial localization, suggesting the chain helps align clinical semantics with visual features.
  • Ablations indicate that the full SFT-to-CoT-to-RFT sequence is required: skipping either supervised stage degrades results, and RL-only training fails.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the Stage-3 reward and the evaluation metrics are the same lexical overlap measures, part of the reported report-generation gain may reflect optimization toward the metric rather than improved clinical content; an independent clinical scoring test would settle this.
  • The think chain may be a byproduct of training rather than the causal driver of diagnosis; removing or randomizing the think tokens at inference and re-measuring report quality and grounding would test whether the reasoning is actually used.
  • The CoT dataset is filtered by a large VLM judge, not by radiologists, so the notion of a 'correct' reasoning chain inherits the judge's blind spots; expert review of kept versus discarded chains could reveal systematic filtering bias.
  • Because only one reference report is treated as ground truth, the reward may suppress clinically valid alternative phrasings; training against multiple reference reports or a reference-ensemble reward could give a fairer picture of diagnostic language quality.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This manuscript presents DiagCoT, a three-stage pipeline for chest X-ray report generation: SFT alignment on MIMIC-CXR, CoT construction using a teacher VLM with backtracking/verification strategies and filter by an optimizer VLM, and GRPO reinforcement tuning with format and n-gram-similarity rewards. The authors report large improvements over many baselines on report generation (BLEU etc.), disease classification AUC on CheXpert, and pneumonia grounding mIoU on RSNA, and claim that these gains reflect improved diagnostic reasoning. The central evidence is compromised because the Stage 3 reward is the same set of n-gram metrics used for evaluation; as a result the report-generation gains do not independently establish better diagnostic reports. Additional inconsistencies in the classification results and unclear training protocols for baselines further undermine the cross-task claims.

Significance. The idea of using free-text reports to supervise CoT reasoning is timely, and the ablation study suggests the three stages interact non-trivially. The authors also compare against a broad set of traditional and VLM baselines and provide qualitative examples. However, the main report-generation result is circular: Eq. (6) trains on a weighted sum of BLEU/ROUGE/METEOR/CIDEr, and the evaluation uses exactly these metrics. Without an out-of-reward clinical metric (e.g., CheXbert, RadGraph, or expert rating), the paper does not support its claim of 'clinical reward signals' or improved diagnostic reasoning. The classification and grounding claims are also not properly controlled. If corrected with independent metrics and matched training protocols, the framework could be a useful contribution; as written, the significance of the headline results is not established.

major comments (4)
  1. [§5.4, Eq. (6); §2.1; Supp. Table 7] Stage 3's precision reward R_acc is defined as a weighted combination of BLEU1-4, ROUGE-L, METEOR, and CIDEr, and Section 2.1 lists exactly these metrics for report-generation evaluation. The abstract calls these 'clinical reward signals', but no clinical/factual reward is used. The GRPO update is therefore optimizing the evaluation scalar, so the reported BLEU/ROUGE/METEOR gains are expected from direct reward maximization and do not independently verify diagnostic report quality. The ablation in Supp. Table 7 confirms this: CIDEr falls from 0.1553 (DiagCoT-RL(W.2)) to 0.1152 (DiagCoT) while BLEU-4 and METEOR rise, and the authors attribute the trade-off to reward design. This is a circularity, not a validation.
  2. [§2.2.2, Fig. 2b, Supp. Table 5; Abstract] The abstract reports zero-shot disease classification AUC improving from 0.52 to 0.76, but Section 2.2.2 describes fine-tuning the two-stage model on CheXpert, and Supp. Table 5 reports mean AUC 62.8% (0.628). The only near-0.76 value is No Finding 77.3%, which is not an average. This discrepancy means the headline classification result as stated is not supported by the tables. Additionally, no error bars or significance tests are provided, despite §2.2.1 claiming 'statistically significant improvements'.
  3. [§2.2.3, Fig. 2c, Supp. Table 6] For RSNA grounding, DiagCoT is explicitly fine-tuned on the augmented RSNA training set, while the comparison models appear to be evaluated without the same fine-tuning; the paper does not state whether baselines received equivalent training. The reported 0.7307 accuracy vs 0.3604 for Qwen2.5-VL-72B-AWQ is therefore not a controlled comparison. Without a matched training protocol, or at least a baseline fine-tuned under identical conditions, the claim of 103%/105% improvement is not supported.
  4. [§2.2, all comparisons] None of the main comparisons include confidence intervals, error bars, or statistical significance tests. Given that several VLM baselines have close scores and the paper explicitly claims statistical significance, this is a load-bearing omission for the 'outperformed' narrative. At minimum, the matched-protocol comparisons need uncertainty quantification before the central claims can be assessed.
minor comments (5)
  1. [Figure 3] Typo: 'Reoprt Generation' should be 'Report Generation'.
  2. [Supplementary Figures 1-5] Typo: 'Ptompt' should be 'Prompt'.
  3. [Section 6.2] The text says Stage 3 rewards are 'solely based on BLEU-1 to BLEU-4 and ROUGE-L', but Eq. (6) includes METEOR and CIDEr as well. This is inconsistent.
  4. [Supplementary Table 3] METEOR values are missing for Qwen2.5-VL-32B-Instruct and Qwen2-VL-7B-Base, and '0.07440' appears to have a formatting error.
  5. [Section 8] The code availability statement says code 'will be made publicly available upon publication', so no reproducibility artifact is currently available to verify the training pipeline.

Circularity Check

1 steps flagged

Stage-3 reward and the report-generation evaluation are the same BLEU/ROUGE/METEOR/CIDEr objective, so the RRG headline is trained-for, not independently validated.

specific steps
  1. fitted input called prediction [Section 5.4, Eq. (6); Section 2.1 Metrics; Section 6.2]
    "Precision Reward Racc: Combines weighted scores of BLEU1-4, ROUGE-L, METEOR, and CIDEr to assess similarity between generated and reference reports. ... This study introduces GRPO for third-stage optimization, automatically generating reward signals solely based on BLEU-1 to BLEU-4 and ROUGE-L NLG metrics."

    The RRG evaluation (§2.1) uses exactly BLEU{1-4}, ROUGE-L, Meteor, and CIDEr. Stage 3's precision reward R_acc is defined as a weighted combination of those same metrics, and GRPO maximizes it via Eq. (5)-(6). So the reported RRG improvements (BLEU 0.11→0.33, etc.) are the direct result of optimizing the evaluation objective, not an independent test of diagnostic quality. The abstract's 'clinical reward signals' is contradicted by §6.2: rewards are 'solely based on BLEU-1 to BLEU-4 and ROUGE-L NLG metrics.' The ablation (CIDEr 0.1553 in W.2 vs 0.1152 in DiagCoT) confirms metric scores are shaped by reward weights. Classification AUC and grounding mIoU are not optimized by this reward, so they remain external.

full rationale

The main circularity is localized to the report-generation claim. Stage 3 optimizes a reward built from the same n-gram/lexical metrics used for evaluation, so the RRG table is partly a check of reward optimization rather than clinical quality. This does not make classification (CheXpert AUC) or grounding (RSNA mIoU) circular, since those tasks use external labels/boxes not in the reward. The paper's own Limitations section admits the RL stage 'currently relies on rule-based reward functions' and calls for 'learnable, domain-specific reward models,' implicitly conceding that the current reward is not a clinical/factual validator. No self-citation chain or imported uniqueness theorem is load-bearing; baselines and datasets are external. Overall score 6: the central RRG result reduces by construction to the training objective, but the multi-task framework retains independent content.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central results rest on several domain assumptions about label quality, model-generated reasoning validity, and the clinical meaning of n-gram metrics. No new physical or formal entities are introduced. The main free parameters are the undisclosed reward weights and training hyperparameters.

free parameters (5)
  • Report reward weights
    Weights in Racc (Equation 6) for BLEU1-4, ROUGE-L, METEOR, CIDEr are not disclosed, but they determine how much the model optimizes each metric during Stage 3.
  • Maximum CoT attempts T = 3
    Set by hand to limit data collection time in Section 5.3 and Algorithm 2.
  • GRPO group size G = 8
    Chosen for advantage estimation in Section 5.4 and Supplementary Table 2.
  • KL divergence coefficient beta = 0.05
    Set in Section 5.4 and Supplementary Table 2 to constrain policy updates.
  • Stage-specific training hyperparameters = LR 2e-6/1e-5/1e-6, batch 64/16/1, steps 3485/1136/435
    Reported in Supplementary Tables 1-2; chosen by hand and not derived from first principles.
axioms (5)
  • domain assumption MIMIC-CXR reports, CheXpert labels, and RSNA bounding boxes are accurate ground truth for chest X-ray diagnosis.
    The method is trained and evaluated against these public annotations without clinical validation; errors in these labels would propagate to the reported results.
  • domain assumption CoT traces generated by Qwen2.5-VL-32B and filtered by Qwen2.5-VL-72B-AWQ are correct and consistent with reference reports.
    The Stage 2 dataset is entirely produced and filtered by the same model family; no human verification is described.
  • domain assumption BLEU, ROUGE-L, METEOR, and CIDEr are acceptable proxies for clinical report quality.
    Used both as the RFT reward (Equation 6) and as the main evaluation metrics; no clinical or human evaluation is provided.
  • ad hoc to paper The six-perspective diagnostic framework (Figure 1a) captures radiologist reasoning sufficiently for training CoT.
    This structured decomposition is introduced by the paper as the target reasoning pattern but is not independently validated.
  • domain assumption Reference reports used in the CoT generation prompts are available at training time and the model learns to reason without them at inference.
    Prompts in Supplementary Figures 1-6 include the reference report during CoT creation, creating a train/inference mismatch.

pith-pipeline@v1.3.0-alltime-deepseek · 24869 in / 17849 out tokens · 171502 ms · 2026-08-04T23:38:26.574295+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Teaching AI Stepwise Diagnostic Reasoning with Report-Guided Chain-of-Thought Learning." pith.science (2026). https://pith.science/paper/6HPLA5BL

@misc{pith2026250906409,
  author       = {Pith},
  title        = {Pith review of: Teaching AI Stepwise Diagnostic Reasoning with Report-Guided Chain-of-Thought Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6HPLA5BL}},
  note         = {Machine review of arXiv:2509.06409}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This study presents DiagCoT, a multi-stage framework that applies supervised fine-tuning to general-purpose vision-language models (VLMs) to emulate radiologists' stepwise diagnostic reasoning using only free-text reports. DiagCoT combines contrastive image-report tuning for domain alignment, chain-of-thought supervision to capture inferential logic, and reinforcement tuning with clinical reward signals to enhance factual accuracy and fluency. On the MIMIC-CXR benchmark, DiagCoT improved zero-shot disease classification AUC from 0.52 to 0.76 (absolute gain of 0.24), pathology grounding mIoU from 0.08 to 0.31 (absolute gain of 0.23), and report generation BLEU from 0.11 to 0.33 (absolute gain of 0.22). It outperformed state-of-the-art models including LLaVA-Med and CXR-LLAVA on long-tailed diseases and external datasets. By converting unstructured clinical narratives into structured supervision, DiagCoT offers a scalable approach for developing interpretable and diagnostically competent AI systems for radiology.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 25 canonical work pages · 1 internal anchor

  1. [1]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  2. [2]

    Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond, 2023

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond, 2023

    2023

  3. [3]

    Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model.arXiv preprint arXiv:2405.04434, 2024

    Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model.arXiv preprint arXiv:2405.04434, 2024

    Pith/arXiv arXiv 2024

  4. [4]

    Cogvlm: Visual expert for pretrained language models.Advances in Neural Information Processing Systems, 37:121475–121499, 2024

    Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Song XiXuan, et al. Cogvlm: Visual expert for pretrained language models.Advances in Neural Information Processing Systems, 37:121475–121499, 2024

    work page 2024

  5. [5]

    Deep learning in generating radiology reports: A survey.Artificial Intelligence in Medicine, 106:101878, 2020

    Maram Mahmoud A Monshi, Josiah Poon, and Vera Chung. Deep learning in generating radiology reports: A survey.Artificial Intelligence in Medicine, 106:101878, 2020

    work page 2020

  6. [6]

    Deep visual-semantic alignments for generating image descriptions

    Andrej Karpathy and Li Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In Computer Vision & Pattern Recognition, 2015

    work page 2015

  7. [7]

    Long-term recurrent convolutional networks for visual recognition and description.Elsevier,, 2015

    Jeff Donahue, Lisa A Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venugopalan, Kate Saenko, and Trevor Darrell. Long-term recurrent convolutional networks for visual recognition and description.Elsevier,, 2015

    work page 2015

  8. [8]

    Show, attend and tell: Neural image caption generation with visual attention

    Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. Computer Science, pages 2048–2057, 2015

    work page 2048

  9. [9]

    Bottom-up and top-down attention for image captioning and visual question answering

    Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, and Lei Zhang. Bottom-up and top-down attention for image captioning and visual question answering. In2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018

    work page 2018

  10. [10]

    Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  11. [11]

    Mimic-cxr-jpg, a large publicly available database of labeled chest radiographs.arXiv preprint arXiv:1901.07042, 2019

    Alistair EW Johnson, Tom J Pollard, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Yifan Peng, Zhiyong Lu, Roger G Mark, Seth J Berkowitz, and Steven Horng. Mimic-cxr-jpg, a large publicly available database of labeled chest radiographs.arXiv preprint arXiv:1901.07042, 2019

    Pith/arXiv arXiv 1901

  12. [12]

    Preparing a collection of radiology examinations for distribution and retrieval.Journal of the American Medical Informatics Association, 23(2):304–310, 2015

    Dina Demner-Fushman, Marc D Kohli, Marc B Rosenman, Sonya E Shooshan, Laritza Rodriguez, Sameer Antani, George R Thoma, and Clement J McDonald. Preparing a collection of radiology examinations for distribution and retrieval.Journal of the American Medical Informatics Association, 23(2):304–310, 2015

    work page 2015

  13. [13]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. InProceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002

    2002

  14. [14]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. InText summarization branches out, pages 74–81, 2004

    2004

  15. [15]

    Meteor 1.3: Automatic metric for reliable optimization and evaluation of machine translation systems

    Michael Denkowski and Alon Lavie. Meteor 1.3: Automatic metric for reliable optimization and evaluation of machine translation systems. InProceedings of the sixth workshop on statistical machine translation, pages 85–91, 2011. |16

    work page 2011

  16. [16]

    Cider: Consensus-based image description evaluation

    Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 4566–4575, 2015

    2015

  17. [17]

    Llava-med: Training a large language-and-vision assistant for biomedicine in one day.Advances in Neural Information Processing Systems, 36:28541–28564, 2023

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Nau- mann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language-and-vision assistant for biomedicine in one day.Advances in Neural Information Processing Systems, 36:28541–28564, 2023

    2023

  18. [18]

    Cxr-llava: a multimodal large language model for interpreting chest x-ray images.European Radiology, pages 1–13, 2025

    Seowoo Lee, Jiwon Youn, Hyungjin Kim, Mansu Kim, and Soon Ho Yoon. Cxr-llava: a multimodal large language model for interpreting chest x-ray images.European Radiology, pages 1–13, 2025

    work page 2025

  19. [19]

    Chestx-reasoner: Advancing radiology foundation models with reasoning through step-by-step verification.arXiv preprint arXiv:2504.20930, 2025

    Ziqing Fan, Cheng Liang, Chaoyi Wu, Ya Zhang, Yanfeng Wang, and Weidi Xie. Chestx-reasoner: Advancing radiology foundation models with reasoning through step-by-step verification.arXiv preprint arXiv:2504.20930, 2025

    Pith/arXiv arXiv 2025

  20. [20]

    Chexagent: Towards a foundation model for chest x-ray interpretation.arXiv preprint arXiv:2401.12208, 2024

    Zhihong Chen, Maya Varma, Jean-Benoit Delbrouck, Magdalini Paschali, Louis Blankemeier, Dave Van Veen, Jeya Maria Jose Valanarasu, Alaa Youssef, Joseph Paul Cohen, Eduardo Pontes Reis, et al. Chexagent: Towards a foundation model for chest x-ray interpretation.arXiv preprint arXiv:2401.12208, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Deepseek-vl: towards real-world vision-language understanding.arXiv preprint arXiv:2403.05525, 2024

    Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, et al. Deepseek-vl: towards real-world vision-language understanding.arXiv preprint arXiv:2403.05525, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Generating radiology reports via memory- driven transformer

    Zhihong Chen, Yan Song, Tsung-Hui Chang, and Xiang Wan. Generating radiology reports via memory- driven transformer. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, November 2020

    work page 2020

  23. [23]

    Generating radiology reports via memory-driven transformer

    Zhihong Chen, Yaling Shen, Yan Song, and Xiang Wan. Generating radiology reports via memory-driven transformer. InProceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, August 2021

    work page 2021

  24. [24]

    Cross-modal prototype driven network for radiology report generation

    Jun Wang, Abhir Bhalerao, and Yulan He. Cross-modal prototype driven network for radiology report generation. InComputer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXV, pages 563–579. Springer, 2022

    work page 2022

  25. [25]

    Radiology report generation with a learned knowledge base and multi-modal alignment.Medical Image Analysis, 86:102798, 2023

    Shuxin Yang, Xian Wu, Shen Ge, Zhuozhao Zheng, S Kevin Zhou, and Li Xiao. Radiology report generation with a learned knowledge base and multi-modal alignment.Medical Image Analysis, 86:102798, 2023

    work page 2023

  26. [26]

    Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison

    Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. InProceedings of the AAAI conference on artificial intelligence, volume 33, pages 590–597, 2019

    2019

  27. [27]

    Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases

    Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M Summers. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2097–2106, 2017

    2097

  28. [28]

    Augmenting the national institutes of health chest radiograph dataset with expert annotations of possible pneumonia.Radiology: Artificial Intelligence, 1(1):e180041, 2019

    George Shih, Carol C Wu, Safwan S Halabi, Marc D Kohli, Luciano M Prevedello, Tessa S Cook, Arjun Sharma, Judith K Amorosa, Veronica Arteaga, Maya Galperin-Aizenberg, et al. Augmenting the national institutes of health chest radiograph dataset with expert annotations of possible pneumonia.Radiology: Artificial Intelligence, 1(1):e180041, 2019

    work page 2019

  29. [29]

    MoColl: Agent-Based Specific and General Model Collaboration for Image Captioning

    Pu Yang and Bin Dong. Mocoll: Agent-based specific and general model collaboration for image captioning.arXiv preprint arXiv:2501.01834, 2025. |17

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  30. [30]

    Huatuogpt-o1, towards medical complex reasoning with llms, 2024

    Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. Huatuogpt-o1, towards medical complex reasoning with llms, 2024

    work page 2024

  31. [31]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021

    2021

  32. [32]

    Deep reinforcement learning from human preferences.arXiv preprint arXiv:1706.03741, 2017

    Paul F Christiano, Jan Leike, Tom B Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences.arXiv preprint arXiv:1706.03741, 2017

    Pith/arXiv arXiv 2017

  33. [33]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  34. [34]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024

    2024

  35. [35]

    Med-r1: Reinforcement learning for generalizable medical reasoning in vision-language models.arXiv preprint arXiv:2503.13939, 2025

    Yuxiang Lai, Jike Zhong, Ming Li, Shitian Zhao, and Xiaofeng Yang. Med-r1: Reinforcement learning for generalizable medical reasoning in vision-language models.arXiv preprint arXiv:2503.13939, 2025

    arXiv 2025

  36. [36]

    Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models.arXiv preprint arXiv:2503.06749, 2025

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models.arXiv preprint arXiv:2503.06749, 2025

    Pith/arXiv arXiv 2025

  38. [38]

    Decoupled weight decay regularization, 2019

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019

    2019

  39. [39]

    Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. InProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’20, page 3505–3506, New York, NY, USA, 2020. Association for Computing Machinery

    work page 2020

  40. [40]

    Pytorch fsdp: Experiences on scaling fully sharded data parallel, 2023

    Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, Alban Desmaison, Can Balioglu, Pritam Damania, Bernard Nguyen, Geeta Chauhan, Yuchen Hao, Ajit Mathews, and Shen Li. Pytorch fsdp: Experiences on scaling fully sharded data parallel, 2023

    work page 2023

  41. [41]

    Easyr1: An efficient, scalable, multi-modality rl training framework.https://github.com/hiyouga/EasyR1, 2025

    Yaowei Zheng, Junting Lu, Shenzhi Wang, Zhangchi Feng, Dongdong Kuang, and Yuwen Xiong. Easyr1: An efficient, scalable, multi-modality rl training framework.https://github.com/hiyouga/EasyR1, 2025

    2025

  42. [42]

    Inner Thinking

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. InProceedings of the Twentieth European Conference on Computer Systems, EuroSys ’25, page 1279–1297, New York, NY, USA, 2025. Association for Computing Machinery. |18 12 Supplementary 12.1 ...

    work page 2025

  43. [45]

    Final Conclusion

    **Verification**: Verify the accuracy of the "Final Conclusion". If it holds, conclude the process. Otherwise, return to "Inner Thinking" for further refinement. </response requirements> <question> represents the task to generate a medical report based on the X-ray image, and <previous reasoning> contains your prior reasoning. Your task is to continue fro...

    work page

  44. [48]

    Final Conclusion

    **Verification**: Verify the accuracy of the "Final Conclusion". If it holds, conclude the process. Otherwise, return to "Inner Thinking" for further refinement. </response requirements> <question> represents the task to generate a medical report based on the X-ray image, and <previous reasoning> contains your prior reasoning. Your task is to continue fro...

    work page

  45. [51]

    Final Conclusion

    **Verification**: Verify the accuracy of the "Final Conclusion". If it holds, conclude the process. Otherwise, return to "Inner Thinking" for further refinement. </response requirements> <question> represents the task to generate a medical report based on the X-ray image, and <previous reasoning> contains your prior reasoning. Your task is to continue fro...

    work page

  46. [52]

    Each step should start with a brief title to clarify its purpose

    **Inner Thinking**: Break down the reasoning process into multiple concise steps. Each step should start with a brief title to clarify its purpose

  47. [53]

    No title is needed for this section

    **Final Conclusion**: Summarize the correct reasoning from all previous 'Inner Thinking' steps and provide the final X-ray diagnosis report. No title is needed for this section

    work page

  48. [54]

    Final Conclusion

    **Verification**: Verify the accuracy of the "Final Conclusion". If it holds, conclude the process. Otherwise, return to "Inner Thinking" for further refinement. </response requirements> <question> represents the task to generate a medical report based on the X-ray image, and <previous reasoning> contains your prior reasoning. Your task is to continue fro...

    work page

  49. [55]

    Be presented as step-by-step reasoning, with each thought on a new line separated by a line break

  50. [56]

    hmm," "oh,

    Avoid structured titles or formatting, focusing on natural transitions. Use casual and natural language for transitions or validations, such as "hmm," "oh," "also," or "wait."

  51. [57]

    NaturalReasoning

    Expand the content, making the reasoning richer, more detailed, and logically clear while still being conversational and intuitive, as if a doctor is explaining their thought process. Return directly the revised natural thinking in JSON format as follows: ```json {{ "NaturalReasoning": "..." }} ``` Supplementary Figure 6| Ptompt 6.This figure presents the...

    work page arXiv 2096