Pith. sign in

REVIEW 4 major objections 5 minor 53 references

Bridging Vision and Language: Optimal Transport-Driven Radiology Report Generation via LLMs

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that aligning chest X-ray image features with disease labels through optimal transport, then fine-tuning an LLM with LoRA, produces radiology reports that are both more fluent and more clinically accurate than current…

desk verdict Plausible incremental OT-based alignment for radiology report generation, but the unvalidated disease-label predictor and overstated SOTA claims make the main result only conditionally supported. read the letter →

arxiv 2507.03908 v1 pith:CXIR25XZ submitted 2025-07-05 cs.CV

Haifeng Zhao , Yufei Zhang , Leilei Ma , Shuo Xu , Dengdi Sun This is my paper
classification cs.CV
keywords radiologyreportgenerationoptimaltransportlargelanguagemodelscross-modalalignmentclinicalefficacydiseaselabelpredictionSinkhorn-KnoppMIMIC-CXR
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Radiology report generation from chest X-rays faces two problems: LLMs write fluent prose that often misses or invents findings, and raw image features do not sit naturally in the text space where LLMs work. This paper proposes OTDRG, which extracts disease labels from reports, projects image patches and labels into the LLM embedding space, and uses optimal transport to find a global alignment between the two. The aligned features are combined into a prompt and the LLM is fine-tuned with LoRA to generate the report. The authors claim this improves both language quality and clinical accuracy over prior methods on MIMIC-CXR and IU X-Ray, with ablations showing the transport alignment module is the main driver of the clinical gains.

What carries the argument

The load-bearing component is FIOT (Feature Integration by Optimal Transport). It builds a Euclidean cost matrix between projected image patch features and projected disease-label features, solves for an optimal transport plan using the Sinkhorn-Knopp algorithm with an entropy term, and returns the Sinkhorn distance as an auxiliary loss. The aligned features are concatenated into a prompt of the form `</IMG> $f_p(I) \oplus f_p(L)$ </IMG>`, which is what the LLM is fine-tuned on via LoRA; the transport plan itself also supplies the heatmaps the paper uses to show that the model attends to disease-relevant image regions.

What would settle it

Take the MIMIC-CXR test set, run the disease prediction module on each image, and compare its predicted labels against CheXbert-extracted labels from the ground-truth reports. Then generate reports twice, once with predicted labels and once with ground-truth labels; if clinical efficacy scores drop substantially in the predicted-label condition, the test-time pipeline does not deliver the reported clinical accuracy.

Watch

Extended reading notes

Core claim

OTDRG's central claim is that a cross-modal optimal transport plan between X-ray image patch features and disease-label features, computed in the LLM's embedding space, bridges the distribution gap that otherwise makes LLMs prioritize fluency over diagnostic content. The transport plan is obtained with the Sinkhorn-Knopp algorithm with entropic regularization, and the aligned features are fed into the LLM as an image-text integrated prompt along with the disease labels. At test time, a separately trained classifier predicts the disease labels from the image, replacing the ground-truth labels used during training. The authors report best-in-comparison NLG scores on both datasets and, on MIMIC-CXR, clinical efficacy precision/recall/F1 second only to PromptMRG, with the FIOT module credited for the largest part of the clinical improvement.

Load-bearing premise

At test time the generated reports depend on disease labels predicted from the X-ray by a small classifier, but the paper reports no accuracy for that classifier; if those predicted labels are unreliable, the claimed clinical gains may shrink or disappear.

Editorial extensions

If this is right

  • LLM-based report generators can be steered toward clinical accuracy by conditioning generation on aligned disease labels rather than on raw image features alone.
  • On MIMIC-CXR, OTDRG reports the highest BLEU-1, BLEU-4, METEOR, and ROUGE-L among the compared methods and a clinical F1 of 0.436, behind only PromptMRG; on IU X-Ray it reports top BLEU-1 and METEOR.
  • The ablation sequence shows clinical F1 rising from 0.263 in the baseline to 0.436 for the full model, with the FIOT module adding the largest precision and recall gain.
  • The transport-plan heatmaps concentrate on disease-related areas of the X-ray, linking the alignment objective to the content of the generated report.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the disease prediction module is never evaluated on its own, the end-to-end results are conditional on it: a direct accuracy, precision, and recall report for predicted labels would determine how much of the clinical gain survives in real deployment.
  • The same label-conditioned optimal transport alignment could be tested on other structured clinical outputs, such as pathology or discharge summaries, where a small set of discrete findings plays the role of disease labels.
  • A cleaner attribution of the FIOT module's contribution would compare it against a control that concatenates the same disease labels without optimal transport; the current ablations do not include such a control.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes OTDRG, a radiology report generation framework that uses Optimal Transport (OT) to align X-ray image features with disease-label features extracted from ground-truth reports, then fine-tunes an LLM with LoRA on the aligned features to generate reports. A separate disease-label prediction module is trained to predict one-hot disease states from images at inference time. The authors evaluate on MIMIC-CXR and IU X-Ray, reporting competitive or state-of-the-art NLG and clinical efficacy (CE) scores, with ablations showing gains from the IRCP contrastive pre-training, disease-label extraction (DLE), and the FIOT alignment module.

Significance. If the results hold, the use of OT for cross-modal alignment in LLM-based report generation is a reasonable and potentially useful idea, and the authors provide a fairly complete pipeline with ablations and qualitative visualizations. However, the claimed state-of-the-art status is not supported by the numbers in Table 1, and the inference-time disease predictor is never evaluated. The central contribution—that FIOT materially improves clinical accuracy—is plausible but not yet convincingly established because the end-to-end evaluation relies on an unvalidated component.

major comments (4)
  1. [Abstract and Section 4.3.1, Table 1] The abstract claims 'state-of-the-art performance in both natural language generation (NLG) and clinical efficacy (CE) metrics,' but Table 1 shows PromptMRG achieving CE F1 0.476 versus OTDRG 0.436, and Section 4.3.1 itself concedes being 'second only to PromptMRG in CE metrics.' Similarly, on IU X-Ray (Table 2), OTDRG does not achieve the best ROUGE-L (R2GenGPT 0.438, MCSAM 0.394, CLR2Gen 0.378 vs. 0.377). The SOTA claim must be revised to reflect these results, or the comparison must be restricted to LLM-based methods and clearly stated as such.
  2. [Section 3.4] The disease-label prediction module is introduced as the method used to provide disease labels during validation and testing, replacing the ground-truth-extracted labels used in training. However, the paper never reports any accuracy, precision, recall, or error analysis for this module. The ablation study in Table 3 uses the DLE module with labels extracted from ground-truth reports, not predicted labels, so the end-to-end results in Tables 1 and 4 confound the FIOT and LLM contributions with the unvalidated predictor. The statement in Section 3.4 that 'accurate extraction of disease status labels is essential (more details can be found in the Supplementary)' is not verifiable because the Supplementary is not provided. The authors should either evaluate the predictor directly or report end-to-end results using predicted labels.
  3. [Table 3, last two rows] The ablation shows that adding FIOT to IRCP+DLE increases CE F1 from 0.349 to 0.436 but decreases ROUGE-L from 0.301 to 0.295. This regression is not discussed in the text. While the CE gain is the central motivation, the unexplained NLG degradation should be addressed, especially since the paper claims improvements in both aspects. A short explanation or additional analysis (e.g., whether the FIOT module trades off language diversity for clinical precision) would strengthen the ablation.
  4. [Tables 1 and 2, experimental setup] All baseline numbers are taken from the original papers, with no error bars, no statistical significance tests, and no indication of whether the same preprocessing, split, or evaluation code was used. Given that the paper's central claim is state-of-the-art performance, at least the most important comparisons (e.g., PromptMRG on MIMIC-CXR) should be re-run under a unified protocol, or the claim should be weakened to 'competitive' rather than 'state-of-the-art.'
minor comments (5)
  1. [Equation (2), Section 3.2.2] The text defines sim(·,·) as a similarity function and gives Euclidean distance as an example. Euclidean distance is a dissimilarity measure; if used directly in the InfoNCE loss it would push positive pairs apart. Please clarify whether the implementation uses negative Euclidean distance or cosine similarity.
  2. [Equation (8) and Algorithm 1] The sign convention in the entropic OT objective is confusing: Ω(T) is defined as Σ T log T, which is the negative entropy, and the objective is written as minimize ⟨T,C⟩ − εΩ(T). This is mathematically equivalent to entropy regularization, but the notation should be aligned with the Sinkhorn-Knopp derivation. In Algorithm 1, the initialization of α and β, the use of softmax on α and β, and the final 'return dOT = minimize <T,C> − h(T*)' are unclear and should be rewritten.
  3. [Section 3.4] The sentence 'more details can be found in the Supplementary' is unverifiable because the arXiv submission does not appear to include a supplementary document. Either include the supplementary material or remove the reference.
  4. [Section 5 (Conclusion)] The conclusion contains a typo: 'MIME-CXR' should be 'MIMIC-CXR.'
  5. [Section 4.3.1] The list of compared methods in the text includes '[46]' and 'Clinical-Bert [45]', but the reference list numbering does not cleanly map to the cited works (e.g., [46] is given as 'Radiology report generation with a learned knowledge base' while the text also cites [47] for the same topic). Please check that all inline citations match the reference list.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the unverified disease-label predictor and overstated SOTA claims are empirical/correctness concerns, not circular reductions.

full rationale

The derivation chain is not circular. The FIOT module minimizes a Sinkhorn OT cost between projected image features and disease-label features extracted from ground-truth reports (Eqs. 6-8), and the LLM is fine-tuned with a standard report-generation cross-entropy loss (Eq. 11); both losses are supervised by external ground-truth annotations, not by the evaluation metrics. The disease-label prediction module (Sec. 3.4) is a separately trained classifier that supplies one-hot labels at test time; although its accuracy is never reported and its contribution is therefore unverified, this is an empirical validation gap rather than a definitional reduction. No load-bearing self-citations, imported uniqueness theorems, or ansatz-by-citation steps appear. The abstract's 'state-of-the-art' phrasing is contradicted by the paper's own Tables 1-2 (PromptMRG has higher CE F1; CLR2Gen/MCSAM have higher BLEU-4/ROUGE-L), but an overstated claim is not circularity. Overall score 0.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The central claim rests on standard math (OT, Sinkhorn, InfoNCE, focal loss, cross-entropy) plus three domain assumptions about the usefulness and consistency of disease labels. No new physical or conceptual entities are postulated. The free parameters are mostly standard hyperparameters, but several are unspecified or tuned without sensitivity analysis, which limits the strength of the empirical claim.

free parameters (6)
  • epsilon (OT entropic regularization) = 0.10
    Tuned on MIMIC-CXR; Table 4 shows BLEU and CE-F1 vary with it; chosen 0.10 for best reported results.
  • lambda (trade-off for L_OT)
    Overall loss L = L_RG + lambda * L_OT in Eq. 12; the value is never reported.
  • tau (InfoNCE temperature) = 0.5
    Set in Eq. 2; no sensitivity analysis is provided.
  • sigma (Gaussian noise std for IRCP positives) = 0.1
    Set in Section 3.2.2; no sensitivity analysis is provided.
  • gamma (focal loss exponent)
    Eq. 10 uses focal loss; the gamma value is not specified.
  • OT iteration count t
    Figure 4 shows performance versus iteration times, but the final t used in experiments is not stated.
assumptions (3)
  • domain assumption Disease labels extracted from ground-truth reports are a faithful summary of the clinical content and are the right intermediate representation for cross-modal alignment.
    Introduced in Sections 3.2.1 and 3.3.2. If labels are noisy or incomplete, the OT alignment learns a wrong mapping and the LLM is conditioned on unreliable information.
  • domain assumption Minimizing Sinkhorn distance between image patch features and disease label features in the LLM embedding space improves downstream report generation.
    Section 3.3.2; no theoretical justification is given, only the ablation in Table 3, which is partially contradictory (ROUGE-L decreases).
  • domain assumption The frozen LLM used for disease label extraction yields labels consistent with the CheXbert labels used for CE evaluation.
    Sections 3.2.1 and 4.2; if the two label sets diverge, the reported CE numbers would not measure what the paper claims.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bridging Vision and Language: Optimal Transport-Driven Radiology Report Generation via LLMs." pith.science (2026). https://pith.science/paper/CXIR25XZ

@misc{pith2026250703908,
  author       = {Pith},
  title        = {Pith review of: Bridging Vision and Language: Optimal Transport-Driven Radiology Report Generation via LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CXIR25XZ}},
  note         = {Machine review of arXiv:2507.03908}
}
read the original abstract

Radiology report generation represents a significant application within medical AI, and has achieved impressive results. Concurrently, large language models (LLMs) have demonstrated remarkable performance across various domains. However, empirical validation indicates that general LLMs tend to focus more on linguistic fluency rather than clinical effectiveness, and lack the ability to effectively capture the relationship between X-ray images and their corresponding texts, thus resulting in poor clinical practicability. To address these challenges, we propose Optimal Transport-Driven Radiology Report Generation (OTDRG), a novel framework that leverages Optimal Transport (OT) to align image features with disease labels extracted from reports, effectively bridging the cross-modal gap. The core component of OTDRG is Alignment \& Fine-Tuning, where OT utilizes results from the encoding of label features and image visual features to minimize cross-modal distances, then integrating image and text features for LLMs fine-tuning. Additionally, we design a novel disease prediction module to predict disease labels contained in X-ray images during validation and testing. Evaluated on the MIMIC-CXR and IU X-Ray datasets, OTDRG achieves state-of-the-art performance in both natural language generation (NLG) and clinical efficacy (CE) metrics, delivering reports that are not only linguistically coherent but also clinically accurate.

Figures

Figures reproduced from arXiv: 2507.03908 by the authors.

Figure 1
Figure 1. The example in the figure briefly illustrates the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overall workflow of OTDRG, which includes a visual encoder, a text encoder, an image projection head, and a label [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Workflow of the disease label prediction. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Ablation Experiments on OT iteration times with [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Visualization of the baseline model and our model. Highlights in different colors correspond to different diagnoses in [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 41 canonical work pages

  1. [1]

    Muhammad Haseeb Aslam, Marco Pedersoli, Alessandro Lameiras Koerich, and Eric Granger. 2024. Multi Teacher Privileged Knowledge Distillation for Multi- modal Expression Recognition. arXiv:2408.09035 [cs.CV] https://arxiv.org/abs/ 2408.09035

    work page Pith review arXiv 2024

  2. [2]

    Jinze Bai, Shuai Bai, Yunfei Chu, and et al. 2023. Qwen Technical Report. arXiv preprint arXiv:2309.16609 (2023)

    arXiv 2023

  3. [3]

    Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization , Jade Goldstein, Alon Lavie, Chin-Yew Lin, and Clare Voss (Eds.). Association for Computationa...

    2005

  4. [4]

    Brown and et al

    Tom B. Brown and et al. 2020. Language models are few-shot learners. In Pro- ceedings of the 34th International Conference on Neural Information Processing Systems (Vancouver, BC, Canada) (NIPS ’20). Curran Associates Inc., Red Hook, NY, USA, Article 159, 25 pages

    work page 2020

  5. [5]

    Wanxing Chang, Ye Shi, Hoang Tuan, and Jingya Wang. 2022. Unified Op- timal Transport Framework for Universal Domain Adaptation. In Advances in Neural Information Processing Systems , S. Koyejo, S. Mohamed, A. Agar- wal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35. Curran Associates, Inc., 29512–29524. https://proceedings.neurips.cc/paper_files/paper/2...

    work page 2022

  6. [6]

    Guangyi Chen, Weiran Yao, Xiangchen Song, Xinyue Li, Yongming Rao, and Kun Zhang. 2023. PLOT: Prompt Learning with Optimal Transport for Vision- Language Models. In The Eleventh International Conference on Learning Represen- tations. https://openreview.net/forum?id=zqwryBoXYnh

    work page 2023

  7. [7]

    Wenting Chen, Linlin Shen, Jingyang Lin, Jiebo Luo, Xiang Li, and Yixuan Yuan

    work page

  8. [8]

    Zhihong Chen, Yaling Shen, Yan Song, and Xiang Wan. 2021. Cross-modal Memory Networks for Radiology Report Generation. In ACL. 5904–5914

    work page 2021

Show all 53 references
  1. [9]

    Zhihong Chen, Yan Song, Tsung-Hui Chang, and Xiang Wan. 2020. Generating Radiology Reports via Memory-driven Transformer. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . 1439– 1449

    2020

  2. [10]

    Dina Demner-Fushman, Marc D Kohli, Marc B Rosenman, Sonya E Shooshan, Laritza Rodriguez, Sameer Antani, George R Thoma, and Clement J McDonald

  3. [11]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human...

    2019

  4. [12]

    Zheng Ge, Songtao Liu, Zeming Li, Osamu Yoshie, and Jian Sun. 2021. OTA: Op- timal Transport Assignment for Object Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 303–312

    2021

  5. [13]

    Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, and Jonathan Herzig. 2024. Does Fine-Tuning LLMs on New Knowledge Encour- age Hallucinations? arXiv:2405.05904 [cs.CL] https://arxiv.org/abs/2405.05904

    2024 arXiv

  6. [14]

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations . https: //openreview.net/forum?id=nZeVKeeFYf9

    2022

  7. [15]

    Zhongzhen Huang, Xiaofan Zhang, and Shaoting Zhang. 2023. Kiut: Knowledge- injected u-transformer for radiology report generation. In CVPR. 19809–19818

    2023

  8. [16]

    Haibo Jin, Haoxuan Che, Yi Lin, and Hao Chen. 2024. Promptmrg: Diagnosis- driven prompts for medical report generation. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, Vol. 38. 2607–2615

    2024

  9. [17]

    Alistair EW Johnson, Tom J Pollard, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Yifan Peng, Zhiyong Lu, Roger G Mark, Seth J Berkowitz, and Steven Horng. 2019. MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs. arXiv preprint arXiv:1...

    2019 arXiv

  10. [18]

    Lee, Paul G

    Cindy S. Lee, Paul G. Nagy, Sallie J. Weaver, and David E. Newman-Toker. 2013. Cognitive and System Factors Contributing to Diagnostic Errors in Radiology. American Journal of Roentgenology 201, 3 (2013), 611–617. doi:10.2214/AJR.12. 10375 PMID: 23971454

    2013 doi

  11. [19]

    Kezhi Li, John Daniels, Chengyuan Liu, Pau Herrero, and Pantelis Georgiou. 2020. Convolutional Recurrent Neural Networks for Glucose Prediction. IEEE Journal of Biomedical and Health Informatics 24, 2 (2020), 603–613

    2020

  12. [20]

    Mingjie Li, Bingqian Lin, Zicong Chen, Haokun Lin, Xiaodan Liang, and Xiaojun Chang. 2023. Dynamic Graph Enhanced Contrastive Learning for Chest X-Ray Report Generation. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 3334–3343

    2023

  13. [21]

    Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and You Zhang. 2023. ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge. arXiv:2303.14070 [cs.CL] https://arxiv.org/abs/2303.14070

    2023 arXiv

  14. [22]

    Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. Association for Computational Linguistics, Barcelona, Spain, 74–81. https://aclanthology.org/W04-1013/

    2004

  15. [23]

    Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar. 2017. Focal Loss for Dense Object Detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV)

    2017

  16. [24]

    Yanbin Liu, Linchao Zhu, Makoto Yamada, and Yi Yang. 2020. Semantic Cor- respondence as an Optimal Transport Problem. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2020

  17. [25]

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) . 10012–10022

    2021

  18. [26]

    Gaspard Monge. 1781. Mémoire sur la théorie des déblais et des remblais.Histoire de l’Académie Royale des Sciences de Paris (1781)

  19. [27]

    Aaron Nicolson, Jason Dowling, and Bevan Koopman. 2022. Improving chest X-Ray report generation by leveraging warm-starting. arXiv preprint arXiv:2201.09405 (2022)

    2022 arXiv

  20. [28]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, and et al

  21. [29]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics , Pierre Isabelle, Eugene Charniak, and Dekang Lin (Eds...

    2002

  22. [30]

    Yossi Rubner, Carlo Tomasi, and Leonidas Guibas. 2000. The Earth Mover’s Distance as a Metric for Image Retrieval. International Journal of Computer Vision 40 (11 2000), 99–121

    2000

  23. [31]

    arXiv:2303.08774 [cs.CL] https://arxiv.org/abs/ 2303.08774

    GPT-4 Technical Report. arXiv:2303.08774 [cs.CL] https://arxiv.org/abs/ 2303.08774

    arXiv

  24. [32]

    Hoo-Chang Shin, Kirk Roberts, Le Lu, Dina Demner-Fushman, Jianhua Yao, and Ronald M. Summers. 2016. Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2016

  25. [33]

    Richard Sinkhorn and Paul Knopp. 1967. Concerning nonnegative matrices and doubly stochastic matrices. Pacific J. Math. 21, 2 (1967), 343–348

    1967

  26. [34]

    Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. 2000. The earth mover’s distance as a metric for image retrieval. International journal of computer vision 40, 2 (2000), 99–121

    2000

  27. [35]

    Tim Tanida, Philip Müller, Georgios Kaissis, and Daniel Rueckert. 2023. Interac- tive and Explainable Region-Guided Radiology Report Generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 7433–7442

    2023

  28. [36]

    Yitian Tao, Liyan Ma, Jing Yu, and Han Zhang. 2024. Memory-based Cross-modal Semantic Alignment Network for Radiology Report Generation. arXiv:2404.00588 [cs.CV] https://arxiv.org/abs/2404.00588

    2024 arXiv

  29. [37]

    Akshay Smit, Saahil Jain, Pranav Rajpurkar, Anuj Pareek, Andrew Y Ng, and Matthew Lungren. 2020. Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT. In Proceedings of the 2020 Conference on Empirical Methods in Natural Languag...

    2020

  30. [38]

    Hugo Touvron, Louis Martin, Kevin Stone, and et al. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv:2307.09288 [cs.CL] https://arxiv. org/abs/2307.09288

    2023 arXiv

  31. [39]

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2019. Representation Learning with Contrastive Predictive Coding. arXiv:1807.03748 [cs.LG] https://arxiv.org/ abs/1807.03748

    2019 arXiv

  32. [40]

    Shaker, Sahal Shaji Mullappilly, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Jorma Laaksonen, and Fahad Khan

    Omkar Chakradhar Thawakar, Abdelrahman M. Shaker, Sahal Shaji Mullappilly, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Jorma Laaksonen, and Fahad Khan. 2024. XrayGPT: Chest Radiographs Summarization using Large Med- ical Vision-Language Models. In Proceedings of the 23r...

    2024

  33. [41]

    Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, and Ronald M. Summers. 2018. TieNet: Text-Image Embedding Network for Common Thorax Disease Classifi- cation and Reporting in Chest X-Rays. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2018

  34. [42]

    Zhanyu Wang, Lingqiao Liu, Lei Wang, and Luping Zhou. 2023. METransformer: Radiology Report Generation by Transformer with Multiple Learnable Expert Tokens. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11558–11567

    2023

  35. [43]

    Gomez, Łukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, ...

    2017

  36. [44]

    Hongchen Xue, Qingzhi Ma, Guanfeng Liu, Jianfeng Qu, Yuanjun Liu, and An Liu

  37. [45]

    Bin Yan and Mingtao Pei. 2022. Clinical-BERT: Vision-Language Pre-training for Radiograph Diagnosis and Reports Generation. Proceedings of the AAAI Conference on Artificial Intelligence 36, 3 (Jun. 2022), 2982–2990. https://ojs.aaai. org/index.php/AAAI/article/view/20204

    2022

  38. [46]

    Zhanyu Wang, Lingqiao Liu, Lei Wang, and Luping Zhou. 2023. R2GenGPT: Radiology Report Generation with frozen LLMs. Meta-Radiology 1, 3 (2023), 100033. https://www.sciencedirect.com/science/article/pii/S2950162823000334

    2023

  39. [47]

    Kevin Zhou, and Li Xiao

    Shuxin Yang, Xian Wu, Shen Ge, S. Kevin Zhou, and Li Xiao. 2022. Knowledge matters: Chest radiology report generation with general and specific knowl- edge. Medical Image Analysis 80 (2022), 102510. https://www.sciencedirect.com/ science/article/pii/S1361841522001578

    2022

  40. [48]

    In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (Boise, ID, USA) (CIKM ’24)

    CLR2G: Cross modal Contrastive Learning on Radiology Report Generation. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (Boise, ID, USA) (CIKM ’24). Association for Computing Machinery, New York, NY, USA, 2742–2752

  41. [50]

    Kevin Zhou, and Li Xiao

    Shuxin Yang, Xian Wu, Shen Ge, Zhuozhao Zheng, S. Kevin Zhou, and Li Xiao

  42. [53]

    Shuxin Yang, Xian Wu, Shen Ge, S Kevin Zhou, and Li Xiao. 2022. Knowledge matters: Chest radiology report generation with general and specific knowledge. Medical image analysis 80 (2022), 102510

    2022

  43. [2016]

    Journal of the American Medical Informatics Association 23, 2 (2016), 304–310

    Preparing a collection of radiology examinations for distribution and retrieval. Journal of the American Medical Informatics Association 23, 2 (2016), 304–310

    2016

  44. [2023]

    Medical Image Analysis 86 (2023), 102798

    Radiology report generation with a learned knowledge base and multi- modal alignment. Medical Image Analysis 86 (2023), 102798. https://www. sciencedirect.com/science/article/pii/S1361841523000592

    2023

  45. [2024]

    InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.)

    Fine-Grained Image-Text Alignment in Medical Imaging Enables Explain- able Cyclic Image-Report Generation. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). A...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.