Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

R2GenKG: Hierarchical Multi-modal Knowledge Graph for LLM-based Radiology Report Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read R2GenKG claims that injecting a large multi-modal medical knowledge graph, built from ground-truth radiology reports and sampled at multiple granularities, feeds an LLM-based decoder and improves X-ray report generation on every measured…

desk verdict The M3KG resource is a real contribution, but the paper's headline claim is contradicted by its own Table 2 and the KG's test-split leakage risk is unresolved. read the letter →

arxiv 2508.03426 v1 pith:N2MLOJD2 submitted 2025-08-05 cs.CV cs.AIcs.LG

Futian Wang , Yuhan Qiao , Xiao Wang , Fuling Wang , Yuxiang Zhang , Dengdi Sun This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords radiologyreportgenerationmulti-modalknowledgegraphlargelanguagemodelsR-GCNcross-attentiondisease-awarevisiontokenschestX-rayclinicalefficacy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a large, multi-modal medical knowledge graph can fix two persistent weaknesses of automated X-ray report generation: hallucinated findings and missed diagnoses. Its central claim is that building such a graph from ground-truth reports (M3KG: 2,477 entities, three relation types, 37,424 triples, and 6,943 disease-aware vision tokens on CheXpert Plus) and routing it into a Llama-2-based generator through multi-scale graph encoding and cross-attention raises every evaluated metric above every baseline on IU-Xray and CheXpert Plus. If true, knowledge-graph augmentation becomes a practical recipe for making generated reports both more fluent and more clinically accurate, since the model's clinical precision and F1 also surpass prior systems. The pipeline replaces manual knowledge-graph annotation with GPT-4o-generated supervision, which is what makes a graph of this size affordable.

What carries the argument

The central object is M3KG, a multi-modal medical knowledge graph whose nodes are clinical entities (anatomy, disorder, concept, device, procedure, size) linked to UMLS concepts and whose edges carry three relation types (suggestive of, modify, located at), with disease-aware vision tokens attached to nodes. The argument is carried by the hierarchical injection pipeline: subgraphs at five granularities are encoded by a relational graph convolutional network, fused by self-attention with scale and positional encodings, and combined with Swin-Transformer image features through two cross-attention modules (KG2V and V2KG) plus a Q-Former retrieval step, all concatenated in Llama-2 embedding space to condition generation. The two-way cross-attention is what lets image content and graph facts reinforce each other, and the multi-scale sampling is what lets the model see both coarse and fine medical semantics.

What would settle it

Rebuild M3KG using only the training and validation splits and rerun the CheXpert Plus evaluation: if BLEU-4, CIDEr, and clinical F1 fall toward the BASE ablation values (0.101, 0.123, and 0.260), the reported gains come from test-set leakage rather than knowledge-graph reasoning. A direct static check would count how many test-split entity mentions appear among M3KG nodes and whether any disease-aware vision tokens were extracted from test images.

Watch

Extended reading notes

Core claim

The paper's discovery, on its own terms, is that medical knowledge can be extracted from radiology reports at scale and layered over an LLM so that generation improves measurably. M3KG is assembled by having GPT-4o annotate a subset of reports, training named-entity recognition and relation-extraction models on those annotations, running the models over the reports to produce triples of the form {head entity, tail entity, relation}, and attaching GradCAM-derived disease-aware vision tokens to the graph. R2GenKG then samples that graph at five node counts, encodes each scale with an R-GCN, fuses the scales with self-attention, and feeds the result into Llama-2-7B together with Swin-Transformer image features and retrieved disease-aware visual tokens. On IU-Xray and CheXpert Plus the paper reports top scores across BLEU-1 to BLEU-4, ROUGE-L, METEOR, CIDEr, and clinical efficacy (F1 0.292, precision 0.338, recall 0.275 on CheXpert Plus), and concludes that both the graph and the framework are effective.

Load-bearing premise

The load-bearing premise is that M3KG is built only from training reports, so no test-report findings leak into the entities, relations, or vision tokens the model can retrieve at evaluation time; the paper never states this.

Editorial extensions

If this is right

  • On CheXpert Plus, the full R2GenKG reaches BLEU-4 0.106, ROUGE-L 0.269, METEOR 0.151, CIDEr 0.125, and clinical F1 0.292, beating every listed baseline on every metric.
  • Ablation results show each component contributes: removing R-GCN, multi-scale fusion, and the disease visual graph together drops clinical F1 from 0.292 to 0.260.
  • Among graph encoders, R-GCN outperforms GCN and GAT (CIDEr 0.125 versus 0.116 and 0.121), indicating that modeling relation types matters.
  • Graph size matters: 300 entity nodes give the best scores, while 400 or 500 nodes degrade them, implying that overly large graphs introduce noise.
  • The optimal configuration uses 500 disease-aware visual tokens; moving to 700 or 1000 tokens slightly hurts all NLG metrics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An editorial concern is that the paper never states whether M3KG excludes test-split reports; if test reports were used to build the graph, the cross-attention modules could retrieve reference-report entities and vision tokens and inflate the reported gains. A clean check is to rebuild the graph from training and validation reports only and rerun the exact evaluation.
  • The paper's own limitation section concedes that the alignment between visual disease features and textual graphs is shallow and cross-modal fusion is limited, so the current numbers may understate what the graph could contribute if the two modalities were aligned more deeply.
  • Because the annotations that train the NER and relation-extraction models come from GPT-4o, the graph inherits whatever systematic errors that model makes; measuring NER/relation precision on a held-out sample of human-annotated reports would bound this effect.
  • The reported improvements over the strongest baselines are small in absolute terms (CIDEr 0.125 versus 0.123, F1 0.292 versus 0.288), so multi-seed runs with confidence intervals would clarify how much of the advantage is structural rather than noise.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes R2GenKG, a radiology report generation framework augmented by a newly built multi-modal knowledge graph called M3KG. M3KG is constructed from ground-truth radiology reports using GPT-4o-annotated training data for named entity recognition and relation extraction, and it includes textual triples and disease-aware vision tokens extracted via GradCAM. The generation model combines Swin Transformer visual features, Q-Former queries, multi-scale R-GCN encoded knowledge subgraphs, and cross-attention fusion before decoding with Llama2-7B. Experiments on IU-Xray and CheXpert Plus report natural language generation (NLG) metrics and clinical efficacy (CE) metrics, and the authors claim the method outperforms all baselines on CheXpert Plus.

Significance. If M3KG were constructed only from training data, the idea of a hierarchical multi-modal knowledge graph with disease-aware visual tokens is a plausible and timely contribution to LLM-based radiology report generation, and the multi-granularity graph encoding plus cross-attention design is technically coherent. The paper compares against many recent baselines and includes ablations over graph encoders, node counts, and visual feature counts. However, the experimental evidence as presented is not reliable: the central 'outperforms all baselines' claim is contradicted by the paper's own Table 2, and the construction pipeline is described using ground-truth reports without any statement that test data are excluded, which creates a direct leakage risk for both the textual and visual components of M3KG. The lack of a detailed CE evaluation protocol further weakens the clinical-efficacy comparison.

major comments (4)
  1. [Section 4.2 and Table 2] The claim 'Our method outperformed all baseline approaches across all metrics' is false as stated: on CheXpert Plus, Token-Mixer [58] achieves BLEU-1 0.378 while R2GenKG achieves 0.376. Even on IU-Xray, where the text only claims 'top-tier' performance, several baselines have higher BLEU-1 (R2GenCMN 0.475, PPKED 0.483, AlignTrans 0.484, DuCo-Net 0.500) and DuCo-Net has higher METEOR (0.240 vs 0.218). The report should be rewritten so that every quantitative claim matches the table entries; the current wording overstates the results and is load-bearing for the paper's central contribution.
  2. [Abstract and Section 3.2] The M3KG construction is described as being 'based on the ground truth medical report' (Abstract) and using GPT-4o to train NER and relation-extraction models that are then 'infer[red] ... to build the preliminary triplet' (Section 3.2). Nowhere do the authors state that reports from the test split are excluded from this construction. The same concern applies to the disease-aware vision tokens, which are extracted via GradCAM using the 14 CheXpert Plus classification labels (Section 3.2): if test images or their labels contribute to the stored vision tokens, then the cross-attention retrieval in Section 3.4 can directly access reference-report content at inference time. This would make the reported improvements a form of self-confirmation rather than generalization. The authors must (i) explicitly state that M3KG is built from the training split only, and (ii) provide evidence (e.g., code or a data-preprocessing description) that no test report entity, relation, or vision token is retrievable at evaluation time. Until then, the quantitative results cannot be interpreted.
  3. [Section 4.1 and Section 4.2] The CE metric is not sufficiently specified: the paper says it 'follow[s] R2Gen and use[s] the CE metric' but does not describe how clinical entities are extracted from generated reports, which label set is used, or how precision, recall, and F1 are computed. In addition, the text in Section 4.2 states 'our method outperforms all others in the two CE metrics,' but Table 5 shows ORGan [12] has a higher Recall (0.287 vs 0.275). A precise evaluation protocol and corrected claims are needed before the clinical-efficacy comparison can be assessed.
  4. [Section 4.4 and Table 4] The ablation claim that 'introducing any individual module or a combination of modules consistently led to performance improvements across both NLG and CE metrics' is contradicted by the table: setting (c) (RG+MF) yields CIDEr 0.120, below the BASE value of 0.123. Similarly, in Table 9, increasing the number of entities from 300 to 500 keeps CIDEr at 0.125, which does not support the statement that 'further increases to 400 and 500 nodes resulted in performance degradation across metrics.' These inconsistencies in the supporting analysis should be corrected.
minor comments (5)
  1. [Section 4.7 and Table 7] The text reports 'the number of parameters in our model is 915.63MB', while Table 7 lists Parameters as 239M and Memory Usage as 915.63MB; clarify whether 915.63MB refers to GPU memory footprint and what 'test speed' means per iteration or per sample.
  2. [References] Reference [40] is a duplicate of reference [39]; remove one of the duplicated Llama 2 entries.
  3. [Figure 4] The caption includes the words 'Submitted for review' as part of the example report text; this appears to be an artifact and should be removed or explained.
  4. [Section 4.3] The text says 'we employ gcn proj to map the graph node feature dimensions to the visual feature dimension'; define this projection in the architecture description or equations.
  5. [Section 3.4] After describing multi-scale fusion, the text says 'We use the output from the 300-node scale as the final graph representation'; clarify how the fused multi-scale representation relates to the chosen 300-node scale.

Circularity Check

1 steps flagged · score 7.0 of 10

M3KG is built from the ground-truth reports the model is asked to predict, and the paper never states that test reports are excluded; the generator then conditions on features retrieved from that same graph.

  1. self definitional [Abstract; Sec. 3.2 (M3KG construction); Sec. 3.4 (Fkv equation and final input F)]
    "we first construct a large-scale multi-modal medical knowledge graph (termed M3KG) based on the ground truth medical report using the GPT-4o. It contains 2477 entities, 3 kinds of relations, 37424 triples, and 6943 disease-aware vision tokens for the CheXpert Plus dataset. ... We infer the two models to build the preliminary triplet in the second stage. ... Fkv = CrossAttention(Fv, Kv, Kv) ... The final representation F is used as input to Llama2 [39] for report generation."

    M3KG is defined from the very ground-truth reports that R2GenKG is trained and evaluated to reproduce. The decoder conditions on F = Concat(Fv, Fkv, Fn2g, Fg2n), where Fkv is retrieved from disease-aware vision tokens stored in M3KG, and Fn2g/Fg2n are RGCN features of triples extracted from those same ground-truth reports. The paper gives M3KG statistics 'for the CheXpert Plus dataset' and describes a train/test partition only for model training and evaluation, never stating that the KG was built exclusively from the training split. On the paper's own description, the graph can include the test sample's entities, triples, and vision tokens, so the cross-attention modules can access reference-report content at inference.

full rationale

The central circularity risk is the unresolved split of M3KG: the knowledge graph is constructed 'based on the ground truth medical report', and the paper does not state that test reports are excluded from the 2477 entities, 37424 triples, and 6943 disease-aware vision tokens. Since the model's final input F includes features retrieved from this graph, the target report can enter the decoder if the graph spans the test split. Because this is not explicitly ruled out, the headline 'outperformed all baseline approaches across all metrics' cannot be taken as evidence for an independent method. Separately, that sentence is factually contradicted by the paper's own Table 2 on CheXpert Plus, where Token-Mixer's BLEU-1 is 0.378 versus R2GenKG's 0.376; that is a correctness flaw rather than a circularity one, so it does not inflate the circularity score beyond the KG issue. The rest of the pipeline (Swin encoder, Q-former, R-GCN, Llama2) is standard and not circular by itself. Score 7 rather than 8-10 because the leakage is a stated-assumption gap: if the authors clarify that M3KG was built only on the training split and that disease tokens are not extracted from test images, the circularity would largely disappear.

Assumptions & free parameters 3 free parameters · 2 assumptions · 1 invented entities

The central claim depends on a knowledge graph that is derived from the target reports themselves, plus tuned hyperparameters for node count and visual feature count. No independent evidence is provided for the graph's quality or its leak-free construction.

free parameters (3)
  • Number of entity nodes = 300
    Tuned by validation search in §4.5, Table 9; final model uses 300 nodes (implementation details say 319 nodes, but text says 300-node scale as final graph).
  • Number of visual disease features = 500
    Tuned in §4.5, Table 8; final model uses 500 features.
  • GradCAM threshold τ = Not reported
    Used in §3.2 to identify regions of interest for disease-aware vision tokens; value not given.
assumptions (2)
  • domain assumption GPT-4o annotations of radiology reports are accurate enough to train NER and relation extraction models that produce a useful knowledge graph.
    The entire M3KG depends on this; no manual validation or quality metrics are reported.
  • ad hoc to paper The knowledge graph is constructed from training reports only and does not contain test report information.
    This is required for valid evaluation but is never stated in the paper; the construction section (§3.2) only mentions 'ground truth medical report'.
invented entities (1)
  • M3KG multi-modal knowledge graph with disease-aware vision tokens
    purpose: Serve as external knowledge during report generation to improve clinical accuracy
    The graph is constructed by the authors, not discovered; its value is only measured through the model's own performance, and no external benchmark or human validation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of R2GenKG: Hierarchical Multi-modal Knowledge Graph for LLM-based Radiology Report Generation." pith.science (2026). https://pith.science/paper/N2MLOJD2

@misc{pith2026250803426,
  author       = {Pith},
  title        = {Pith review of: R2GenKG: Hierarchical Multi-modal Knowledge Graph for LLM-based Radiology Report Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N2MLOJD2}},
  note         = {Machine review of arXiv:2508.03426}
}
read the original abstract

X-ray medical report generation is one of the important applications of artificial intelligence in healthcare. With the support of large foundation models, the quality of medical report generation has significantly improved. However, challenges such as hallucination and weak disease diagnostic capability still persist. In this paper, we first construct a large-scale multi-modal medical knowledge graph (termed M3KG) based on the ground truth medical report using the GPT-4o. It contains 2477 entities, 3 kinds of relations, 37424 triples, and 6943 disease-aware vision tokens for the CheXpert Plus dataset. Then, we sample it to obtain multi-granularity semantic graphs and use an R-GCN encoder for feature extraction. For the input X-ray image, we adopt the Swin-Transformer to extract the vision features and interact with the knowledge using cross-attention. The vision tokens are fed into a Q-former and retrieved the disease-aware vision tokens using another cross-attention. Finally, we adopt the large language model to map the semantic knowledge graph, input X-ray image, and disease-aware vision tokens into language descriptions. Extensive experiments on multiple datasets fully validated the effectiveness of our proposed knowledge graph and X-ray report generation framework. The source code of this paper will be released on https://github.com/Event-AHU/Medical_Image_Analysis.

Figures

Figures reproduced from arXiv: 2508.03426 by the authors.

Figure 1
Figure 1. An overview of the existing benchmark models on the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An illustration of the proposed multi-modal medical knowledge graph M3KG. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. An overview of our proposed hierarchical knowledge graph guided X-ray medical report generation framework, termed [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: X-ray images and their corresponding ground-truths, along with the output of our model and R2GenGPT model generation [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: An illustration of the part of our proposed multi-modal [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CogRad: A Cognitively-Inspired Multi-Agent Framework for Radiology Report Generation

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A four-agent Scout–Investigator–Writer–Verifier pipeline with slot-attention regions and inference-time sentence re-examination leads NLG baselines on CheXpert Plus and IU X-Ray, with weaker clinical entity scores.

Reference graph

Works this paper leans on

60 extracted references · 40 canonical work pages · cited by 1 Pith paper

  1. [58]

    Token-mixer: Bind image and text in one embedding space for medical image reporting

    Yan Yang, Jun Yu, Zhenqi Fu, Ke Zhang, Ting Yu, Xianyun Wang, Hanliang Jiang, Junhui Lv, Qingming Huang, and Weidong Han. Token-mixer: Bind image and text in one embedding space for medical image reporting. IEEE Trans- actions on Medical Imaging, 43(11):4017–4028, 2024

    work page 2024

  2. [12]

    Organ: Observation-guided radiology report generation via tree reasoning

    Wenjun Hou, Kaishuai Xu, Yi Cheng, Wenjie Li, and Jiang Liu. Organ: Observation-guided radiology report generation via tree reasoning. arXiv preprint arXiv:2306.06466, 2023

    arXiv 2023

  3. [1]

    Automated radiology report genera- tion using conditioned transformers.Informatics in Medicine Unlocked, 24:100557, 2021

    Omar Alfarghaly, Rana Khaled, Abeer Elkorany, Maha Helal, and Aly Fahmy. Automated radiology report genera- tion using conditioned transformers.Informatics in Medicine Unlocked, 24:100557, 2021

    work page 2021

  4. [2]

    Publicly available clinical bert embeddings

    Emily Alsentzer, John R Murphy, Willie Boag, Wei-Hung Weng, Di Jin, Tristan Naumann, and Matthew McDermott. Publicly available clinical bert embeddings. arXiv preprint arXiv:1904.03323, 2019

    arXiv 1904

  5. [3]

    Meteor: An automatic metric for mt evaluation with improved correlation with hu- man judgments

    Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with hu- man judgments. In Proceedings of the acl workshop on in- trinsic and extrinsic evaluation measures for machine trans- lation and/or summarization, pages 65–72, 2005

    work page 2005

  6. [4]

    Chexpert plus: Augmenting a large chest x-ray dataset with text ra- diology reports, patient demographics and additional image formats

    Pierre Chambon, Jean-Benoit Delbrouck, Thomas Sounack, Shih-Cheng Huang, Zhihong Chen, Maya Varma, Steven QH Truong, Chu The Chuong, and Curtis P Langlotz. Chexpert plus: Augmenting a large chest x-ray dataset with text ra- diology reports, patient demographics and additional image formats. arXiv preprint arXiv:2405.19538, 2024

    arXiv 2024

  7. [5]

    Crossvit: Cross-attention multi-scale vision transformer for image classification

    Chun-Fu Richard Chen, Quanfu Fan, and Rameswar Panda. Crossvit: Cross-attention multi-scale vision transformer for image classification. In Proceedings of the IEEE/CVF in- ternational conference on computer vision , pages 357–366, 2021

    work page 2021

  8. [6]

    Generating radiology reports via memory-driven trans- former

    Zhihong Chen, Yan Song, Tsung-Hui Chang, and Xiang Wan. Generating radiology reports via memory-driven trans- former. arXiv preprint arXiv:2010.16056, 2020

    arXiv 2010

Show all 60 references
  1. [7]

    Cross-modal memory networks for radiology report gener- ation, 2022

    Zhihong Chen, Yaling Shen, Yan Song, and Xiang Wan. Cross-modal memory networks for radiology report gener- ation, 2022

    2022

  2. [8]

    Preparing a collection of radiology examinations for distribution and re- trieval

    Dina Demner-Fushman, Marc D Kohli, Marc B Rosen- man, Sonya E Shooshan, Laritza Rodriguez, Sameer Antani, George R Thoma, and Clement J McDonald. Preparing a collection of radiology examinations for distribution and re- trieval. Journal of the American Medical Informatics Asso...

    2016

  3. [9]

    Translating medical image to radiological report: Adaptive multilevel multi-attention approach

    Gaurav O Gajbhiye, Abhijeet V Nandedkar, and Ibrahima Faye. Translating medical image to radiological report: Adaptive multilevel multi-attention approach. Computer Methods and Programs in Biomedicine, 221:106853, 2022

    2022

  4. [10]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  5. [11]

    Recap: Towards precise radiology report generation via dynamic disease progression reasoning

    Wenjun Hou, Yi Cheng, Kaishuai Xu, Wenjie Li, and Jiang Liu. Recap: Towards precise radiology report generation via dynamic disease progression reasoning. arXiv preprint arXiv:2310.13864, 2023

    2023 arXiv

  6. [13]

    Radar: Enhancing radiology report generation with supplementary knowledge injection, 2025

    Wenjun Hou, Yi Cheng, Kaishuai Xu, Heng Li, Yan Hu, Wenjie Li, and Jiang Liu. Radar: Enhancing radiology report generation with supplementary knowledge injection, 2025

    2025

  7. [14]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perel- man, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Weli- hinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  8. [15]

    Pytorch library for cam methods, 2021

    Gildenblat Jacob. Pytorch library for cam methods, 2021

    2021

  9. [16]

    Duong, Tan Bui, Pierre Chambon, Yuhao Zhang, Matthew P

    Saahil Jain, Ashwin Agrawal, Adriel Saporta, Steven Truong, D. Duong, Tan Bui, Pierre Chambon, Yuhao Zhang, Matthew P. Lungren, Andrew Y . Ng, Curt P. Langlotz, and 10 Pranav Rajpurkar. Radgraph: Extracting clinical entities and relations from radiology reports. ArXiv, abs/210...

    2021 arXiv

  10. [17]

    Promptmrg: Diagnosis-driven prompts for medical report generation

    Haibo Jin, Haoxuan Che, Yi Lin, and Hao Chen. Promptmrg: Diagnosis-driven prompts for medical report generation. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 2607–2615, 2024

    2024

  11. [18]

    On the auto- matic generation of medical imaging reports

    Baoyu Jing, Pengtao Xie, and Eric Xing. On the auto- matic generation of medical imaging reports. arXiv preprint arXiv:1711.08195, 2017

    2017 arXiv

  12. [19]

    Semi-supervised classification with graph convo- lutional networks

    TN Kipf. Semi-supervised classification with graph convo- lutional networks. arXiv preprint arXiv:1609.02907, 2016

    2016 arXiv

  13. [20]

    Context-enhanced framework for medical image re- port generation using multimodal contexts

    Hongzhao Li, Hongyu Wang, Xia Sun, Hua He, and Jun Feng. Context-enhanced framework for medical image re- port generation using multimodal contexts. Knowledge- Based Systems, 310:112913, 2025

    2025

  14. [21]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In In- ternational conference on machine learning , pages 19730– 19742. PMLR, 2023

    2023

  15. [22]

    Dynamic graph enhanced contrastive learning for chest x-ray report generation

    Mingjie Li, Bingqian Lin, Zicong Chen, Haokun Lin, Xi- aodan Liang, and Xiaojun Chang. Dynamic graph enhanced contrastive learning for chest x-ray report generation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3334–3343, 2023

    2023

  16. [23]

    Con- trastive learning with counterfactual explanations for radiol- ogy report generation

    Mingjie Li, Haokun Lin, Liang Qiu, Xiaodan Liang, Ling Chen, Abdulmotaleb Elsaddik, and Xiaojun Chang. Con- trastive learning with counterfactual explanations for radiol- ogy report generation. InEuropean Conference on Computer Vision, pages 162–180. Springer, 2024

    2024

  17. [24]

    Divide and conquer: Isolating normal- abnormal attributes in knowledge graph-enhanced radiology report generation

    Xiao Liang, Yanlei Zhang, Di Wang, Haodi Zhong, Ronghan Li, and Quan Wang. Divide and conquer: Isolating normal- abnormal attributes in knowledge graph-enhanced radiology report generation. In ACM Multimedia 2024, 2024

    2024

  18. [25]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out , pages 74–81, 2004

    2004

  19. [26]

    Multi-grained radiology report generation with sentence- level image-language contrastive learning

    Aohan Liu, Yuchen Guo, Jun-hai Yong, and Feng Xu. Multi-grained radiology report generation with sentence- level image-language contrastive learning. IEEE Transac- tions on Medical Imaging, 43(7):2657–2669, 2024

    2024

  20. [27]

    Bootstrapping large language models for radiology report generation

    Chang Liu, Yuanhe Tian, Weidong Chen, Yan Song, and Yongdong Zhang. Bootstrapping large language models for radiology report generation. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 18635–18643, 2024

    2024

  21. [28]

    Exploring and distilling posterior and prior knowl- edge for radiology report generation

    Fenglin Liu, Xian Wu, Shen Ge, Wei Fan, and Yuexian Zou. Exploring and distilling posterior and prior knowl- edge for radiology report generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13753–13762, 2021

    2021

  22. [29]

    Contrastive attention for automatic chest x-ray report generation

    Fenglin Liu, Changchang Yin, Xian Wu, Shen Ge, Yuex- ian Zou, Ping Zhang, and Xu Sun. Contrastive attention for automatic chest x-ray report generation. arXiv preprint arXiv:2106.06965, 2021

    2021

  23. [30]

    Auto-encoding knowledge graph for unsupervised medical report generation

    Fenglin Liu, Chenyu You, Xian Wu, Shen Ge, Xu Sun, et al. Auto-encoding knowledge graph for unsupervised medical report generation. Advances in Neural Information Process- ing Systems, 34:16266–16279, 2021

    2021

  24. [31]

    Competence-based multimodal curriculum learning for med- ical report generation

    Fenglin Liu, Shen Ge, Yuexian Zou, and Xian Wu. Competence-based multimodal curriculum learning for med- ical report generation. arXiv preprint arXiv:2206.14579 , 2022

    2022 arXiv

  25. [32]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021

    2021

  26. [33]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  27. [34]

    A survey on deep learning and explainability for automatic report generation from medical images

    Pablo Messina, Pablo Pino, Denis Parra, Alvaro Soto, Cecilia Besa, Sergio Uribe, Marcelo And ´ıa, Cristian Tejos, Claudia Prieto, and Daniel Capurro. A survey on deep learning and explainability for automatic report generation from medical images. ACM Computing Surveys (CSUR) ...

    2022

  28. [35]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages 311–318, 2002

    2002

  29. [36]

    Pytorch: An im- perative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An im- perative style, high-performance deep learning library. Ad- vances in neural information processing systems,...

    2019

  30. [37]

    Duco-net: Dual-contrastive learning network for medical report retrieval leveraging en- hanced encoders and augmentations

    Zahid Ur Rahman, Ju-Hwan Lee, Dang Thanh Vu, Iqbal Murtza, and Jin-Young Kim. Duco-net: Dual-contrastive learning network for medical report retrieval leveraging en- hanced encoders and augmentations. IEEE Access, 2025

    2025

  31. [38]

    Modeling rela- tional data with graph convolutional networks

    Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling rela- tional data with graph convolutional networks. In European semantic web conference, pages 593–607. Springer, 2018

    2018

  32. [40]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  33. [41]

    Cider: Consensus-based image description evalua- tion

    Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evalua- tion. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566–4575, 2015

    2015

  34. [42]

    Graph at- tention networks

    Petar Veli ˇckovi´c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph at- tention networks. arXiv preprint arXiv:1710.10903, 2017

    2017 arXiv

  35. [43]

    Cross-modal pro- totype driven network for radiology report generation

    Jun Wang, Abhir Bhalerao, and Yulan He. Cross-modal pro- totype driven network for radiology report generation. In European Conference on Computer Vision, pages 563–579. Springer, 2022. 11

    2022

  36. [44]

    Large-scale multi-modal pre-trained models: A comprehensive survey

    Xiao Wang, Guangyao Chen, Guangwu Qian, Pengcheng Gao, Xiao-Yong Wei, Yaowei Wang, Yonghong Tian, and Wen Gao. Large-scale multi-modal pre-trained models: A comprehensive survey. Machine Intelligence Research , 20 (4):447–482, 2023

    2023

  37. [45]

    R2gencsr: Retrieving context samples for large language model based x-ray medical report generation

    Xiao Wang, Yuehang Li, Fuling Wang, Shiao Wang, Chuanfu Li, and Bo Jiang. R2gencsr: Retrieving context samples for large language model based x-ray medical report generation. arXiv preprint arXiv:2408.09743, 2024

    2024

  38. [46]

    Pre-training on high definition x-ray images: An experimental study

    Xiao Wang, Yuehang Li, Wentao Wu, Jiandong Jin, Yao Rong, Bo Jiang, Chuanfu Li, and Jin Tang. Pre-training on high definition x-ray images: An experimental study. arXiv preprint arXiv:2404.17926, 2024

    2024 arXiv

  39. [47]

    Cxpmrg-bench: Pre-training and benchmarking for x-ray medical report generation on chexpert plus dataset

    Xiao Wang, Fuling Wang, Yuehang Li, Qingchuan Ma, Shiao Wang, Bo Jiang, and Jin Tang. Cxpmrg-bench: Pre-training and benchmarking for x-ray medical report generation on chexpert plus dataset. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR) , pag...

    2025

  40. [48]

    Ac- tivating associative disease-aware vision token memory for llm-based x-ray report generation

    Xiao Wang, Fuling Wang, Haowen Wang, Bo Jiang, Chuanfu Li, Yaowei Wang, Yonghong Tian, and Jin Tang. Ac- tivating associative disease-aware vision token memory for llm-based x-ray report generation. arXiv preprint arXiv:2501.03458, 2025

    2025 arXiv

  41. [49]

    Rethinking med- ical report generation: Disease revealing enhancement with knowledge graph

    Yixin Wang, Zihao Lin, and Haoyu Dong. Rethinking med- ical report generation: Disease revealing enhancement with knowledge graph. ArXiv, abs/2307.12526, 2023

    2023 arXiv

  42. [50]

    Metransformer: Radiology report generation by transformer with multiple learnable expert tokens

    Zhanyu Wang, Lingqiao Liu, Lei Wang, and Luping Zhou. Metransformer: Radiology report generation by transformer with multiple learnable expert tokens. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11558–11567, 2023

    2023

  43. [51]

    R2gengpt: Radiology report generation with frozen llms

    Zhanyu Wang, Lingqiao Liu, Lei Wang, and Luping Zhou. R2gengpt: Radiology report generation with frozen llms. Meta-Radiology, 1(3):100033, 2023

    2023

  44. [52]

    Mca-rg: Enhancing llms with medical concept alignment for radiology report generation

    Qilong Xing, Zikai Song, Youjia Zhang, Na Feng, Junqing Yu, and Wei Yang. Mca-rg: Enhancing llms with medical concept alignment for radiology report generation. arXiv preprint arXiv:2507.06992, 2025

    2025 arXiv

  45. [53]

    Generating radiology reports via auxiliary signal guidance and a memory-driven network.Expert Systems with Applications, 237:121260, 2024

    Youyuan Xue, Yun Tan, Ling Tan, Jiaohua Qin, and Xuyu Xiang. Generating radiology reports via auxiliary signal guidance and a memory-driven network.Expert Systems with Applications, 237:121260, 2024

    2024

  46. [54]

    Weakly super- vised contrastive learning for chest x-ray report generation

    An Yan, Zexue He, Xing Lu, Jiang Du, Eric Chang, Amilcare Gentili, Julian McAuley, and Chun-Nan Hsu. Weakly super- vised contrastive learning for chest x-ray report generation. arXiv preprint arXiv:2109.12242, 2021

    2021 arXiv

  47. [55]

    Memory-aligned knowledge graph for clinically accurate radiology image report generation

    Sixing Yan. Memory-aligned knowledge graph for clinically accurate radiology image report generation. In Proceedings of the 21st Workshop on Biomedical Language Processing , pages 116–122, 2022

    2022

  48. [56]

    Attributed abnor- mality graph embedding for clinically accurate x-ray report generation

    Sixing Yan, William K Cheung, Keith Chiu, Terence M Tong, Ka Chun Cheung, and Simon See. Attributed abnor- mality graph embedding for clinically accurate x-ray report generation. IEEE Transactions on Medical Imaging, 42(8): 2211–2222, 2023

    2023

  49. [57]

    Qwen3 technical report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025

    2025 arXiv

  50. [59]

    Aligntransformer: Hierarchical alignment of vi- sual regions and disease tags for medical report generation

    Di You, Fenglin Liu, Shen Ge, Xiaoxia Xie, Jing Zhang, and Xian Wu. Aligntransformer: Hierarchical alignment of vi- sual regions and disease tags for medical report generation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 72–...

    2021

  51. [60]

    Uncovering knowledge gaps in radiology report generation models through knowledge graphs

    Xiaoman Zhang, Juli ´an N Acosta, Hong-Yu Zhou, and Pranav Rajpurkar. Uncovering knowledge gaps in radiology report generation models through knowledge graphs. arXiv preprint arXiv:2408.14397, 2024

    2024 arXiv

  52. [61]

    When radiology report generation meets knowledge graph

    Yixiao Zhang, Xiaosong Wang, Ziyue Xu, Qihang Yu, Alan Yuille, and Daguang Xu. When radiology report generation meets knowledge graph. In Proceedings of the AAAI confer- ence on artificial intelligence, pages 12910–12917, 2020. 12

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.