Pith. sign in

REVIEW 5 major objections 5 minor 43 references

Learnable Retrieval Enhanced Visual-Text Alignment and Fusion for Radiology Report Generation

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read REVTAF claims that making reference-report retrieval learnable, with a hyperbolic ranking metric supervised by CheXbert disease-status hashes and a fine-grained attention-consistency fusion, raises average performance on two radiology…

desk verdict Plausible new combination, but the headline clinical-efficacy gains are partly contaminated by CheXbert being used for both retrieval supervision and evaluation; still worth refereeing. read the letter →

arxiv 2507.07568 v1 pith:TI7Y5L2H submitted 2025-07-10 stat.ME eess.IV

Qin Zhou , Guoyan Liang , Xindi Li , Jingyuan Chen , Wang Zhe , Chang Yao , Sai Wu This is my paper
classification stat.MEeess.IV
keywords radiologyreportgenerationlearnableretrievalhyperbolicspacemetriccross-modalalignmentoptimaltransportattentionclassimbalanceMIMIC-CXRIUX-Ray
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that radiology report generation fails partly because retrieval of reference reports uses generic image-text similarity, and partly because retrieved text is fused with images at too coarse a level. It proposes REVTAF, a framework whose learnable retrieval enhancer selects reference reports using a hyperbolic-space ranking metric supervised by disease-status hashing distances, so that underrepresented tail-class cases receive more relevant references. A fine-grained alignment and fusion module then enforces consistency between attention maps from global and local reference prompts and uses optimal-transport cross-attention to blend them with visual features. The claim is that this joint design outperforms prior state-of-the-art methods by 7.4% average on MIMIC-CXR and 2.9% on IU X-Ray, and also beats GPT-series multimodal LLMs on language quality and clinical efficacy. If true, it shows that retrieval augmentation and fine-grained cross-modal consistency can be made learnable and effective under weak image-level supervision.

What carries the argument

The load-bearing pieces are the Learnable Retrieval Enhancer (LRE) and the Fine-grained Visual-Text Alignment and Fusion (FVTAF) module. LRE maps MedKLIP entity logits into a Poincaré ball (a hyperbolic space where distances grow toward the boundary, capturing hierarchical similarity), computes pairwise hyperbolic geodesic distances within a batch, and trains a cross-entropy ranking loss against ground-truth hashing distances derived from CheXbert's 72-dimensional disease-status vectors (Eqs. 3-7). FVTAF builds two cross-attention branches — global reference prompts from the retrieved report and local reference prompts from MedKLIP entity triplets — and fuses them via Multi-Prompts Sinkhorn Attention, an optimal-transport reweighting of attention. A Fine-grained Cross-modal Consistency loss (Eq. 14) penalizes mismatch between sentence-entity cosine similarity and the intersection-over-union of the two attention map sets. The ranking supervision is what makes retrieval learnable; the FCC loss and Sinkhorn attention are what make the fusion fine-grained.

What would settle it

Compare models on a MIMIC-CXR test split using a second, independent annotation source (radiologist labels or a different labeler) for both retrieval supervision and clinical scoring; if the reported 7.4% and 2.9% average improvements shrink materially, the gains are partly an artifact of CheXbert-label contamination rather than a pure generation improvement.

Watch

Extended reading notes

Core claim

The central claim is that an end-to-end learnable retrieval mechanism, guided by medical disease-status labels rather than pooled image features, produces more useful reference reports for generation, especially for rare findings, and that aligning and fusing reference text at sentence-entity granularity further closes the visual-text gap. The authors show that each of the two modules independently improves the baseline, and together they raise mean NLG and CE metrics on MIMIC-CXR by 2.7% and 2.9% over a MedKLIP-based baseline, with the full model surpassing published state-of-the-art methods across all reported metrics. The paper also reports that the model outperforms GPT-4, GPT-4o, GPT-4o-mini, and GPT-4.5 on a 16-sample MIMIC-CXR subset, with shorter inference time.

Load-bearing premise

The load-bearing premise is that CheXbert's disease-status labels are accurate enough to serve both as the training signal that picks reference reports and as the oracle that scores clinical correctness, so label noise or bias would contaminate both the retrieval guidance and the reported gains.

Editorial extensions

If this is right

  • Retrieving reference reports should be trained with medical semantic supervision (disease-status hashing distances) rather than generic image-text similarity, because that is what the LRE's hyperbolic ranking metric learns from.
  • Tail-class inputs — uncommon findings — get the largest benefit from adaptive retrieval, since the retrieved references are chosen per input instead of by a fixed nearest-neighbor rule.
  • Enforcing fine-grained consistency between global and local attention maps, together with optimal-transport reweighted fusion, raises both BLEU/ROUGE-style language metrics and CheXbert-based precision, recall, and F1.
  • These gains come without scaling to large multimodal LLMs: the framework uses a ResNet-101 encoder, MedKLIP text features, and about 10 hours of single-GPU training, which is a plausible budget for clinical deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper's train/test gap — reference reports chosen by ground-truth hashing distances at training but by learned hyperbolic distance at inference — could inflate measured gains; a consistent retrieval policy across training and inference would give a cleaner estimate.
  • Editorial inference: the same hashing-distance supervision could be repurposed as a reward for reinforcement fine-tuning of the decoder, which the paper does not explore.
  • Editorial inference: the FCC idea — semantic similarity between multi-source prompts should match spatial overlap of their attention maps — transfers to other weakly supervised vision-language tasks such as medical visual question answering.
  • Editorial inference: the comparison with GPT models uses 16 samples and automatic metrics; a larger, blinded radiologist study would test whether the advantage persists on clinically consequential errors.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The manuscript proposes REVTAF, a radiology report generation framework with two main components: a Learnable Retrieval Enhancer (LRE) that retrieves Global Reference Prompts via a hyperbolic-space ranking metric supervised by CheXbert-derived hashing distances, and a Fine-grained Visual-Text Alignment and Fusion (FVTAF) module that enforces cross-modal consistency and uses optimal-transport-based cross-attention to fuse visual features with global and local text prompts. The authors report state-of-the-art results on MIMIC-CXR and IU X-Ray, with average improvements of 7.4% and 2.9%, and claim superiority over GPT-series multimodal LLMs on a small sample.

Significance. If the empirical claims are robust, the paper makes a useful architectural contribution: the LRE is a plausible way to inject adaptively retrieved reference knowledge, and the FCC constraint plus multi-prompt Sinkhorn attention are reasonable mechanisms for improving visual-text fusion under weak supervision. The authors provide a public code link, use standard benchmarks, and include ablations for the two proposed modules, which are positive features. However, the headline clinical-efficacy gains are weakened by the dual use of CheXbert as both the supervision signal and the evaluation oracle, and the lack of statistical significance testing and the very small LLM comparison sample leave the central SOTA claim only partially supported.

major comments (5)
  1. [§3.3 and §4.1] The LRE ranking loss in Section 3.3, Eq. (6)-(7), is supervised by Hamming distances between CheXbert status vectors, and the clinical efficacy evaluation in Section 4.1 computes precision, recall, and F1 using the same CheXbert labeler over the same 14/18 categories. The large CE gains in Table 1 (11.1%, 13.0%, and 9.3% over EKAGen) are therefore partly self-confirming: the retrieval module is optimized to reproduce CheXbert's labeling pattern, and the CE metrics reward generated text that matches those same labels. To support the clinical-efficacy claim, please evaluate with an independent labeler or a human-verified subset, and present the NLG metrics as the primary evidence for the method's advantage.
  2. [§3.3] The paper states that during training the GRP is selected using ground-truth hashing distances between report labels, while at inference it is selected by the learned hyperbolic distance. This is a train/test shift whose effect is never quantified. Please report retrieval quality under both selection modes on the test set, for example oracle retrieval versus learned retrieval, with retrieval precision/recall or an end-to-end comparison, so that the LRE benefit is not an upper-bound estimate from an oracle-informed training procedure.
  3. [Tables 1-4] All reported numbers are single-run values without error bars, confidence intervals, or significance tests. Some margins are very small, such as Table 2 BLEU-2 (0.249 versus 0.247) and BLEU-3 (0.159 versus 0.160), so the claim of consistent state-of-the-art performance is not statistically supported. Please provide multiple seeds with mean and standard deviation, or at least significance tests for the key comparisons against the strongest baselines.
  4. [Table 4] The comparison with GPT-series multimodal LLMs uses only 16 randomly selected samples, with no confidence intervals, no description of the prompt/decoding settings used for the GPT models, and no statistical test. This sample size is far too small to support the claim that REVTAF surpasses GPT-series models. Please either enlarge the evaluation set substantially or present a paired significance test and acknowledge the limited scope.
  5. [§3.3 and §4.2] The paper motivates LRE as particularly beneficial for underrepresented tail classes, but no experiment stratifies results by disease frequency or analyzes tail-class performance. The class-imbalance claim is load-bearing for the contribution, so please add a per-disease or frequency-stratified analysis to verify that the gains actually concentrate on tail classes rather than common findings.
minor comments (5)
  1. [Table 1 and Table 2] The table headers contain formatting errors: "PromptMRGAAAI 2024" and "CVT2Dis. Artif.Intell.Med 20220.392" are missing spaces, which makes them difficult to parse.
  2. [§4.1] The clinical efficacy evaluation is attributed to reference [12], which is the CheXpert paper by Irvin et al.; CheXbert is reference [27] (Smit et al.). Please correct the citation.
  3. [§3.4] There is a typo in the phrase "fused local viusal-text features" on the paragraph following Eq. (10); it should read "visual-text".
  4. [§4.3] The ablation text says LRE improves average NLG and CE metrics by 1.8% and 0.9%, but these are relative improvements over the baseline; please specify whether they are percentage points or relative percentages to avoid ambiguity.
  5. [§3.4 and §4.1] The balancing coefficients α and β in Eq. (15) are set to 2 and 0.5 without any sensitivity analysis or ablation; a brief study of these hyperparameters would strengthen the presentation.

Circularity Check

1 steps flagged · score 6.0 of 10

CheXbert is used both to supervise the retrieval module (Eqs. 6–7) and to compute the clinical-efficacy metrics (Sec. 4.1), so the headline CE gains are partly self-confirming even though NLG gains are independent.

  1. fitted input called prediction [Section 3.3 (Eqs. 6–7) and Section 4.1 (Evaluation Metrics)]
    "To incorporate semantic guidance into hyperbolic feature learning, we leverage the semantic similarity between paired radiology reports as supervisory signal. Specifically, for each sample, we extract structured classification labels from its corresponding report R using CheXbert [27], which maps R to K = 18 predefined disease categories. ... Then we can calculate the ground-truth semantic distance matrix D ∈ R^{B×B} using hashing distance between their status vectors. ..."

    Equation 6 defines the retrieval supervision D_ij as the Hamming distance between CheXbert status vectors v_i and v_j, and Equation 7 trains the Learnable Retrieval Enhancer's hyperbolic ranking to match that exact matrix. Section 4.1 then evaluates clinical efficacy by running the same CheXbert labeler on the generated reports and comparing with ground-truth labels. The LRE is therefore explicitly optimized to produce reference reports whose CheXbert status vector is close to the target's, and the CE metric rewards generated text that CheXbert maps to those statuses. This makes the LRE-specific CE improvement over baselines that do not use CheXbert as supervision partly constructed by the training objective.

full rationale

REVTAF's NLG improvements are measured against fixed reference reports using BLEU, METEOR, and ROUGE-L; those metrics do not share the CheXbert supervision channel and are therefore independently informative. The CE claim, however, passes through CheXbert twice: LRE's ranking loss (Eqs. 6–7) is supervised by CheXbert hashing distances among disease status vectors, and Section 4.1 measures clinical efficacy as CheXbert precision/recall/F1. The retrieval guidance and the evaluation oracle are thus the same function, so a model that learns to reproduce CheXbert's labeling pattern is rewarded on CE by construction. This does not collapse the entire paper: the generator still has to convert the retrieved reference into fluent sentences, and all baselines are evaluated with the same CheXbert oracle, so the relative CE gap can still reflect real differences. Even so, the LRE-specific CE gain is partly a self-confirming artifact of fitting to the same labeler that later scores the generated report. There are no load-bearing self-citations or imported uniqueness theorems, and the train/test retrieval gap mentioned in the paper is a robustness concern rather than a circularity. A score of 6 reflects the partial circularity in the CE channel while recognizing that the NLG results and the overall architecture retain independent content.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on external pretrained models (MedKLIP, CheXbert), empirically set loss weights, and two unproven domain assumptions about labeler reliability and hyperbolic hierarchy. No new physical or mathematical entities are introduced.

free parameters (2)
  • alpha = 2
    Loss weight for L_rank in Eq. 15; set empirically, no sensitivity analysis reported.
  • beta = 0.5
    Loss weight for L_fcc in Eq. 15; set empirically, no sensitivity analysis reported.
assumptions (3)
  • domain assumption CheXbert status vectors are a reliable proxy for semantic similarity between reports
    Used to define ground-truth distance matrix D in Eq. 6 and to compute CE evaluation metrics in Section 4.1.
  • domain assumption MedKLIP provides useful entity-level and report-level text prompts
    LRPs and GRPs are both generated by the pretrained MedKLIP encoder (Section 3.2).
  • domain assumption Hyperbolic distance on MedKLIP logits reflects anatomical hierarchy
    Motivates the LRE ranking metric (Section 3.3), but no evidence is given that the HNN actually learns anatomical hierarchy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learnable Retrieval Enhanced Visual-Text Alignment and Fusion for Radiology Report Generation." pith.science (2026). https://pith.science/paper/TI7Y5L2H

@misc{pith2026250707568,
  author       = {Pith},
  title        = {Pith review of: Learnable Retrieval Enhanced Visual-Text Alignment and Fusion for Radiology Report Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TI7Y5L2H}},
  note         = {Machine review of arXiv:2507.07568}
}
read the original abstract

Automated radiology report generation is essential for improving diagnostic efficiency and reducing the workload of medical professionals. However, existing methods face significant challenges, such as disease class imbalance and insufficient cross-modal fusion. To address these issues, we propose the learnable Retrieval Enhanced Visual-Text Alignment and Fusion (REVTAF) framework, which effectively tackles both class imbalance and visual-text fusion in report generation. REVTAF incorporates two core components: (1) a Learnable Retrieval Enhancer (LRE) that utilizes semantic hierarchies from hyperbolic space and intra-batch context through a ranking-based metric. LRE adaptively retrieves the most relevant reference reports, enhancing image representations, particularly for underrepresented (tail) class inputs; and (2) a fine-grained visual-text alignment and fusion strategy that ensures consistency across multi-source cross-attention maps for precise alignment. This component further employs an optimal transport-based cross-attention mechanism to dynamically integrate task-relevant textual knowledge for improved report generation. By combining adaptive retrieval with multi-source alignment and fusion, REVTAF achieves fine-grained visual-text integration under weak image-report level supervision while effectively mitigating data imbalance issues. The experiments demonstrate that REVTAF outperforms state-of-the-art methods, achieving an average improvement of 7.4% on the MIMIC-CXR dataset and 2.9% on the IU X-Ray dataset. Comparisons with mainstream multimodal LLMs (e.g., GPT-series models), further highlight its superiority in radiology report generation https://github.com/banbooliang/REVTAF-RRG.

Figures

Figures reproduced from arXiv: 2507.07568 by the authors.

Figure 1
Figure 1. Overview of the proposed Retrieval Enhanced Visual-Text Alignment and Fusion (REVTAF) framework. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the Fine-grained Visual-Text Alignment [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Qualitative results on the MIMIC-CXR dataset. Matches with the ground truth are highlighted in the same colors, while [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 32 canonical work pages

  1. [1]

    Bottom-up and top-down attention for image captioning and visual question answering

    Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 6077–6086, 2018. 2

    2018

  2. [2]

    Instance-level expert knowledge and aggregate discrimina- tive attention for radiology report generation

    Shenshen Bu, Taiji Li, Yuedong Yang, and Zhiming Dai. Instance-level expert knowledge and aggregate discrimina- tive attention for radiology report generation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14194–14204, 2024. 2, 7

    work page 2024

  3. [3]

    Fine-grained image-text alignment in medical imaging en- ables cyclic image-report generation

    Wenting Chen, Xiang Li, Linlin Shen, and Yixuan Yuan. Fine-grained image-text alignment in medical imaging en- ables cyclic image-report generation. arXiv preprint arXiv:2312.08078, 2023. 3, 4

    arXiv 2023

  4. [4]

    Generating radiology reports via memory-driven trans- former

    Zhihong Chen, Yan Song, Tsung-Hui Chang, and Xiang Wan. Generating radiology reports via memory-driven trans- former. arXiv preprint arXiv:2010.16056, 2020. 2, 7

    arXiv 2010

  5. [5]

    Cross-modal memory networks for radiology report gener- ation

    Zhihong Chen, Yaling Shen, Yan Song, and Xiang Wan. Cross-modal memory networks for radiology report gener- ation. arXiv preprint arXiv:2204.13258, 2022. 2, 6

    arXiv 2022

  6. [6]

    Meshed-memory transformer for image cap- tioning

    Marcella Cornia, Matteo Stefanini, Lorenzo Baraldi, and Rita Cucchiara. Meshed-memory transformer for image cap- tioning. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 10578–10587,

  7. [7]

    To- wards diverse and natural image descriptions via a condi- tional gan

    Bo Dai, Sanja Fidler, Raquel Urtasun, and Dahua Lin. To- wards diverse and natural image descriptions via a condi- tional gan. In Proceedings of the IEEE international confer- ence on computer vision, pages 2970–2979, 2017. 2

    work page 2017

  8. [8]

    Meteor 1.3: Automatic metric for reliable optimization and evaluation of machine translation systems

    Michael Denkowski and Alon Lavie. Meteor 1.3: Automatic metric for reliable optimization and evaluation of machine translation systems. In Proceedings of the sixth workshop on statistical machine translation, pages 85–91, 2011. 6

    work page 2011

Show all 43 references
  1. [9]

    Long short-term memory

    Alex Graves and Alex Graves. Long short-term memory. Su- pervised sequence labelling with recurrent neural networks, pages 37–45, 2012. 2

    2012

  2. [10]

    Scaling up vision-language pre-training for image captioning

    Xiaowei Hu, Zhe Gan, Jianfeng Wang, Zhengyuan Yang, Zicheng Liu, Yumao Lu, and Lijuan Wang. Scaling up vision-language pre-training for image captioning. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17980–17989, 2022. 2

    2022

  3. [11]

    Kiut: Knowledge-injected u-transformer for radiology report generation

    Zhongzhen Huang, Xiaofan Zhang, and Shaoting Zhang. Kiut: Knowledge-injected u-transformer for radiology report generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19809– 19818, 2023. 2, 6, 7

    2023

  4. [12]

    Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison

    Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Sil- viana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAA...

    2019

  5. [13]

    Promptmrg: Diagnosis-driven prompts for medical report generation

    Haibo Jin, Haoxuan Che, Yi Lin, and Hao Chen. Promptmrg: Diagnosis-driven prompts for medical report generation. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 2607–2615, 2024. 1, 2, 3, 4, 6, 7

    2024

  6. [14]

    Zero-shot camouflaged object detection

    Haoran Li, Chun-Mei Feng, Yong Xu, Tao Zhou, Lina Yao, and Xiaojun Chang. Zero-shot camouflaged object detection. IEEE Transactions on Image Processing, 2023. 2

    2023

  7. [15]

    Dynamic graph enhanced contrastive learning for chest x-ray report generation

    Mingjie Li, Bingqian Lin, Zicong Chen, Haokun Lin, Xi- aodan Liang, and Xiaojun Chang. Dynamic graph enhanced contrastive learning for chest x-ray report generation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3334–3343, 2023. 2, 7

    2023

  8. [16]

    Unify, align and refine: Multi- level semantic alignment for radiology report generation

    Yaowei Li, Bang Yang, Xuxin Cheng, Zhihong Zhu, Hongx- iang Li, and Yuexian Zou. Unify, align and refine: Multi- level semantic alignment for radiology report generation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2863–2874, 2023. 1, 7

    2023

  9. [17]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out , pages 74–81, 2004. 6

    2004

  10. [18]

    Exploring and distilling posterior and prior knowl- edge for radiology report generation

    Fenglin Liu, Xian Wu, Shen Ge, Wei Fan, and Yuexian Zou. Exploring and distilling posterior and prior knowl- edge for radiology report generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13753–13762, 2021. 1, 2

    2021

  11. [19]

    Grounding dino: Marry- ing dino with grounded pre-training for open-set object de- tection, 2024

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang. Grounding dino: Marry- ing dino with grounded pre-training for open-set object de- tection, 2024. 5

    2024

  12. [20]

    Knowing when to look: Adaptive attention via a visual sen- tinel for image captioning

    Jiasen Lu, Caiming Xiong, Devi Parikh, and Richard Socher. Knowing when to look: Adaptive attention via a visual sen- tinel for image captioning. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 375–383, 2017. 2

    2017

  13. [21]

    Im- proving chest x-ray report generation by leveraging warm starting

    Aaron Nicolson, Jason Dowling, and Bevan Koopman. Im- proving chest x-ray report generation by leveraging warm starting. Artificial intelligence in medicine , 144:102633,

  14. [22]

    Progressive transformer-based generation of radiology re- ports

    Farhad Nooralahzadeh, Nicolas Perez Gonzalez, Thomas Frauenfelder, Koji Fujimoto, and Michael Krauthammer. Progressive transformer-based generation of radiology re- ports. arXiv preprint arXiv:2102.09777, 2021. 7

    2021 arXiv

  15. [23]

    An introduction to convolutional neural networks

    K O’Shea. An introduction to convolutional neural networks. arXiv preprint arXiv:1511.08458, 2015. 2

    2015 arXiv

  16. [24]

    X-linear attention networks for image captioning

    Yingwei Pan, Ting Yao, Yehao Li, and Tao Mei. X-linear attention networks for image captioning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10971–10980, 2020. 1, 2

    2020

  17. [25]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages 311–318,

  18. [26]

    Self-critical sequence training for image captioning

    Steven J Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. Self-critical sequence training for image captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7008–7024,

  19. [27]

    Chexbert: com- bining automatic labelers and expert annotations for accu- rate radiology report labeling using bert

    Akshay Smit, Saahil Jain, Pranav Rajpurkar, Anuj Pareek, Andrew Y Ng, and Matthew P Lungren. Chexbert: com- bining automatic labelers and expert annotations for accu- rate radiology report labeling using bert. arXiv preprint arXiv:2004.09167, 2020. 4

    2004 arXiv

  20. [28]

    Interactive and explainable region-guided radiol- ogy report generation

    Tim Tanida, Philip M ¨uller, Georgios Kaissis, and Daniel Rueckert. Interactive and explainable region-guided radiol- ogy report generation. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 7433–7442, 2023. 7

    2023

  21. [29]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 2

    2017

  22. [30]

    Hergen: El- evating radiology report generation with longitudinal data,

    Fuying Wang, Shenghui Du, and Lequan Yu. Hergen: El- evating radiology report generation with longitudinal data,

  23. [31]

    Cross-modal pro- totype driven network for radiology report generation

    Jun Wang, Abhir Bhalerao, and Yulan He. Cross-modal pro- totype driven network for radiology report generation. In European Conference on Computer Vision, pages 563–579. Springer, 2022. 2

    2022

  24. [32]

    Multi-view feature fusion and visual prompt for remote sensing image captioning

    Shuang Wang, Qiaoling Lin, Xiutiao Ye, Yu Liao, Dou Quan, Zhongqian Jin, Biao Hou, and Licheng Jiao. Multi-view feature fusion and visual prompt for remote sensing image captioning. IEEE Transactions on Geoscience and Remote Sensing, 2024. 1, 5

    2024

  25. [33]

    Medclip: Contrastive learning from unpaired medical images and text, 2022

    Zifeng Wang, Zhenbang Wu, Dinesh Agarwal, and Jimeng Sun. Medclip: Contrastive learning from unpaired medical images and text, 2022. 2

    2022

  26. [34]

    Metransformer: Radiology report generation by transformer with multiple learnable expert tokens

    Zhanyu Wang, Lingqiao Liu, Lei Wang, and Luping Zhou. Metransformer: Radiology report generation by transformer with multiple learnable expert tokens. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11558–11567, 2023. 2, 7

    2023

  27. [35]

    Medklip: Medical knowledge enhanced language-image pre-training in radiology, 2023

    Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. Medklip: Medical knowledge enhanced language-image pre-training in radiology, 2023. 2, 6

    2023

  28. [36]

    Clinical-bert: Vision-language pre-training for radiograph diagnosis and reports generation

    Bin Yan and Mingtao Pei. Clinical-bert: Vision-language pre-training for radiograph diagnosis and reports generation. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 2982–2990, 2022. 2, 7

    2022

  29. [37]

    Knowledge matters: Chest radiology report genera- tion with general and specific knowledge

    Shuxin Yang, Xian Wu, Shen Ge, S Kevin Zhou, and Li Xiao. Knowledge matters: Chest radiology report genera- tion with general and specific knowledge. Medical image analysis, 80:102510, 2022. 7

    2022

  30. [38]

    Radiology report generation with a learned knowledge base and multi-modal alignment

    Shuxin Yang, Xian Wu, Shen Ge, Zhuozhao Zheng, S Kevin Zhou, and Li Xiao. Radiology report generation with a learned knowledge base and multi-modal alignment. Med- ical Image Analysis, 86:102798, 2023. 7

    2023

  31. [39]

    Improving hyperbolic representations via gromov- wasserstein regularization

    Yifei Yang, Wonjun Lee, Dongmian Zou, and Gilad Ler- man. Improving hyperbolic representations via gromov- wasserstein regularization. InEuropean Conference on Com- puter Vision, pages 211–227. Springer, 2024. 4

    2024

  32. [40]

    Otseg: Multi-prompt sinkhorn attention for zero-shot semantic segmentation

    Jong Chul Ye, Yujin Oh, et al. Otseg: Multi-prompt sinkhorn attention for zero-shot semantic segmentation. In The 18th European Conference on Computer Vision, ECCV 2024. Eu- ropean Computer Vision Association (ECV A), 2024. 5

    2024

  33. [41]

    Coca: Contrastive captioners are image-text foundation models

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mo- jtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917, 2022. 1, 2

    2022 arXiv

  34. [42]

    Anatomy-guided weakly- supervised abnormality localization in chest x-rays

    Ke Yu, Shantanu Ghosh, Zhexiong Liu, Christopher Deible, and Kayhan Batmanghelich. Anatomy-guided weakly- supervised abnormality localization in chest x-rays. In In- ternational Conference on Medical Image Computing and Computer-Assisted Intervention , pages 658–668. Springer,

  35. [43]

    Sam-guided enhanced fine-grained encoding with mixed semantic learning for medical image captioning

    Zhenyu Zhang, Benlu Wang, Weijie Liang, Yizhi Li, Xuechen Guo, Guanhong Wang, Shiyan Li, and Gaoang Wang. Sam-guided enhanced fine-grained encoding with mixed semantic learning for medical image captioning. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.