Pith. sign in

REVIEW 4 major objections 5 minor 59 references

AI Sees X-ray Details Without Paying the Token Tax

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-09 23:07 UTC pith:J2E5AHJ7

load-bearing objection Solid high-res RRG framework with one missing ablation control the 4 major comments →

arxiv 2607.06909 v1 pith:J2E5AHJ7 submitted 2026-07-08 cs.CV

Seeing What Matters: Lesion-Aware High-Resolution Patch Discovery and Fusion for Chest X-ray Report Generation

Yingshu Li , Yunyi Liu , Zhenghao Chen , Tong Chen , Zailong Chen , Lingqiao Liu , Lei Wang , Luping Zhou This is my paper
classification cs.CV
keywords radiology report generationhigh-resolution perceptionvisual token budgetspatial resolution allocationchest X-raymultimodal large language model
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Radiology report generation models typically downsample chest X-rays to low resolutions (e.g., 256×256), losing subtle pathological cues like faint nodules or fine vascular markings. Naively increasing resolution by tiling the image into patches inflates the number of visual tokens the language model must process, creating prohibitive computational cost. This paper proposes LePaX, a framework that mimics how radiologists work: first surveying the whole image globally to find suspicious regions, then zooming into those specific areas at high resolution. LePaX learns a spatial utility map that predicts which regions deserve high-resolution processing, extracts a small number of high-res patches (up to 1920×1920) from those locations, and then fuses the detailed patch features back into the global feature grid at their original spatial coordinates. This write-back mechanism enriches the global representation with fine-grained diagnostic evidence without adding any new tokens to the visual budget. The result is a system that processes native-resolution details using the same fixed token count as a low-resolution baseline, achieving consistent improvements across linguistic and clinical metrics on three benchmarks.

Core claim

The central mechanism is a two-stage process where high-resolution perception is formulated as a constrained spatial resolution allocation problem. A lightweight policy head (LSRA) learns to predict which grid locations on a low-resolution global feature map carry the most diagnostic utility, then extracts high-resolution patches from those locations in the native image. A fusion module (GRF) performs spatially grounded resolution write-back, replacing the corresponding tokens in the global feature grid with enriched versions that incorporate high-resolution regional evidence via cross-attention. This allows the model to see fine-grained details in diagnostically relevant regions while the L

What carries the argument

Learnable Spatial Resolution Allocation (LSRA) and Global-Regional Fusion (GRF)

Load-bearing premise

The framework assumes that Grad-CAM heatmaps from a pretrained disease classifier provide meaningful spatial guidance for where high-resolution patches should be extracted during training. If the classifier's saliency maps are poorly aligned with the subtle lesions that high-resolution processing is designed to capture, the allocation policy could be guided toward classifier-salient but report-irrelevant regions.

What would settle it

If replacing Grad-CAM priors with random spatial priors during training yields equivalent final performance, the contribution of the learnable allocation policy is indistinguishable from the fusion mechanism alone.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the constrained allocation formulation is sound, the same principle could extend to other medical imaging modalities where abnormalities are spatially sparse, such as pathology slides or retinal scans, avoiding the need to process gigapixel images uniformly.
  • The token-preserving write-back mechanism could be applied to general vision-language tasks where fine-grained detail in specific regions matters (e.g., document understanding, satellite imagery), allowing high-resolution reasoning without quadratic attention costs.
  • The framework's reliance on weak localization priors (Grad-CAM) during training but not inference suggests a path toward self-supervised region discovery, where the report generation objective alone learns to identify diagnostically relevant areas.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The Grad-CAM prior serves as a training stabilizer for the allocation policy, but the ablation showing report-only supervision still yields B-4 of 0.246 (vs. 0.253 with the prior) suggests the policy can learn meaningful allocation from the generation objective alone, raising the question of whether stronger generation signals could eventually replace external classifier priors entirely.
  • The fixed token budget constraint is a design choice tied to pretrained encoder architectures; if future vision encoders natively support variable-length or hierarchical token sequences, the allocation formulation could shift from selecting discrete patches to learning continuous resolution density functions over the image.
  • The 10× token reduction claim is relative to naive tiling at the same resolution, which is a weak baseline; a stronger comparison would be against dynamic-resolution approaches that use learned token merging or adaptive tiling, though the paper does not benchmark against those.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes LePaX, a radiology report generation (RRG) framework that enables high-resolution (up to 1920x1920) chest X-ray perception without increasing the visual token budget. It introduces two components: Learnable Spatial Resolution Allocation (LSRA), which learns to select diagnostically relevant regions for high-resolution patch extraction, and Global-Regional Fusion (GRF), which writes high-resolution regional features back into the global feature grid via cross-attention. The method is evaluated on MIMIC-CXR, IU-Xray, and CheXpertPlus, showing improvements in both NLG and clinical metrics over prior baselines.

Significance. The problem addressed is well-motivated: standard RRG pipelines downsample CXRs to low resolutions (e.g., 256x256), potentially suppressing subtle lesions. The formulation of high-resolution perception as a constrained spatial resolution allocation problem is a principled and novel contribution to the RRG field. The framework is largely self-contained, and the use of a Grad-CAM prior from an external classifier to regularize the allocation policy during training, while dropping it at inference, is a practical design choice. The reported efficiency gains (maintaining a fixed 128-token budget while processing 1920x1920 images) are significant for the deployment of MLLMs in medical imaging.

major comments (4)
  1. The central claim that gains are attributable to 'high-resolution perception' is not fully isolated by the ablation study. In Table 5, introducing GRF with high-resolution patches improves B-4 from 0.237 to 0.243. However, GRF inherently adds a Transformer cross-attention block (Eq. 11), increasing model capacity. The manuscript lacks a critical control: applying the GRF mechanism to low-resolution patches (e.g., upscaled crops from the low-res global image) to determine if the performance gain comes from the high-resolution patch content or simply from the additional attention mechanism. Without this control, the paper cannot definitively distinguish 'high-resolution perception helps' from 'an extra attention block helps.' The authors should add this ablation to substantiate the core claim.
  2. Table 1 compares LePaX (4B parameters) against baselines with varying model sizes, including 7B and 14.2B models. While LePaX achieves superior performance, the comparison is confounded by parameter count. The manuscript does not discuss whether the baselines were re-implemented or evaluated under the same parameter budget, nor does it normalize for model size. The authors should clarify if the 4B model size is a deliberate architectural constraint or a limitation, and ideally provide a comparison against a baseline of equivalent size to ensure fair assessment.
  3. The '10x fewer visual tokens' claim in the abstract requires explicit validation in the experiments. Table 6 shows that Uniform-1024 uses 1280 tokens and Ours-1920 uses 128 tokens, which is indeed a 10x reduction. However, the abstract frames this as a comparison against 'naive high-res tiling.' The manuscript should explicitly state the resolution and tiling configuration of the naive baseline used for this 10x calculation in the main text (e.g., in Section 4.4) rather than leaving it to be inferred from Table 6.
  4. The weakest assumption identified is that the Grad-CAM spatial prior from a ResNet-34 classifier provides meaningful guidance for the allocation policy. Table 5 shows that report-only supervision yields B-4 of 0.246, while adding policy supervision (including Grad-CAM) yields 0.253. This suggests the Grad-CAM prior is load-bearing. However, the manuscript does not evaluate the quality of these Grad-CAM maps or their alignment with the subtle lesions that high-resolution processing is designed to capture. If the classifier's saliency maps are poorly aligned with report-relevant regions, the policy could be misguided. The authors should provide a qualitative or quantitative analysis of the Grad-CAM prior quality, or at least discuss this risk in Section 3.3.
minor comments (5)
  1. Section 3.1, Eq. (1): The notation uses non-standard characters (e.g., 'F“ϕ imgpXq PR NvˆDv'). This appears to be a rendering issue, but it should be corrected to standard LaTeX/math notation for clarity.
  2. Section 3.3, Eq. (6): The Grad-CAM formula uses 'Ca' and 'H0W0' which are not defined in the surrounding text. Please define these variables (e.g., number of channels, spatial dimensions).
  3. Table 2: The 'Ours' row reports BLEU-4 of 0.138, which is significantly lower than the BLEU-4 reported in Table 1 for MIMIC-CXR (0.253). While this is likely due to the different dataset (CheXpertPlus) and benchmark setup (CXPMRG-Bench), a brief footnote or note in the table caption explaining the discrepancy would aid the reader.
  4. Figure 2: The text in the figure is quite small and difficult to read. Consider enlarging the font sizes or simplifying the diagram for better legibility in the final version.
  5. Section 4.4, Table 5: The row 'GRF + LSRA (report-only)' reports B-4 of 0.246, which is lower than 'GRF + Grad-CAM' (0.251). This suggests that LSRA with report-only supervision performs worse than a simple Grad-CAM-based selection. The authors should clarify whether LSRA without policy supervision is detrimental or simply unhelpful compared to Grad-CAM.

Circularity Check

0 steps flagged

No significant circularity: the framework is self-contained against external benchmarks with no self-citation chain or fitted-input-as-prediction pattern.

full rationale

The paper's derivation chain is self-contained. The Grad-CAM spatial prior (Eq. 6-7) comes from an external ResNet-34 disease classifier, not from the paper's own results, and is used only as a training regularizer—not at inference. The report generation loss (Eq. 13) is standard cross-entropy. The benchmarks (MIMIC-CXR, IU-Xray, CheXpertPlus) are external public datasets. No 'prediction' or 'first-principles result' reduces to a fitted parameter by construction. The LSRA allocation policy is learned end-to-end and evaluated on held-out test sets. The GRF fusion mechanism (Eq. 10-11) is a standard cross-attention block whose gains are measured against external baselines. While the ablation in Table 5 does not fully isolate whether gains come from high-resolution content versus the additional attention mechanism (a correctness concern, not a circularity concern), this does not constitute circularity in the derivation chain. The only minor self-referential element is that the framework's components (LSRA, GRF) are jointly optimized, but this is standard end-to-end training, not a circular definition. No step in the paper's logical chain reduces to its inputs by construction, self-citation, or definitional equivalence.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 2 invented entities

The framework introduces two learned components (LSRA utility map, GRF module) with clear purposes and provides ablation evidence for their contributions. The free parameters are mostly standard or ablation-selected. The key domain assumptions (downsampling hurts, Grad-CAM is a useful prior) are plausible but not independently verified within the paper.

free parameters (6)
  • patch budget K (Top-K) = 5
    Selected via ablation (Fig. 3b); K=5 yields best B-4/METEOR/ROUGE on MIMIC-CXR.
  • patch size (p_w, p_h) = 384
    Fixed at 384x384 pixels; stated in Eq. 9 but not justified against alternatives.
  • minimum grid distance d_min (NMS) = unspecified
    Used in Eq. 8 for spatial diversity but exact value not stated.
  • loss weight lambda = unspecified
    Joint loss L = L_RRG + lambda * L_policy (Eq. 13); lambda value not reported.
  • LoRA rank r / alpha = r=32, alpha=60
    Stated in implementation details; standard hyperparameter choices.
  • global feature grid size = 27x27
    Mentioned in passing in implementation details; determines allocation policy resolution.
axioms (4)
  • domain assumption Subtle pathological cues in CXRs are suppressed by aggressive downsampling to 256x256 and are better preserved at native resolution.
    Core motivation (Section 1); supported by clinical literature cited (refs 5, 38) but not independently verified in this paper for the specific resolutions used.
  • domain assumption Grad-CAM activations from a ResNet-34 disease classifier provide a meaningful spatial prior for report-relevant regions.
    Section 3.3; used to regularize LSRA during training. The quality of this prior is assumed, not independently validated.
  • domain assumption A shared vision backbone produces geometrically aligned features between global and patch views.
    Section 3.1; enables GRF's spatial write-back. Plausible but not formally proven.
  • standard math Gumbel-Top-K with straight-through estimator provides sufficient gradient signal for learning the allocation policy.
    Section 3.3; standard relaxation technique, but its effectiveness for this specific discrete selection problem is assumed.
invented entities (2)
  • Spatial utility map U (LSRA) independent evidence
    purpose: Predicts where high-resolution capacity should be allocated on the global feature grid.
    Learned end-to-end and evaluated on held-out test sets; visualized in Fig. 6 showing alignment with Grad-CAM and clinical findings.
  • Global-Regional Fusion (GRF) module independent evidence
    purpose: Injects high-res patch features back into global tokens at corresponding spatial locations without token inflation.
    Ablated in Table 5 showing contribution to performance; feature statistics in Fig. 5b show increased variance/L2 norm after fusion.

pith-pipeline@v1.1.0-glm · 20202 in / 3082 out tokens · 278906 ms · 2026-07-09T23:07:41.385424+00:00 · methodology

0 comments
read the original abstract

Despite rapid advances in chest X-ray (CXR) foundation models, most radiology report generation (RRG) systems still rely on heavily downsampled inputs (e.g., 256x256) due to the fixed visual token budgets of pretrained vision encoders, suppressing subtle yet clinically important cues present in native-resolution images. However, enabling high-resolution (high-res) perception remains challenging: naive tiling causes prohibitive token inflation, while global compression suppresses subtle lesions and degrades diagnostic fidelity. Inspired by radiologists' workflow, localizing suspicious regions before detailed high-res assessment. We propose Lesion-Aware High-Resolution Patch Discovery and Fusion for Chest X-ray Reporting (LePaX), the first RRG framework that enables efficient high-res CXR perception (up to 1920x1920) without increasing the vision-token count. LePaX formulates high-res perception as a constrained spatial resolution allocation problem under a fixed token budget and introduces two key components: Learnable Spatial Resolution Allocation (LSRA), which learns a spatial utility map that adaptively allocates limited high-res capacity to diagnostically relevant regions, enabling targeted extraction of high-res patches from native CXRs; and Global-Regional Fusion (GRF), which performs token-preserving region-to-global refinement by projecting high-resolution regional evidence back onto the global feature grid through spatially aligned resolution write-back, avoiding token inflation. Experiments on multiple CXR benchmarks demonstrate that LePaX consistently improves both clinical and linguistic metrics while enabling native-resolution CXR perception with over 10x fewer visual tokens than naive high-res tiling.

Figures

Figures reproduced from arXiv: 2607.06909 by Lei Wang, Lingqiao Liu, Luping Zhou, Tong Chen, Yingshu Li, Yunyi Liu, Zailong Chen, Zhenghao Chen.

Figure 1
Figure 1. Figure 1: Left: Radiologists typically follow a two-stage workflow: a global survey to localize suspicious regions, followed by focused examination of these areas to verify specific findings (e.g., enlarged heart). Right: Conventional RRG models rely on fixed￾size, low-resolution inputs and lack this global–local reasoning process. Our radiologist￾inspired framework instead extracts and integrates high-resolution re… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed LePaX framework. A lightweight policy head pre￾dicts a spatial utility map from global features to allocate a fixed number of high￾resolution patches under a resolution budget (LSRA). Global and patch features share a vision encoder and are fused via utility-guided Global–Regional Fusion (GRF), which writes high-resolution evidence back to the global feature grid. The fused visual … view at source ↗
Figure 3
Figure 3. Figure 3: Ablation studies on MIMIC-CXR [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison between the baseline and our method on two MIMIC￾CXR cases. The main clinical findings are highlighted in different colors for clarity. processing. In Case I, attention highlights subtle post-surgical structures such as surgical clips, while in Case II it follows the course of the right internal jugular line in the upper mediastinum. These examples show how GRF refines coarse spatial… view at source ↗
Figure 5
Figure 5. Figure 5: Analysis of the proposed Global–Regional Fusion (GRF). (a) Visualization of selected regions and LRÑHR fusion attention. (b) Feature statistics before and after fusion, showing increased variance and L2 norm after GRF [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative comparison of region allocation. Red boxes denote LSRA predic￾tions and green boxes indicate Grad-CAM guidance. LSRA consistently identifies clin￾ically relevant regions similar to Grad-CAM. Cases 2–4 further show that report-level supervision helps the learned policy better localize medical devices (Cases 2–3) and pleural effusion (Case 4). References 1. Abdin, M., Aneja, J., Awadalla, H., Awa… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

59 extracted references · 59 canonical work pages · 11 internal anchors

  1. [1]

    Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

    Abdin, M., Aneja, J., Awadalla, H., Awadallah, A., Awan, A.A., Bach, N., Bahree, A., Bakhtiari, A., Bao, J., Behl, H., et al.: Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  2. [2]

    Advances in neural information processing systems35, 23716– 23736 (2022)

    Alayrac, J.B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Men- sch, A., Millican, K., Reynolds, M., et al.: Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems35, 23716– 23736 (2022)

    work page 2022

  3. [3]

    Qwen2.5-VL Technical Report

    Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., et al.: Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  4. [4]

    In: Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summa- rization

    Banerjee, S., Lavie, A.: Meteor: An automatic metric for mt evaluation with im- proved correlation with human judgments. In: Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summa- rization. pp. 65–72 (2005)

    work page 2005

  5. [5]

    Scientific reports8(1), 8717 (2018)

    Brennan, P.C., Gandomkar, Z., Ekpo, E.U., Tapia, K., Trieu, P.D., Lewis, S.J., Wolfe, J.M., Evans, K.K.: Radiologists can detect the ‘gist’of breast cancer before any overt signs of cancer appear. Scientific reports8(1), 8717 (2018)

    work page 2018

  6. [6]

    In: Proceedings of the 16 Y

    Bu, S., Li, T., Yang, Y., Dai, Z.: Instance-level expert knowledge and aggregate discriminative attention for radiology report generation. In: Proceedings of the 16 Y. Li et al. IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14194– 14204 (2024)

    work page 2024

  7. [7]

    CheXpert Plus: Augmenting a Large Chest X-ray Dataset with Text Radiology Reports, Patient Demographics and Additional Image Formats

    Chambon, P., Delbrouck, J.B., Sounack, T., Huang, S.C., Chen, Z., Varma, M., Truong, S.Q., Chuong, C.T., Langlotz, C.P.: Chexpert plus: Augmenting a large chest x-ray dataset with text radiology reports, patient demographics and addi- tional image formats. arXiv preprint arXiv:2405.19538 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  8. [8]

    IEEE Transactions on Medical Imaging (2025)

    Chen, Z., Li, Y., Wang, Z., Gao, P., Barthelemy, J., Zhou, L., Wang, L.: Enhancing radiology report generation via multi-phased supervision. IEEE Transactions on Medical Imaging (2025)

    work page 2025

  9. [9]

    Chen, Z., Wang, W., Tian, H., Ye, S., Gao, Z., Cui, E., Tong, W., Hu, K., Luo, J., Ma, Z., et al.: How far are we to gpt-4v? closing the gap to commercial multimodal modelswith open-source suites.ScienceChina InformationSciences67(12),220101 (2024)

    work page 2024

  10. [10]

    Cross-modal Memory Networks for Radiology Report Generation

    Chen, Z., Shen, Y., Song, Y., Wan, X.: Cross-modal memory networks for radiology report generation. arXiv preprint arXiv:2204.13258 (2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  11. [11]

    Generating Radiology Reports via Memory-driven Transformer

    Chen, Z., Song, Y., Chang, T.H., Wan, X.: Generating radiology reports via memory-driven transformer. arXiv preprint arXiv:2010.16056 (2020)

    work page internal anchor Pith review Pith/arXiv arXiv 2010

  12. [12]

    Journal of the American Medical Informatics Association23(2), 304–310 (2016)

    Demner-Fushman, D., Kohli, M.D., Rosenman, M.B., Shooshan, S.E., Rodriguez, L., Antani, S., Thoma, G.R., McDonald, C.J.: Preparing a collection of radiol- ogy examinations for distribution and retrieval. Journal of the American Medical Informatics Association23(2), 304–310 (2016)

    work page 2016

  13. [13]

    Scientific reports11(1), 20122 (2021)

    Gandomkar, Z., Siviengphanom, S., Ekpo, E.U., Suleiman, M., Taba, S.T., Li, T., Xu, D., Evans, K.K., Lewis, S.J., Wolfe, J.M., et al.: Global processing pro- vides malignancy evidence complementary to the information captured by humans or machines following detailed mammogram inspection. Scientific reports11(1), 20122 (2021)

    work page 2021

  14. [14]

    He,K.,Zhang,X.,Ren,S.,Sun,J.:Deepresiduallearningforimagerecognition.In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)

    work page 2016

  15. [15]

    ORGAN: Observation-Guided Radiology Report Generation via Tree Reasoning

    Hou, W., Xu, K., Cheng, Y., Li, W., Liu, J.: Organ: observation-guided radiology report generation via tree reasoning. arXiv preprint arXiv:2306.06466 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  16. [16]

    ICLR1(2), 3 (2022)

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: Lora: Low-rank adaptation of large language models. ICLR1(2), 3 (2022)

    work page 2022

  17. [17]

    Huang, X., Chen, W., Liu, J., Lu, Q., Luo, X., Shen, L.: Damper: A dual-stage medical report generation framework with coarse-grained mesh alignment and fine- grainedhypergraphmatching.In:ProceedingsoftheAAAIConferenceonArtificial Intelligence. vol. 39, pp. 3769–3778 (2025)

    work page 2025

  18. [18]

    In: Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition

    Huang, Z., Zhang, X., Zhang, S.: Kiut: Knowledge-injected u-transformer for ra- diology report generation. In: Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition. pp. 19809–19818 (2023)

    work page 2023

  19. [19]

    In: Proceedings of the AAAI conference on artificial intelligence

    Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al.: Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI conference on artificial intelligence. vol. 33, pp. 590–597 (2019)

    work page 2019

  20. [20]

    RadGraph: Extracting Clinical Entities and Relations from Radiology Reports

    Jain, S., Agrawal, A., Saporta, A., Truong, S.Q., Duong, D.N., Bui, T., Chambon, P., Zhang, Y., Lungren, M.P., Ng, A.Y., et al.: Radgraph: Extracting clinical enti- ties and relations from radiology reports. arXiv preprint arXiv:2106.14463 (2021)

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  21. [21]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Jin, H., Che, H., Lin, Y., Chen, H.: Promptmrg: Diagnosis-driven prompts for medical report generation. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 2607–2615 (2024) LePaX: Lesion-Aware High-Resolution Patch Discovery and Fusion 17

    work page 2024

  22. [22]

    MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs

    Johnson, A.E., Pollard, T.J., Greenbaum, N.R., Lungren, M.P., Deng, C.y., Peng, Y., Lu, Z., Mark, R.G., Berkowitz, S.J., Horng, S.: Mimic-cxr-jpg, a large publicly available database of labeled chest radiographs. arXiv preprint arXiv:1901.07042 (2019)

    work page internal anchor Pith review Pith/arXiv arXiv 1901

  23. [23]

    Radiology 242(2), 396–402 (2007)

    Kundel, H.L., Nodine, C.F., Conant, E.F., Weinstein, S.P.: Holistic component of image perception in mammogram interpretation: gaze-tracking study. Radiology 242(2), 396–402 (2007)

    work page 2007

  24. [24]

    Transactions on Machine Learning Research (2024)

    Li, B., Zhang, Y., Guo, D., Zhang, R., Li, F., Zhang, H., Zhang, K., Zhang, P., Li, Y., Liu, Z., et al.: Llava-onevision: Easy visual task transfer. Transactions on Machine Learning Research (2024)

    work page 2024

  25. [25]

    Advances in Neural Information Processing Systems36, 28541–28564 (2023)

    Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., Gao, J.: Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems36, 28541–28564 (2023)

    work page 2023

  26. [26]

    Li, M., Lin, B., Chen, Z., Lin, H., Liang, X., Chang, X.: Dynamic graph en- hancedcontrastivelearningforchestx-rayreportgeneration.In:Proceedingsofthe IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3334– 3343 (2023)

    work page 2023

  27. [27]

    medRxiv pp

    Li, Y., Liu, Y., Wang, Z., Liang, X., Liu, L., Wang, L., Cui, L., Tu, Z., Wang, L., Zhou, L.: A comprehensive study of gpt-4v’s multimodal capabilities in medical imaging. medRxiv pp. 2023–11 (2023)

    work page 2023

  28. [28]

    In: Interna- tional Conference on Medical Image Computing and Computer-Assisted Interven- tion

    Li, Y., Wang, Z., Liu, Y., Wang, L., Liu, L., Zhou, L.: Kargen: Knowledge-enhanced automated radiology report generation using large language models. In: Interna- tional Conference on Medical Image Computing and Computer-Assisted Interven- tion. pp. 382–392. Springer (2024)

    work page 2024

  29. [29]

    In: Text sum- marization branches out

    Lin, C.Y.: Rouge: A package for automatic evaluation of summaries. In: Text sum- marization branches out. pp. 74–81 (2004)

    work page 2004

  30. [30]

    IEEE Transactions on Medical Imaging (2024)

    Liu,A.,Guo,Y.,Yong,J.h.,Xu,F.: Multi-grainedradiologyreportgenerationwith sentence-level image-language contrastive learning. IEEE Transactions on Medical Imaging (2024)

    work page 2024

  31. [31]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Liu, C., Tian, Y., Chen, W., Song, Y., Zhang, Y.: Bootstrapping large language models for radiology report generation. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 18635–18643 (2024)

    work page 2024

  32. [32]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Liu, Y., Li, Y., Chen, T., Liu, L., Wang, L., Zhou, L.: Sat-rrg: Llm-guided self- adaptive training for radiology report generation with token-level push-pull opti- mization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 35363–35372 (2026)

    work page 2026

  33. [33]

    Artificial intelligence in medicine144, 102633 (2023)

    Nicolson, A., Dowling, J., Koopman, B.: Improving chest x-ray report generation by leveraging warm starting. Artificial intelligence in medicine144, 102633 (2023)

    work page 2023

  34. [34]

    In: Findings of the Association for Computational Linguistics: EMNLP 2024

    Ostmeier, S., Xu, J., Chen, Z., Varma, M., Blankemeier, L., Bluethgen, C., Md, A., Moseley,M.,Langlotz,C.,Chaudhari,A.,etal.:Green:Generativeradiologyreport evaluation and error notation. In: Findings of the Association for Computational Linguistics: EMNLP 2024. pp. 374–390 (2024)

    work page 2024

  35. [35]

    In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics

    Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics. pp. 311–318 (2002)

    work page 2002

  36. [36]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Park, S.J., Heo, K.S., Shin, D.H., Son, Y.H., Oh, J.H., Kam, T.E.: Dart: Disease- aware image-text alignment and self-correcting re-alignment for trustworthy radiol- ogy report generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 15580–15589 (June 2025) 18 Y. Li et al

    work page 2025

  37. [37]

    RaDialog: A Large Vision-Language Model for Radiology Report Generation and Conversational Assistance

    Pellegrini, C., Özsoy, E., Busam, B., Navab, N., Keicher, M.: Radialog: A large vision-language model for radiology report generation and conversational assis- tance. arXiv preprint arXiv:2311.18681 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  38. [38]

    The Lancet Digital Health3(8), e496– e506 (2021)

    Seah, J.C., Tang, C.H., Buchlak, Q.D., Holt, X.G., Wardman, J.B., Aimoldin, A., Esmaili, N., Ahmad, H., Pham, H., Lambert, J.F., et al.: Effect of a comprehensive deep-learning model on the accuracy of chest x-ray interpretation by radiologists: a retrospective, multireader multicase study. The Lancet Digital Health3(8), e496– e506 (2021)

    work page 2021

  39. [39]

    In: Proceedings of the IEEE international conference on computer vision

    Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad- cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision. pp. 618–626 (2017)

    work page 2017

  40. [40]

    In: International Conference on Medical Im- age Computing and Computer-Assisted Intervention

    Sha, Y., Pan, H., Meng, W., Li, K.: Contrastive knowledge-guided large language models for medical report generation. In: International Conference on Medical Im- age Computing and Computer-Assisted Intervention. pp. 111–120. Springer (2025)

    work page 2025

  41. [41]

    Advances in Neural Information Processing Systems (2017)

    Vaswani, A.: Attention is all you need. Advances in Neural Information Processing Systems (2017)

    work page 2017

  42. [42]

    IEEE Journal of Biomed- ical and Health Informatics28(4), 2199–2210 (2024)

    Wang, J., Bhalerao, A., Yin, T., See, S., He, Y.: Camanet: class activation map guided attention network for radiology report generation. IEEE Journal of Biomed- ical and Health Informatics28(4), 2199–2210 (2024)

    work page 2024

  43. [43]

    arXiv preprint arXiv:2408.09743 (2024)

    Wang, X., Li, Y., Wang, F., Wang, S., Li, C., Jiang, B.: R2gencsr: Retrieving context samples for large language model based x-ray medical report generation. arXiv preprint arXiv:2408.09743 (2024)

    work page arXiv 2024

  44. [44]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Wang, X., Wang, F., Li, Y., Ma, Q., Wang, S., Jiang, B., Tang, J.: Cxpmrg-bench: Pre-training and benchmarking for x-ray medical report generation on chexpert plus dataset. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 5123–5133 (2025)

    work page 2025

  45. [45]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wang, Z., Liu, L., Wang, L., Zhou, L.: Metransformer: Radiology report genera- tion by transformer with multiple learnable expert tokens. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11558– 11567 (2023)

    work page 2023

  46. [46]

    Meta-Radiology1(3), 100033 (2023)

    Wang, Z., Liu, L., Wang, L., Zhou, L.: R2gengpt: Radiology report generation with frozen llms. Meta-Radiology1(3), 100033 (2023)

    work page 2023

  47. [47]

    In: Conference on Health, Inference, and Learning

    Wu, Y., Huang, I.C., Huang, X.: Token imbalance adaptation for radiology report generation. In: Conference on Health, Inference, and Learning. pp. 72–85. PMLR (2023)

    work page 2023

  48. [48]

    arXiv preprint arXiv:2408.08872 (2024)

    Xue, L., Shu, M., Awadalla, A., Wang, J., Yan, A., Purushwalkam, S., Zhou, H., Prabhu, V., Dai, Y., Ryoo, M.S., et al.: xgen-mm (blip-3): A family of open large multimodal models. arXiv preprint arXiv:2408.08872 (2024)

    work page arXiv 2024

  49. [49]

    Medical Image Analysis86, 102798 (2023)

    Yang, S., Wu, X., Ge, S., Zheng, Z., Zhou, S.K., Xiao, L.: Radiology report gen- eration with a learned knowledge base and multi-modal alignment. Medical Image Analysis86, 102798 (2023)

    work page 2023

  50. [50]

    IEEE Transactions on Medical Imaging43(11), 4017–4028 (2024)

    Yang, Y., Yu, J., Fu, Z., Zhang, K., Yu, T., Wang, X., Jiang, H., Lv, J., Huang, Q., Han, W.: Token-mixer: Bind image and text in one embedding space for medical image reporting. IEEE Transactions on Medical Imaging43(11), 4017–4028 (2024)

    work page 2024

  51. [51]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Ye, Q., Xu, H., Ye, J., Yan, M., Hu, A., Liu, H., Qian, Q., Zhang, J., Huang, F.: mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13040–13051 (2024) LePaX: Lesion-Aware High-Resolution Patch Discovery and Fusion 19

    work page 2024

  52. [52]

    Patterns4(9) (2023)

    Yu, F., Endo, M., Krishnan, R., Pan, I., Tsai, A., Reis, E.P., Fonseca, E.K.U.N., Lee, H.M.H., Abad, Z.S.H., Ng, A.Y., et al.: Evaluating progress in automatic chest x-ray radiology report generation. Patterns4(9) (2023)

    work page 2023

  53. [53]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Zhai, X., Mustafa, B., Kolesnikov, A., Beyer, L.: Sigmoid loss for language im- age pre-training. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 11975–11986 (2023)

    work page 2023

  54. [54]

    BERTScore: Evaluating Text Generation with BERT

    Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., Artzi, Y.: Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675 (2019)

    work page internal anchor Pith review Pith/arXiv arXiv 1904

  55. [55]

    In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing

    Zhao,W.,Wu,C.,Zhang,X.,Zhang,Y.,Wang,Y.,Xie,W.:Ratescore:Ametricfor radiology report generation. In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. pp. 15004–15019 (2024)

    work page 2024

  56. [56]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Zhou, Q., Liang, G., Li, X., Chen, J., Wang, Z., Yao, C., Wu, S.: Learnable re- trieval enhanced visual-text alignment and fusion for radiology report generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 22529–22538 (2025)

    work page 2025

  57. [57]

    arXiv preprint arXiv:2510.12444 (2025)

    Zhou, S., Li, Y., Liu, Y., Liu, L., Wang, L., Zhou, L.: A review of longitudinal radiology report generation: Dataset composition, methods, and performance eval- uation. arXiv preprint arXiv:2510.12444 (2025)

    work page arXiv 2025

  58. [58]

    MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models

    Zhu, D., Chen, J., Shen, X., Li, X., Elhoseiny, M.: Minigpt-4: Enhancing vision- language understanding with advanced large language models. arXiv preprint arXiv:2304.10592 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  59. [59]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention

    Zhu, Q., Mathai, T.S., Mukherjee, P., Peng, Y., Summers, R.M., Lu, Z.: Utilizing longitudinal chest x-rays and reports to pre-fill radiology reports. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 189–198. Springer (2023)

    work page 2023