Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Large Language Model with Region-guided Referring and Grounding for CT Report Generation

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A region-guided framework for CT report generation claims to outperform global-feature-only methods by grounding each report section in the anatomical region it describes.

desk verdict Useful region-guided CT report generator, but the SOTA claim is undercut by an asymmetric baseline comparison that misses the most relevant 3D methods. read the letter →

arxiv 2411.15539 v2 pith:DKJYXIJG submitted 2024-11-23 cs.CV cs.AI

Zhixuan Chen , Yequan Bie , Haibo Jin , Hao Chen This is my paper · ORCID
classification cs.CVcs.AI
keywords CTreportgenerationreferringandgroundingregion-levelunderstandingchestlargelanguagemodellocalfeaturedecouplingregion-reportalignmentinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes Reg2RG, a framework for chest CT report generation that focuses on individual anatomical regions rather than only the whole volume. It uses a universal segmentation module to carve out each region, decouples the region's texture from its geometry, and feeds both, together with global context, into a large language model. A region-report alignment training step makes the model first name the region it is looking at and then write that region's findings, so each report section is explicitly grounded in the image region it describes. The paper reports that this region-guided design outperforms global-feature-only baselines on standard language metrics and on abnormality-detection precision, recall, and F1, with the grounding step doubling as an interpretability check.

What carries the argument

The central object is the decoupled region feature pair: texture features $L^t_{A_j}=f_A(f_V(\mathrm{Crop}(M_{A_j}\odot V)))$ and geometry features $L^g_{A_j}=f_P(f_M(M_{A_j}))$, concatenated into $L_{A_j}$. The mask $M_{A_j}$ from the universal segmentation module defines the referring region; texture preserves high-resolution details of the cropped masked volume, and geometry preserves the original size and position from the uncropped mask. These local features are shuffled and injected into the LLM prompt with a recognition prefix, so the model must identify the region before writing its report. The region-report alignment (RRA) training objective is the language-modeling loss over this restructured sequence, and this is what carries the claimed referring and grounding behavior.

What would settle it

Corrupt the input masks on a held-out set—swap the lung and pleura masks, erode or dilate them, or replace them with oracle ground-truth masks—and measure region recognition accuracy and clinical F1; if performance does not degrade substantially, the mask dependency is weaker than claimed, and if oracle masks do not improve results, the local features are not carrying the diagnostic signal.

Watch

Extended reading notes

Core claim

Reg2RG's central claim is that CT report generation should be treated as a referring-and-grounding task over anatomical regions, not a volume-to-text mapping. The paper argues that global volume features alone cause decoders to miss abnormalities, and that region-level local features—texture from cropped masked volumes plus geometry from the uncropped mask—give the model the information needed to diagnose each region. The region-report alignment strategy forces the LLM to recognize the anatomical area from the local features before generating that region's report, and shuffling region order prevents the model from relying on a fixed sequence. On two large chest CT datasets, the paper reports that Reg2RG outperforms prior 3D and 2D methods on BLEU, METEOR, and ROUGE-L, and on the region-grounded dataset it also achieves higher precision, recall, and F1 for abnormality labels, with region recognition serving as an early-stage validation of which anatomy each report is grounded in.

Load-bearing premise

The load-bearing premise is that the segmentation module's masks are accurate enough to define the regions that drive diagnosis, because every local feature is built by masking and cropping the volume; a wrong or incomplete mask grounds the report in the wrong anatomy and the claimed interpretability collapses.

Editorial extensions

If this is right

  • If the central claim holds, CT report generators no longer need to encode every diagnostic cue into a single global embedding; region-level detail can be delegated to local feature extractors.
  • Region recognition before report generation gives a built-in check: a report section is only trustworthy if the model correctly names the region it refers to.
  • The decoupling strategy shows that high-resolution texture and mask-derived geometry can be preserved without a large increase in compute, making region-level report generation practical for 3D volumes.
  • Explicit grounding of each finding to an anatomical region makes the generated reports more useful to radiologists, because the visual basis of each statement can be inspected.
  • The reported gains in clinical efficacy suggest that region-guided features matter for abnormality detection even when the language decoder is held fixed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If masks are the bottleneck, improving the segmentation module or learning region proposals jointly should directly raise region recognition and clinical F1, especially for lung and pleura where the paper's own Table III shows F1 around 0.44.
  • The shuffling mechanism implies the model learns region identity from local features rather than from report text, so the approach could transfer to datasets without region-grounded reports by using a text-to-region splitter to create supervision.
  • A natural extension is to move from organ-level regions to lesion-level regions; the paper's limitation section notes that organ-level features miss or mislocate lesions, making lesion segmentation the next testbed.
  • If clinical efficacy gains persist under mask corruption, the framework would be robust enough for deployment with imperfect universal segmenters; if not, mask quality becomes the main cost driver of the whole pipeline.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Reg2RG, a region-guided framework for CT report generation. It uses a universal segmentation module to extract masks for anatomical regions, decouples local features into texture (mask-cropped, high-resolution) and geometry (uncropped mask encoding), and injects them alongside global features into an LLM decoder. A region-report alignment (RRA) training strategy adds a prefix predicting the region name before each region-specific report and shuffles the order of local features during training. Experiments on RadGenome-ChestCT and CTRG-Chest-548K report NLG and clinical-efficacy metrics, ablations, region recognition, and qualitative examples that support the claim that region-level grounding improves diagnostic accuracy.

Significance. If the results are reproducible, the paper makes a useful contribution by demonstrating that region-level features can improve CT report generation over global-only baselines, and the RRA strategy provides a concrete type of interpretability. The code release and use of public datasets are strengths. However, the significance is currently limited because the most relevant 3D CT baselines are not included in the comparison, the comparison to baselines is asymmetric with respect to mask supervision, and no statistical significance is reported. The interpretability claim is also weakened by low region-recognition accuracy for lung and pleura.

major comments (4)
  1. [Section IV-C, Tables I and II] The SOTA comparison omits Dia-LLaMA [20] and HILT [9], both described in Section II-A as recent 3D CT report generation methods. Since HILT in particular targets high-resolution volume encoding and is directly related to the LFD claim, its absence means the 'outperforms several state-of-the-art methods' claim in the Abstract is not yet established against the most relevant baselines. Please include these models or provide a justified reason for their exclusion.
  2. [Section IV-D, Table VI] The ablation does not control for the additional mask supervision. Settings (a)-(c) all use mask-derived local features, while the baseline (RadFM) receives only the global volume. The geometry features Lg_Aj in Eq. (4) are pure mask encodings, and the texture features in Eq. (3) are mask-cropped volumes. A simple baseline that injects the segmentation masks (or a global feature concatenated with a mask summary) into the same LLM decoder is needed to attribute the gains to the LFD/RRA mechanisms rather than to the extra mask information.
  3. [Section IV-C, Tables I and II] No error bars, multiple seeds, or significance tests are reported. On CTRG-Chest-548K, the differences over the second-best method are small (e.g., BLEU-4 32.04 vs 30.89 for RadFM) and ROUGE-L is actually lower than several baselines (47.76 vs 50.93 for R2GenGPT). The paper should report variance across at least 3 seeds or a significance test for the headline claims.
  4. [Section IV-C.3 and Table III] The region recognition F1 for lung and pleura is about 0.44, yet Table IV shows lung is among the most frequently reported regions (1514 of 1564 test reports). This means the claimed interpretability via grounding is unreliable for a major organ. The paper should quantify how often reports for lung/pleura are grounded in the correct region, and discuss whether the interpretability benefit actually holds for these common regions.
minor comments (6)
  1. [Section IV-A/B and Table I] The dataset name is inconsistent: Section IV-A refers to 'CTRG-Chest-548K' while Section IV-B, Table I, and other places use 'CTRG-Chest-584K'; please unify.
  2. [Figure 1 caption area] The text contains a placeholder 'Toy firgure' near the abstract/first page; this should be removed.
  3. [Tables V and VI captions] The captions contain the typo 'Metrcis' instead of 'Metrics'; please correct.
  4. [Abstract and Section III-D] The acronym 'RRA' is used in the abstract but is not defined until Section III-D; define it at first use.
  5. [Reference [33]] The text cites Qwen2.5-14B but reference [33] is titled 'Qwen2 technical report'; please update the citation to the correct Qwen2.5 technical report or correct the model name.
  6. [Figure 2] Figure 2 is dense and the shuffle mechanism is hard to follow; a higher-resolution figure or a step-by-step schematic would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims rest on held-out benchmark evaluations and ablations, not on fitted parameters relabeled as predictions or on load-bearing self-citations.

full rationale

This is an empirical systems paper rather than a derivation. The central claim of improved CT report generation is supported by training on public datasets and evaluating on held-out test splits, with baselines initialized and fine-tuned under the same protocol. The local features in Eq. (3) and Eq. (4) are constructed from segmentation masks, but they are inputs to a supervised language-modeling objective (Eq. (9)) optimized over training data; no fitted constant is later renamed as a prediction. The region-recognition evaluation in Table III tests the model's ability to predict the area-name prefix on unseen volumes, which is ordinary supervised generalization rather than a self-defined or statistically forced quantity. The RRA strategy is ablated in Table VII, and the LFD and global-local collaboration effects are ablated in Tables V and VI, so the contributions are empirically isolated rather than assumed. The only self-citation is Dia-LLaMA [20] in the related-work discussion, and it is descriptive rather than load-bearing; no uniqueness theorem or central premise is justified solely by the authors' prior work. The paper honestly states in Section V that inaccurate segmentation masks could affect performance and that local features are currently organ-level, which is a dependency on external inputs, not circularity. The absence of Dia-LLaMA and HILT from the comparison tables is a legitimate completeness concern about the SOTA claim, but it does not make the reported results equivalent to the paper's own assumptions. Overall, no step in the claimed chain reduces to its own inputs by construction.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No fitted constants are used to derive a theoretical result; the paper is an empirical system paper. The listed axioms are background assumptions about the segmentation module, dataset construction, evaluation classifier, and training setup that the reported performance depends on.

assumptions (5)
  • domain assumption The universal segmentation model SAT provides sufficiently accurate anatomical masks for all ten regions.
    Local features are constructed by masking and cropping the volume (Eq. 3); Table III shows lung and pleura F1 around 0.44, and Section V acknowledges segmentation failure degrades performance.
  • domain assumption The GPT-4-generated region-level reports in RadGenome-ChestCT are valid ground truth.
    Region reports are generated by GPT-4 plus NER (Sec. IV-A); the paper uses them as training targets and evaluation references without radiologist validation.
  • domain assumption RadBERT abnormality labels extracted from CT-RATE are a valid clinical efficacy measure for GPT-4-style generated reports.
    CE metrics depend on a classifier fine-tuned on CT-RATE; its behavior on GPT-4-generated text is not validated (Sec. IV-A).
  • domain assumption The LLM can learn to recognize anatomical regions from local visual features alone, without relying on positional order.
    The RRA strategy shuffles local features to force region recognition from content (Sec. III-D); if this fails, grounding is not achieved.
  • domain assumption Pretrained RadFM encoder and adapter features are suitable for both global and local inputs.
    The volume encoder and adapter are initialized from RadFM (Sec. IV-B); the ablation baseline is RadFM itself.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Large Language Model with Region-guided Referring and Grounding for CT Report Generation." pith.science (2026). https://pith.science/paper/DKJYXIJG

@misc{pith2026241115539,
  author       = {Pith},
  title        = {Pith review of: Large Language Model with Region-guided Referring and Grounding for CT Report Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DKJYXIJG}},
  note         = {Machine review of arXiv:2411.15539}
}
read the original abstract

Computed tomography (CT) report generation is crucial to assist radiologists in interpreting CT volumes, which can be time-consuming and labor-intensive. Existing methods primarily only consider the global features of the entire volume, making it struggle to focus on specific regions and potentially missing abnormalities. To address this issue, we propose Reg2RG, the first region-guided referring and grounding framework for CT report generation, which enhances diagnostic performance by focusing on anatomical regions within the volume. Specifically, we utilize masks from a universal segmentation module to capture local features for each referring region. A local feature decoupling (LFD) strategy is proposed to preserve the local high-resolution details with little computational overhead. Then the local features are integrated with global features to capture inter-regional relationships within a cohesive context. Moreover, we propose a novel region-report alignment (RRA) training strategy. It leverages the recognition of referring regions to guide the generation of region-specific reports, enhancing the model's referring and grounding capabilities while also improving the report's interpretability. A large language model (LLM) is further employed as the language decoder to generate reports from integrated visual features, facilitating region-level comprehension. Extensive experiments on two large-scale chest CT-report datasets demonstrate the superiority of our method, which outperforms several state-of-the-art methods in terms of both natural language generation and clinical efficacy metrics while preserving promising interpretability. The code is available at https://github.com/zhi-xuan-chen/Reg2RG.

Figures

Figures reproduced from arXiv: 2411.15539 by the authors.

Figure 1
Figure 1. Our method vs. the vanilla method. The gray background highlights instances of incorrect diagnosis. For ease of comparison, we divide the report into region-level sections. (a) The vanilla method based on global features is prone to neglecting some abnormalities since it fails to explore local details. (b) In contrast, our method can correctly detect all abnormalities with the region-guided local features. volumes a… view at source ↗
Figure 2
Figure 2. Overview of the proposed Reg2RG framework. It integrates global and local features as visual embeddings for the LLM to generate reports. Global features are encoded from the entire volume, while local features are extracted using segmentation masks to capture lesion details in sub-regions. The local features are decoupled into texture and geometry, where texture is derived from cropped masked volumes and geometry is… view at source ↗
Figure 3
Figure 3. The report length distributions of the ground-truth reports, along with those generated by our proposed method and the SOTA Med￾VInT [22]. The Kullback-Leibler (KL) divergence is utilized to quantify the differences between the distributions of our method and MedVInT relative to the ground-truth reports. regions as normal, missing critical abnormalities. In contrast, our model identifies most abnormalities except th… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Case studies of our model and the SOTA MedVInT [22]. The different colors represent distinct anatomical areas, as shown at the bottom of each example. The gray background highlights incorrect diagnoses. Our Region-level Reports The region 0 is abdomen: There are calcif…
Figure 5
Figure 5. Figure 5: Region-level reports generated by our model. Each regional report refers to a specific region and is grounded in the anatomical area depicted in the left figure. The different colors correspond to distinct anatomical regions [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Segmentation results on the RadGenome-ChestCT and CTRG-Chest-584K datasets. The different colors represent distinct anatomical areas, as shown at the bottom. valuable for improving the reliability of the generated reports. Given that the relative positions of anatomica…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LLM-driven Medical Report Generation via Communication-efficient Heterogeneous Federated Learning

    cs.CV 2025-06 conditional novelty 5.0 of 10

    FedMRG trains federated LLM-based report generators with low-rank adapters, diagnosis prompts, and dual-adapter mutual boosting, beating baselines on chest X-ray benchmarks.

  2. Region-Aware Multimodal Large Language Model via SlowFast Tokenization and Pseudo-Mask Guidance for 3D CT Report Generation

    eess.IV 2025-06 conditional novelty 4.0 of 10

    MedRegion-CT integrates region-representative tokens, mask-driven segmentation tokens, and patient-specific attribute prompts into a multimodal LLM, reporting state-of-the-art scores on RadGenome-Chest CT report generation.

Reference graph

Works this paper leans on

46 extracted references · 11 canonical work pages · cited by 2 Pith papers

  1. [20]

    Dia-llama: Towards large language model-driven ct report generation,

    Z. Chen, L. Luo, Y . Bie, and H. Chen, “Dia-llama: Towards large language model-driven ct report generation,” arXiv preprint arXiv:2403.16386, 2024

    arXiv 2024

  2. [9]

    Benchmarking and boosting radiology report generation for 3d high-resolution medical images,

    C. Liu, Z. Wan, Y . Wang, H. Shen, H. Wang, K. Zheng, M. Zhang, and R. Arcucci, “Benchmarking and boosting radiology report generation for 3d high-resolution medical images,” arXiv preprint arXiv:2406.07146 , 2024. 12 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XX, NO. XX, XXXX 2024

    arXiv 2024

  3. [1]

    Modern diagnostic imaging technique applications and risk factors in the medical field: a review,

    S. Hussain, I. Mubeen, N. Ullah, S. S. U. D. Shah, B. A. Khan, M. Zahoor, R. Ullah, F. A. Khan, and M. A. Sultan, “Modern diagnostic imaging technique applications and risk factors in the medical field: a review,” BioMed research international , vol. 2022, no. 1, p. 5164970, 2022

    work page 2022

  4. [2]

    Evidence-based guideline for the written radiology report: Methods, recommendations and implementation challenges,

    S. K. Goergen, F. J. Pool, T. J. Turner, J. E. Grimm, M. N. Appleyard, C. Crock, M. C. Fahey, M. F. Fay, N. J. Ferris, S. M. Liew et al. , “Evidence-based guideline for the written radiology report: Methods, recommendations and implementation challenges,” Journal of medical imaging and radiation oncology , vol. 57, no. 1, pp. 1–7, 2013

    work page 2013

  5. [3]

    Ct2rep: Automated radi- ology report generation for 3d medical imaging,

    I. E. Hamamci, S. Er, and B. Menze, “Ct2rep: Automated radi- ology report generation for 3d medical imaging,” arXiv preprint arXiv:2403.06801, 2024

    arXiv 2024

  6. [4]

    Towards generalist foundation model for radiology,

    C. Wu, X. Zhang, Y . Zhang, Y . Wang, and W. Xie, “Towards generalist foundation model for radiology,” arXiv preprint arXiv:2308.02463 , 2023

    arXiv 2023

  7. [5]

    M3d: Advancing 3d medical image analysis with multi-modal large language models,

    F. Bai, Y . Du, T. Huang, M. Q.-H. Meng, and B. Zhao, “M3d: Advancing 3d medical image analysis with multi-modal large language models,” arXiv preprint arXiv:2404.00578 , 2024

    arXiv 2024

  8. [6]

    One model to rule them all: Towards universal segmentation for medical images with text prompts,

    Z. Zhao, Y . Zhang, C. Wu, X. Zhang, Y . Zhang, Y . Wang, and W. Xie, “One model to rule them all: Towards universal segmentation for medical images with text prompts,” arXiv preprint arXiv:2312.17183 , 2023

    arXiv 2023

Show all 46 references
  1. [7]

    Segment anything in medical images,

    J. Ma, Y . He, F. Li, L. Han, C. You, and B. Wang, “Segment anything in medical images,” Nature Communications, vol. 15, no. 1, p. 654, 2024

    2024

  2. [8]

    Segment anything in medical images with nnunet,

    R. Stock, Y . Kirchhoff, M. R. Rokuss, A. Ravindran, and K. Maier- Hein, “Segment anything in medical images with nnunet,” in CVPR 2024: Segment Anything In Medical Images On Laptop

    2024

  3. [10]

    An organ-aware diagnosis framework for radiology report generation,

    S. Li, P. Qiao, L. Wang, M. Ning, L. Yuan, Y . Zheng, and J. Chen, “An organ-aware diagnosis framework for radiology report generation,” IEEE Transactions on Medical Imaging , 2024

    2024

  4. [11]

    Generating radiology re- ports via memory-driven transformer,

    Z. Chen, Y . Song, T.-H. Chang, and X. Wan, “Generating radiology re- ports via memory-driven transformer,” arXiv preprint arXiv:2010.16056, 2020

    2010 arXiv

  5. [12]

    Cross-modal memory networks for radiology report generation,

    Z. Chen, Y . Shen, Y . Song, and X. Wan, “Cross-modal memory networks for radiology report generation,” arXiv preprint arXiv:2204.13258, 2022

    2022 arXiv

  6. [13]

    Promptmrg: Diagnosis-driven prompts for medical report generation,

    H. Jin, H. Che, Y . Lin, and H. Chen, “Promptmrg: Diagnosis-driven prompts for medical report generation,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 3, 2024, pp. 2607– 2615

    2024

  7. [14]

    Interactive and ex- plainable region-guided radiology report generation,

    T. Tanida, P. M ¨uller, G. Kaissis, and D. Rueckert, “Interactive and ex- plainable region-guided radiology report generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7433–7442

    2023

  8. [15]

    A foundation model utilizing chest ct volumes and radiology reports for supervised-level zero-shot detection of abnormalities,

    I. E. Hamamci, S. Er, F. Almas, A. G. Simsek, S. N. Esirgun, I. Do- gan, M. F. Dasdelen, B. Wittmann, E. Simsar, M. Simsar et al. , “A foundation model utilizing chest ct volumes and radiology reports for supervised-level zero-shot detection of abnormalities,” arXiv preprint a...

    2024

  9. [16]

    Radgenome-chest ct: A grounded vision-language dataset for chest ct analysis,

    X. Zhang, C. Wu, Z. Zhao, J. Lei, Y . Zhang, Y . Wang, and W. Xie, “Radgenome-chest ct: A grounded vision-language dataset for chest ct analysis,” arXiv preprint arXiv:2404.16754 , 2024

    2024 arXiv

  10. [17]

    Work like a doctor: Unifying scan localizer and dynamic generator for automated computed tomography report generation,

    Y . Tang, H. Yang, L. Zhang, and Y . Yuan, “Work like a doctor: Unifying scan localizer and dynamic generator for automated computed tomography report generation,” Expert Systems with Applications , vol. 237, p. 121442, 2024

    2024

  11. [18]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,

    J. Li, D. Li, C. Xiong, and S. Hoi, “Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,” in International conference on machine learning . PMLR, 2022, pp. 12 888–12 900

    2022

  12. [19]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” Advances in neural information processing systems , vol. 36, 2024

    2024

  13. [21]

    Towards a holistic framework for multimodal large language models in three-dimensional brain ct report generation,

    C.-Y . Li, K.-J. Chang, C.-F. Yang, H.-Y . Wu, W. Chen, H. Bansal, L. Chen, Y .-P. Yang, Y .-C. Chen, S.-P. Chenet al., “Towards a holistic framework for multimodal large language models in three-dimensional brain ct report generation,” arXiv preprint arXiv:2407.02235 , 2024

    2024 arXiv

  14. [22]

    Pmc-vqa: Visual instruction tuning for medical visual question answer- ing,

    X. Zhang, C. Wu, Z. Zhao, W. Lin, Y . Zhang, Y . Wang, and W. Xie, “Pmc-vqa: Visual instruction tuning for medical visual question answer- ing,” arXiv preprint arXiv:2305.10415 , 2023

    2023 arXiv

  15. [23]

    Visionllm: Large language model is also an open-ended decoder for vision-centric tasks,

    W. Wang, Z. Chen, X. Chen, J. Wu, X. Zhu, G. Zeng, P. Luo, T. Lu, J. Zhou, Y . Qiao et al. , “Visionllm: Large language model is also an open-ended decoder for vision-centric tasks,” Advances in Neural Information Processing Systems , vol. 36, 2024

    2024

  16. [24]

    Gpt4roi: Instruction tuning large language model on region- of-interest,

    S. Zhang, P. Sun, S. Chen, M. Xiao, W. Shao, W. Zhang, Y . Liu, K. Chen, and P. Luo, “Gpt4roi: Instruction tuning large language model on region- of-interest,” arXiv preprint arXiv:2307.03601 , 2023

    2023 arXiv

  17. [25]

    Groma: Localized visual tokenization for grounding multimodal large language models,

    C. Ma, Y . Jiang, J. Wu, Z. Yuan, and X. Qi, “Groma: Localized visual tokenization for grounding multimodal large language models,” arXiv preprint arXiv:2404.13013, 2024

    2024 arXiv

  18. [26]

    Regiongpt: Towards region understanding vision language model,

    Q. Guo, S. De Mello, H. Yin, W. Byeon, K. C. Cheung, Y . Yu, P. Luo, and S. Liu, “Regiongpt: Towards region understanding vision language model,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 13 796–13 806

    2024

  19. [27]

    Ferret: Refer and ground anything anywhere at any granularity,

    H. You, H. Zhang, Z. Gan, X. Du, B. Zhang, Z. Wang, L. Cao, S.-F. Chang, and Y . Yang, “Ferret: Refer and ground anything anywhere at any granularity,” arXiv preprint arXiv:2310.07704 , 2023

    2023 arXiv

  20. [28]

    Maira-2: Grounded radiology report generation,

    S. Bannur, K. Bouzid, D. C. Castro, A. Schwaighofer, S. Bond- Taylor, M. Ilse, F. P ´erez-Garc´ıa, V . Salvatelli, H. Sharma, F. Meissen et al., “Maira-2: Grounded radiology report generation,” arXiv preprint arXiv:2406.04449, 2024

    2024 arXiv

  21. [29]

    Minigpt-med: Large language model as a general interface for radiology diagnosis,

    A. Alkhaldi, R. Alnajim, L. Alabdullatef, R. Alyahya, J. Chen, D. Zhu, A. Alsinan, and M. Elhoseiny, “Minigpt-med: Large language model as a general interface for radiology diagnosis,” arXiv preprint arXiv:2407.04106, 2024

    2024 arXiv

  22. [30]

    A refer-and-ground multimodal large language model for biomedicine,

    X. Huang, H. Huang, L. Shen, Y . Yang, F. Shang, J. Liu, and J. Liu, “A refer-and-ground multimodal large language model for biomedicine,” in International Conference on Medical Image Computing and Computer- Assisted Intervention. Springer, 2024, pp. 399–409

    2024

  23. [31]

    Interpretable bilingual multimodal large language model for diverse biomedical tasks,

    L. Wang, H. Wang, H. Yang, J. Mao, Z. Yang, J. Shen, and X. Li, “Interpretable bilingual multimodal large language model for diverse biomedical tasks,” arXiv preprint arXiv:2410.18387 , 2024

    2024 arXiv

  24. [32]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

    2023 arXiv

  25. [33]

    Qwen2 technical report,

    A. Yang, B. Yang, B. Hui, B. Zheng, B. Yu, C. Zhou, C. Li, C. Li, D. Liu, F. Huang, G. Dong, H. Wei, H. Lin, J. Tang, J. Wang, J. Yang, J. Tu, J. Zhang, J. Ma, J. Xu, J. Zhou, J. Bai, J. He, J. Lin, K. Dang, K. Lu, K. Chen, K. Yang, M. Li, M. Xue, N. Ni, P. Zhang, P. Wang, R. ...

    2024 arXiv

  26. [34]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting of the Association for Computational Linguistics , 2002, pp. 311–318

    2002

  27. [35]

    Meteor: An automatic metric for mt evalua- tion with improved correlation with human judgments,

    S. Banerjee and A. Lavie, “Meteor: An automatic metric for mt evalua- tion with improved correlation with human judgments,” in Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization , 2005, pp. 65–72

    2005

  28. [36]

    Rouge: A package for automatic evaluation of summaries,

    C.-Y . Lin, “Rouge: A package for automatic evaluation of summaries,” in Text summarization branches out , 2004, pp. 74–81

    2004

  29. [37]

    Radbert: adapting transformer-based language models to radiology,

    A. Yan, J. McAuley, X. Lu, J. Du, E. Y . Chang, A. Gentili, and C.-N. Hsu, “Radbert: adapting transformer-based language models to radiology,” Radiology: Artificial Intelligence , vol. 4, no. 4, p. e210258, 2022

    2022

  30. [38]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al. , “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  31. [39]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685 , 2021

    2021 arXiv

  32. [40]

    Decoupled weight decay regularization,

    I. Loshchilov, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  33. [41]

    Zero: Memory optimizations toward training trillion parameter models,

    S. Rajbhandari, J. Rasley, O. Ruwase, and Y . He, “Zero: Memory optimizations toward training trillion parameter models,” in SC20: International Conference for High Performance Computing, Networking, Storage and Analysis . IEEE, 2020, pp. 1–16

    2020

  34. [42]

    Training deep nets with sublinear memory cost,

    T. Chen, B. Xu, C. Zhang, and C. Guestrin, “Training deep nets with sublinear memory cost,” arXiv preprint arXiv:1604.06174 , 2016

    2016 arXiv

  35. [43]

    R2gengpt: Radiology report generation with frozen llms,

    Z. Wang, L. Liu, L. Wang, and L. Zhou, “R2gengpt: Radiology report generation with frozen llms,” Meta-Radiology, vol. 1, no. 3, p. 100033, 2023

    2023

  36. [44]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019

    2019

  37. [45]

    Improving chest x-ray report generation by leveraging warm starting,

    A. Nicolson, J. Dowling, and B. Koopman, “Improving chest x-ray report generation by leveraging warm starting,” Artificial intelligence in medicine, vol. 144, p. 102633, 2023

    2023

  38. [46]

    Automated radiology report generation using conditioned transformers,

    O. Alfarghaly, R. Khaled, A. Elkorany, M. Helal, and A. Fahmy, “Automated radiology report generation using conditioned transformers,” Informatics in Medicine Unlocked , vol. 24, p. 100557, 2021

    2021

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.