Pith. sign in

REVIEW 4 major objections 5 minor 71 references

LLM-driven Medical Report Generation via Communication-efficient Heterogeneous Federated Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Federated learning can train LLM radiology report writers without moving patient images.

desk verdict First Fed+LLM MRG paper with a useful benchmark; the privacy claim needs a caveat and the algorithm description needs a fix, but it deserves review. read the letter →

arxiv 2506.17562 v2 pith:47EAWPA2 submitted 2025-06-21 cs.CV cs.CL

Haoxuan Che , Haibo Jin , Zhengrui Guo , Yi Lin , Cheng Jin , Hao Chen This is my paper · ORCID
classification cs.CVcs.CL
keywords FederatedLearningMedicalReportGenerationLargeLanguageModelsLow-rankAdaptationDataHeterogeneityChestX-rayContrastiveKnowledgeDistillation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes FedMRG, a federated learning framework for training large language model (LLM) based medical report generators across hospitals without moving patient data. The central claim is that FedMRG is the first framework to make such training practical by cutting communication cost through low-rank factorization of parameter updates, and by handling two kinds of data heterogeneity at once: image differences across centers and differences in reporting style and terminology. A sympathetic reader would care because medical report generation needs large image-report corpora that no single institution can supply, and privacy rules block centralizing them. The paper argues that with FedMRG, multiple centers can jointly train a clinically accurate report generator while only exchanging small adapter weights and encoder parameters.

What carries the argument

The load-bearing mechanism is the separation of trainable parameters into three roles. Low-rank adaptation (LoRA) factorizes each update into two small matrices, so only these matrices and the image encoder travel to the server. The generic adapter in DMB aggregates global report knowledge through federated averaging, while the specialized adapter stays local and inherits the generic weights each round. Bidirectional knowledge distillation losses ($L_{l2g}$, $L_{g2l}$) transfer knowledge between the two adapters, and the hierarchical contrastive loss ($L_{hcl}$) pulls local features toward a global memory bank while pushing apart features from different clients.

What would settle it

Run a feature-inversion or membership-inference attack on the global memory bank features after training; if a third party can reconstruct recognizable chest X-rays or determine whether a known patient's image contributed a feature, the privacy-preserving claim fails even though benchmark accuracy holds.

Watch

Extended reading notes

Core claim

FedMRG's discovery is that the two obstacles to federated LLM report generation, communication overhead and multi-modal heterogeneity, can be addressed jointly without sacrificing accuracy. On the communication side, the frozen LLM is tuned only through low-rank adapters, so clients upload small matrices rather than billions of weights. On the image side, hierarchical contrasting and prompting (HCP) uses client-aware contrastive learning with a global memory bank of features from other clients, plus diagnosis tokens converted from disease labels, to keep features both generalizable and locally distinctive. On the text side, dual-adapter mutual boosting (DMB) maintains a generic adapter shared across clients and a specialized adapter kept local, with bidirectional knowledge distillation between them. The paper shows on its FL-MRG benchmark, built from MIMIC-CXR, IU X-Ray, and CheXpert+ splits, that FedMRG beats federated and MRG baselines on language metrics and clinical efficacy metrics while keeping communication cost at the level of a 59M-parameter update, and that it generalizes to unseen centers.

Load-bearing premise

The load-bearing premise is that sharing randomly sampled image features across clients through the global memory bank preserves patient privacy, but the paper provides no secure aggregation, differential privacy, or inversion-resistance analysis for those shared features.

Editorial extensions

If this is right

  • Only encoder weights and generic adapter weights are uploaded, so communication cost scales with adapter size rather than with the 7-billion-parameter decoder.
  • A hospital can deploy the shared generic adapter alone at a new site, giving a path to out-of-domain generalization without retraining.
  • Participating hospitals can personalize at inference by mixing generic and specialized adapter outputs with a fixed weight, preserving local terminology.
  • Adding more clients generally improves FedMRG performance, suggesting the framework can exploit larger federations.
  • The FL-MRG benchmark, with random and cluster-based splits plus real MIMIC-CXR and CheXpert+ partitions, gives the community a common testbed for federated report generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the global memory bank shares raw image features with no secure aggregation or differential privacy, the privacy claim is weaker than the benchmark numbers; a feature-inversion or membership-inference attack could test it directly.
  • The same three-way split (shared low-rank adapter, local adapter, distillation bridge) could transfer to other multi-modal federated tasks such as pathology or retinal imaging, where report style varies by institution.
  • The fixed inference weight $\alpha=0.5$ between generic and specialized adapters is likely suboptimal; learning $\alpha$ per client or per report could improve personalization beyond what the paper reports.
  • The FL-MRG benchmark simulates heterogeneity with visual transformations and report clustering; real-world heterogeneity also includes disease prevalence and demographic shifts, so reported margins may shrink under those conditions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes FedMRG, a federated learning framework for medical report generation (MRG) with LLM decoders. The method combines low-rank adaptation (LoRA) to reduce communication cost, a hierarchical contrastive and prompting (HCP) module for the image encoder, and a dual-adapter mutual boosting (DMB) mechanism for the text decoder. The authors also introduce FL-MRG, a benchmark that splits MIMIC-CXR into ten clients either randomly or by report-based clustering, and they evaluate on MIMIC-CXR test and IU X-Ray, plus a real-world setting that adds CheXpert+. The central claims are that FedMRG is the first privacy-preserving, communication-efficient FL framework for LLM-driven MRG, and that it outperforms existing FL and MRG baselines on both intra-domain and unseen-domain tests.

Significance. If the performance and communication-efficiency claims hold, this is a useful step toward multi-center LLM-based report generation: the paper ships a benchmark (FL-MRG), compares against many baselines, and its component ablations are consistent with the design rationale. The main risk is that the headline 'privacy-preserving' claim is not backed by any privacy analysis for the shared feature memory, and an internal inconsistency in the upload protocol muddies the communication-cost claims. The empirical gains in Tables I–III are plausible but would be strengthened by error bars or significance tests. Overall the contribution is potentially significant for the federated medical imaging community, but the privacy claim needs to be either substantiated or substantially qualified before publication.

major comments (4)
  1. [Sec. III-B.1, Eq. (3)] The global memory bank M stores image features randomly sampled from each local client and 'shares these features through parameter communication' (Sec. III-B.1). These features are per-patient image embeddings transmitted to the server and other clients, yet the paper provides no secure aggregation, differential privacy, feature sanitization, or inversion/attribute-inference analysis for this channel. l2-normalized pooled features can carry diagnostic and potentially identity-relevant information, and feature/model inversion from embeddings is a known risk. Because the abstract's central claim is 'privacy-preserving, multi-center development,' this gap is load-bearing: either add a formal privacy analysis or a concrete protection mechanism (e.g., DP noise, secure aggregation, or feature perturbation), or revise the privacy claim to reflect that only raw data are kept local while feature-level representations are shared.
  2. [Algorithm 1, line 16 vs. Sec. III-D and Sec. III-C] Algorithm 1 line 16 says 'Upload Fe and each Ag and As', but Sec. III-D states that 'clients upload only their encoder parameters and generic adapter parameters' and Sec. III-C.1 says As 'remains localized without participation in the aggregation of models.' This is a direct internal inconsistency. If As is actually uploaded, the communication cost reported in the tables (59M) is understated and the privacy surface expands; if As is not uploaded, Algorithm 1 is wrong. Either way, the paper must be corrected and the communication-overhead numbers recalculated consistently.
  3. [Sec. IV-A.3 and Table III] The real-world federated setting description is ambiguous. 'We strategically partitioned each institutional dataset across three distinct federated clients' can mean three clients total (one per institution with unequal splits) or three clients per institution (six clients total), and the number of clients actually used in Table III is never stated. It is also unclear how the internal test set is constructed for CheXpert+ and MIMIC-CXR after extracting ~55,000 pairs from each, and whether the IU X-Ray external test is the full dataset. Without a precise client count, split ratios, and test-set construction, Table III cannot be reproduced or interpreted as evidence for the claimed generalization.
  4. [Tables I–IV] All tables report results from what appear to be single runs, with no error bars, multiple seeds, or significance tests. Given that the improvements over baselines are often small (e.g., Table I FedMRG vs. Fed-LoRA on several BLEU/ROUGE scores differ by less than 1 point), the claims of consistent superiority would be much more persuasive with at least three random seeds plus standard deviations, or a paired significance test. The paper already reports standard deviations in Sec. IV-D.2 for the heterogeneity study, so this is feasible within the existing experimental setup.
minor comments (5)
  1. [Sec. IV-A.4 and Tables I–III] The evaluation section lists CIDEr among the NLG metrics, but CIDEr does not appear in any of the result tables; either add the CIDEr column or remove it from the metric list.
  2. [Fig. 6] The axis label 'Addition Score of BLU1 and BLU4' contains typos: it should be 'Sum of BLEU1 and BLEU4' or similar. Also, the hyperparameter curves would benefit from error bars and a clear description of which dataset split is used for each curve.
  3. [Sec. III-C.3, Eq. (6)] The symbol α is used both as the loss weight in Algorithm 1 (L1 := αL_HCL + ...) and as the adapter fusion weight in Eq. (6); using a different symbol (e.g., λ) for one of these would avoid confusion.
  4. [Section III-A.3 and References [52], [58]] References [52] and [58] both cite the same LoRA paper (Hu et al., ICLR 2022); please consolidate the duplicate reference.
  5. [Sec. IV-C.1] The related-work discussion mentions FedAvg as a compared method, but FedAvg is not listed in the model names of Tables I–III; clarify whether the non-federated MRG methods are trained with FedAvg as the default algorithm and label the tables accordingly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: FedMRG's reported gains are measured on external test sets and its components are not defined in terms of the quantities they predict.

full rationale

I examined the derivation chain of FedMRG against the six circularity patterns. The method's central outputs — communication cost reduction, heterogeneity handling, and report quality — do not reduce by construction to any fitted input or to a self-citation. LoRA is a standard external parameterization; HCP's contrastive loss (Eq. 3) uses a global memory bank of shared features as a training signal rather than as the evaluation target; DMB's distillation losses (Eqs. 4-5) are mutual regularizers between adapters, not predictions. The benchmark FL-MRG is built from held-out partitions of MIMIC-CXR and IU X-Ray, and no constants are fitted to those test results. CheXbert is used both to derive training labels for the diagnosis branch and to compute CE metrics on generated text; this is a label-source overlap, not a circular reduction, because the CE score is computed from the full generated report and is not equal to the training loss by any equation in the paper. PromptMRG [7] is cited as inspiration, but the FL-specific contributions are independently implemented and ablated. The limitations paragraph in Sec. V acknowledges that simulated heterogeneity and synchronized participation are simplifications, which are scope caveats rather than circular steps. The shared-feature privacy exposure and the Algorithm 1 inconsistency (line 16 uploading As while Sec. III-D says only Fe and Ag are uploaded) are correctness or privacy risks, not circularity, and do not change the verdict that the benchmark comparisons are externally grounded.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are postulated; the memory bank, adapters, and prompt tokens are architectural components of the proposed system. The free parameters are mostly standard hyperparameters, but the omission of final values for alpha and beta, plus the unspecified LoRA rank, leaves the exact configuration underdetermined.

free parameters (5)
  • HCP loss weight alpha = Not reported (tuned in Fig. 6)
    Weights the contrastive loss L_HCL in the total objective; final value omitted from text.
  • Distillation loss weight beta = Not reported (tuned in Fig. 6)
    Weights the knowledge-distillation losses L_l2g and L_g2l; final value omitted from text.
  • Contrastive temperature tau = 0.07
    Set for the client-aware contrastive loss; sensitivity shown in Fig. 6.
  • Adapter fusion weight alpha = 0.5
    Balances generic and specialized adapters at inference (Eq. 6); chosen by hand.
  • LoRA rank/dropout = Unspecified
    Low-rank adapter hyperparameters are not given, preventing exact replication.
assumptions (4)
  • domain assumption CheXbert-generated 14 disease labels are sufficiently accurate to serve both as training targets for the diagnosis branch and as labels for clinical efficacy evaluation.
    The framework's prompt tokens and CE metrics both rely on CheXbert; if the labeler is biased, training signal and evaluation are biased together.
  • ad hoc to paper Sharing image features across clients through the global memory bank does not expose private patient information.
    The paper asserts privacy-preservation but provides no secure aggregation, differential privacy, or inversion analysis for the shared features in Sec. III-B.1.
  • domain assumption Simulated heterogeneity from visual transformations and report-based clustering resembles real cross-center variation.
    The benchmark's validity for real deployments depends on this approximation, acknowledged as a limitation in the conclusion.
  • domain assumption Federated averaging of LoRA adapters and the ResNet encoder converges to a useful global model under the stated heterogeneity.
    FedAvg convergence under multimodal heterogeneity with partial client participation is assumed following prior FL literature; no convergence analysis is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLM-driven Medical Report Generation via Communication-efficient Heterogeneous Federated Learning." pith.science (2026). https://pith.science/paper/47EAWPA2

@misc{pith2026250617562,
  author       = {Pith},
  title        = {Pith review of: LLM-driven Medical Report Generation via Communication-efficient Heterogeneous Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/47EAWPA2}},
  note         = {Machine review of arXiv:2506.17562}
}
read the original abstract

LLMs have demonstrated significant potential in Medical Report Generation (MRG), yet their development requires large amounts of medical image-report pairs, which are commonly scattered across multiple centers. Centralizing these data is exceptionally challenging due to privacy regulations, thereby impeding model development and broader adoption of LLM-driven MRG models. To address this challenge, we present FedMRG, the first framework that leverages Federated Learning (FL) to enable privacy-preserving, multi-center development of LLM-driven MRG models, specifically designed to overcome the critical challenge of communication-efficient LLM training under multi-modal data heterogeneity. To start with, our framework tackles the fundamental challenge of communication overhead in FL-LLM tuning by employing low-rank factorization to efficiently decompose parameter updates, significantly reducing gradient transmission costs and making LLM-driven MRG feasible in bandwidth-constrained FL settings. Furthermore, we observed the dual heterogeneity in MRG under the FL scenario: varying image characteristics across medical centers, as well as diverse reporting styles and terminology preferences. To address this, we further enhance FedMRG with (1) client-aware contrastive learning in the MRG encoder, coupled with diagnosis-driven prompts, which capture both globally generalizable and locally distinctive features while maintaining diagnostic accuracy; and (2) a dual-adapter mutual boosting mechanism in the MRG decoder that harmonizes generic and specialized adapters to address variations in reporting styles and terminology. Through extensive evaluation of our established FL-MRG benchmark, we demonstrate the generalizability and adaptability of FedMRG, underscoring its potential in harnessing multi-center data and generating clinically accurate reports while maintaining communication efficiency.

Figures

Figures reproduced from arXiv: 2506.17562 by the authors.

Figure 1
Figure 1. Training a LLM-driven MRG model in FL settings face chal￾lenges: 1) heavy communication overheads, and 2) multi-modal data heterogeneity among centers. Index Terms— Federated Learning, Large Language Model, Medical Report Generation, Data Heterogeneity. I. INTRODUCTION Recently, Large Language Models (LLMs) have emerged as a promising approach for Medical Report Generation (MRG), enabling automatic creation of detai… view at source ↗
Figure 2
Figure 2. The framework of FedMRG with three stages after model initialization: (1) Distribution: The server distributes initialized models to clients. (2) Local Client Training: Clients train models using our two key modules: Hierarchical Contrasting and Prompting (HCP) addresses image heterogeneity through client-aware feature contrasting (enabling inter- and intra-client knowledge sharing) and diagnosis-aware prompting (co… view at source ↗
Figure 3
Figure 3. The DMB involves two adapter configurations and tuning strategies, and bi-directional knowledge enhancement via distillation. As previously established, healthcare facilities serve diverse patient populations and maintain distinct documentation pro￾tocols and clinical specializations, resulting in highly vari￾able report structures and terminology. To address this chal￾lenge, we propose the Dual-adapter Mutual Boost… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Client split by report-based clustering causes obvious data heterogeneity among clients at both word and semantics levels. participating clients, both the generic adapter Ag and the client’s specialized adapter As are activated and integrated through a specific mechani…
Figure 5
Figure 5. Figure 5: Performance of the baseline and FedMRG towards different levels of client heterogeneity and different numbers of clients. 0.03 0.05 0.07 0.10 0.12 0.15 Contrastive Tempurature 48 51 54 57 60 Addition Score of BLU1 and BLU4 IU X-Ray MIMIC 0.1 0.3 0.5 0.7 0.9 Representat…
Figure 6
Figure 6. Figure 6: Effects on hyper-parameters on the model performance. prove to be substantial. Their removal causes consistent perfor￾mance reductions across metrics, illustrating their synergistic effect in optimizing model performance and confirming the effectiveness of our mutual k…
Figure 7
Figure 7. Figure 7: Qualitative examples of the baseline and FedMRG. Blue font indicates consistent content with the ground-truth, while red font indicates patient-aware content. and patient-specific context awareness, which may translate to better patient-friendly clinical utility in rea…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

71 extracted references · 59 canonical work pages

  1. [1]

    R2gengpt: Radiology report generation with frozen llms,

    Z. Wang, L. Liu, L. Wang, and L. Zhou, “R2gengpt: Radiology report generation with frozen llms,” Meta-Radiology, vol. 1, no. 3, p. 100033, 2023

    work page 2023

  2. [2]

    Llm-cxr: Instruction- finetuned llm for cxr image understanding and generation,

    S. Lee, W. J. Kim, J. Chang, and J. C. Ye, “Llm-cxr: Instruction- finetuned llm for cxr image understanding and generation,” in ICLR, 2024

    work page 2024

  3. [3]

    MedXChat: A Unified Multimodal Large Language Model Framework towards CXRs Understanding and Generation

    L. Yang, Z. Wang, and L. Zhou, “Medxchat: Bridging cxr modal- ities with a unified multimodal large model,” arXiv preprint arXiv:2312.02233, 2023

    work page Pith review arXiv 2023

  4. [4]

    Xraygpt: Chest radiographs summarization using large medical vision-language models,

    O. Thawkar, A. Shaker, S. S. Mullappilly, H. Cholakkal, R. M. Anwer, S. Khan et al., “Xraygpt: Chest radiographs summarization using large medical vision-language models,” arXiv preprint arXiv:2306.07971 , 2023

    arXiv 2023

  5. [5]

    Dia-llama: Towards large language model-driven ct report generation,

    Z. Chen, L. Luo, Y . Bie, and H. Chen, “Dia-llama: Towards large language model-driven ct report generation,” arXiv preprint arXiv:2403.16386, 2024

    arXiv 2024

  6. [6]

    Bootstrapping large language models for radiology report generation,

    C. Liu, Y . Tian, W. Chen, Y . Song, and Y . Zhang, “Bootstrapping large language models for radiology report generation,” in AAAI, 2024

    work page 2024

  7. [7]

    Promptmrg: Diagnosis-driven prompts for medical report generation,

    H. Jin, H. Che, Y . Lin, and H. Chen, “Promptmrg: Diagnosis-driven prompts for medical report generation,” AAAI, 2024

    work page 2024

  8. [8]

    Towards generalizable diabetic retinopathy grading in unseen domains,

    H. Che, Y . Cheng, H. Jin, and H. Chen, “Towards generalizable diabetic retinopathy grading in unseen domains,” in MICCAI, 2023

    work page 2023

Show all 71 references
  1. [9]

    Learning robust representation for joint grading of ophthalmic diseases via adaptive curriculum and feature disentanglement,

    H. Che, H. Jin, and H. Chen, “Learning robust representation for joint grading of ophthalmic diseases via adaptive curriculum and feature disentanglement,” in MICCAI, 2022

    2022

  2. [10]

    Image quality-aware diagnosis via meta- knowledge co-embedding,

    H. Che, S. Chen, and H. Chen, “Image quality-aware diagnosis via meta- knowledge co-embedding,” in CVPR, 2023

    2023

  3. [11]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov et al. , “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288 , 2023

    2023 arXiv

  4. [12]

    On the automatic generation of medical imaging reports,

    B. Jing, P. Xie, and E. Xing, “On the automatic generation of medical imaging reports,” in ACL, 2018

    2018

  5. [13]

    Pain- fl: Personalized privacy-preserving incentive for federated learning,

    P. Sun, H. Che, Z. Wang, Y . Wang, T. Wang, L. Wu, and H. Shao, “Pain- fl: Personalized privacy-preserving incentive for federated learning,” IEEE J. Sel. Areas Commun. , vol. 39, no. 12, pp. 3805–3820, 2021

    2021

  6. [14]

    Unpaired optical coherence tomography angiography image super-resolution via frequency-aware inverse-consistency gan,

    W. Zhang, D. Yang, H. Che, A. R. Ran, C. Y . Cheung, and H. Chen, “Unpaired optical coherence tomography angiography image super-resolution via frequency-aware inverse-consistency gan,” IEEE J. Biomed. Health Inform. , 2024. 12 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XX, NO...

    2024

  7. [15]

    A deep learning system for predicting time to progression of diabetic retinopathy,

    L. Dai, B. Sheng, T. Chen, Q. Wu, R. Liu, C. Cai, L. Wu, D. Yang, H. Hamzah, Y . Liu et al., “A deep learning system for predicting time to progression of diabetic retinopathy,” Nature Medicine, vol. 30, no. 2, pp. 584–594, 2024

    2024

  8. [16]

    Feddag: Federated domain adversarial generation towards generalizable medical image analysis,

    H. Che, Y . Wu, H. Jin, Y . Xia, and H. Chen, “Feddag: Federated domain adversarial generation towards generalizable medical image analysis,” arXiv preprint arXiv:2501.13967 , 2025

    2025 arXiv

  9. [17]

    Privacy in the age of medical big data,

    W. N. Price and I. G. Cohen, “Privacy in the age of medical big data,” Nat. Med., vol. 25, no. 1, pp. 37–43, 2019

    2019

  10. [18]

    Mimic-cxr-jpg, a large publicly available database of labeled chest radiographs,

    A. E. Johnson, T. J. Pollard, N. R. Greenbaum, M. P. Lungren, C.-y. Deng, Y . Penget al., “Mimic-cxr-jpg, a large publicly available database of labeled chest radiographs,” arXiv preprint arXiv:1901.07042 , 2019

    1901 arXiv

  11. [19]

    Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data,

    M. J. Sheller, B. Edwards, G. A. Reina, J. Martin, S. Pati, A. Kotrotsou et al. , “Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data,” Sci. Rep., vol. 10, no. 1, p. 12598, 2020

    2020

  12. [20]

    The future landscape of large language models in medicine,

    J. Clusmann, F. R. Kolbinger, H. S. Muti, Z. I. Carrero, J.-N. Eckardt, N. G. Laleh et al. , “The future landscape of large language models in medicine,” Commun. Med., vol. 3, no. 1, p. 141, 2023

    2023

  13. [21]

    The future of digital health with federated learning,

    N. Rieke, J. Hancox, W. Li, F. Milletari, H. R. Roth, S. Albarqouni et al., “The future of digital health with federated learning,” NPJ Digit. Med., vol. 3, no. 1, pp. 1–7, 2020

    2020

  14. [22]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in AISTATS, 2017

    2017

  15. [23]

    Palm: Scaling language modeling with pathways,

    A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann et al., “Palm: Scaling language modeling with pathways,” J. Mach. Learn. Res. , vol. 24, no. 240, pp. 1–113, 2023

    2023

  16. [24]

    Feddat: An approach for foundation model finetuning in multi-modal heterogeneous federated learning,

    H. Chen, Y . Zhang, D. Krompass, J. Gu, and V . Tresp, “Feddat: An approach for foundation model finetuning in multi-modal heterogeneous federated learning,” AAAI, 2024

    2024

  17. [25]

    Multi-modal domain adaptation for fine- grained action recognition,

    J. Munro and D. Damen, “Multi-modal domain adaptation for fine- grained action recognition,” in CVPR, 2020

    2020

  18. [26]

    Towards better understanding of domain shift on linear-probed visual foundation models,

    E. Heim, “Towards better understanding of domain shift on linear-probed visual foundation models,” NeurIPS Workshop, 2023

    2023

  19. [27]

    Deep learning in generating radiology reports: A survey,

    M. M. A. Monshi, J. Poon, and V . Chung, “Deep learning in generating radiology reports: A survey,” Artif. Intell. Med. , vol. 106, p. 101878, 2020

    2020

  20. [28]

    Generating radiology reports via memory-driven transformer,

    Z. Chen, Y . Song, T.-H. Chang, and X. Wan, “Generating radiology reports via memory-driven transformer,” in EMNLP, 2020

    2020

  21. [29]

    Radiology report generation with a learned knowledge base and multi-modal alignment,

    S. Yang, X. Wu, S. Ge, Z. Zheng, S. K. Zhou, and L. Xiao, “Radiology report generation with a learned knowledge base and multi-modal alignment,” Med. Image Anal. , vol. 86, p. 102798, 2023

    2023

  22. [30]

    Histgen: Histopathology report generation via local-global feature encoding and cross-modal context interaction,

    Z. Guo, J. Ma, Y . Xu, Y . Wang, L. Wang, and H. Chen, “Histgen: Histopathology report generation via local-global feature encoding and cross-modal context interaction,” in MICCAI, 2024

    2024

  23. [31]

    When radiology report generation meets knowledge graph,

    Y . Zhang, X. Wang, Z. Xu, Q. Yu, A. Yuille, and D. Xu, “When radiology report generation meets knowledge graph,” in AAAI, 2020

    2020

  24. [32]

    Exploring and distilling posterior and prior knowledge for radiology report generation,

    F. Liu, X. Wu, S. Ge, W. Fan, and Y . Zou, “Exploring and distilling posterior and prior knowledge for radiology report generation,” inCVPR, 2021

    2021

  25. [33]

    Dynamic graph enhanced contrastive learning for chest x-ray report generation,

    M. Li, B. Lin, Z. Chen, H. Lin, X. Liang, and X. Chang, “Dynamic graph enhanced contrastive learning for chest x-ray report generation,” in CVPR, 2023

    2023

  26. [34]

    Kiut: Knowledge-injected u- transformer for radiology report generation,

    Z. Huang, X. Zhang, and S. Zhang, “Kiut: Knowledge-injected u- transformer for radiology report generation,” in CVPR, 2023

    2023

  27. [35]

    Automated radiographic report generation purely on transformer: A multicriteria supervised approach,

    Z. Wang, H. Han, L. Wang, X. Li, and L. Zhou, “Automated radiographic report generation purely on transformer: A multicriteria supervised approach,” IEEE Trans. Med. Imaging., vol. 41, no. 10, pp. 2803–2813, 2022

    2022

  28. [36]

    A self-boosting framework for automated radiographic report generation,

    Z. Wang, L. Zhou, L. Wang, and X. Li, “A self-boosting framework for automated radiographic report generation,” in CVPR, 2021

    2021

  29. [37]

    Weakly supervised contrastive learning for chest x-ray report generation,

    A. Yan, Z. He, X. Lu, J. Du, E. Chang, A. Gentili, J. McAuley, and C.- N. Hsu, “Weakly supervised contrastive learning for chest x-ray report generation,” in EMNLP, 2021

    2021

  30. [38]

    Clinical-bert: Vision-language pre-training for radiograph diagnosis and reports generation,

    B. Yan and M. Pei, “Clinical-bert: Vision-language pre-training for radiograph diagnosis and reports generation,” in AAAI, 2022

    2022

  31. [39]

    Interactive and explainable region-guided radiology report generation,

    T. Tanida, P. M ¨uller, G. Kaissis, and D. Rueckert, “Interactive and explainable region-guided radiology report generation,” in CVPR, 2023

    2023

  32. [40]

    Metransformer: Radiology report generation by transformer with multiple learnable expert tokens,

    Z. Wang, L. Liu, L. Wang, and L. Zhou, “Metransformer: Radiology report generation by transformer with multiple learnable expert tokens,” in CVPR, 2023

    2023

  33. [41]

    A chain of diagnosis framework for accurate and explainable radiology report generation,

    H. Jin, H. Che, S. He, and H. Chen, “A chain of diagnosis framework for accurate and explainable radiology report generation,” IEEE Trans. Med. Imaging., 2025

    2025

  34. [42]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models,

    D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “Minigpt-4: Enhancing vision-language understanding with advanced large language models,” in ICLR, 2024

    2024

  35. [43]

    Large language model with region-guided referring and grounding for ct report generation,

    Z. Chen, Y . Bie, H. Jin, and H. Chen, “Large language model with region-guided referring and grounding for ct report generation,” arXiv preprint arXiv:2411.15539, 2024

    2024 arXiv

  36. [44]

    Cocktailsgd: Fine- tuning foundation models over 500mbps networks,

    J. Wang, Y . Lu, B. Yuan, B. Chen, P. Liang et al., “Cocktailsgd: Fine- tuning foundation models over 500mbps networks,” in ICML

  37. [45]

    Promptfl: Let federated participants cooperatively learn prompts instead of models-federated learning in age of foundation model,

    T. Guo, S. Guo, J. Wang, X. Tang, and W. Xu, “Promptfl: Let federated participants cooperatively learn prompts instead of models-federated learning in age of foundation model,” IEEE Trans. Mob. Comput., 2023

    2023

  38. [46]

    Efficient model person- alization in federated learning via client-specific prompt generation,

    F.-E. Yang, C.-Y . Wang, and Y .-C. F. Wang, “Efficient model person- alization in federated learning via client-specific prompt generation,” in CVPR, 2023

    2023

  39. [47]

    Global and local prompts cooperation via optimal transport for federated learning,

    H. Li, W. Huang, J. Wang, and Y . Shi, “Global and local prompts cooperation via optimal transport for federated learning,” inCVPR, 2024

    2024

  40. [48]

    Federated adaptive prompt tuning for multi-domain collaborative learning,

    S. Su, M. Yang, B. Li, and X. Xue, “Federated adaptive prompt tuning for multi-domain collaborative learning,” in AAAI, 2024

    2024

  41. [49]

    Towards building the federatedgpt: Federated instruction tuning,

    J. Zhang, S. Vahidian, M. Kuo, C. Li, R. Zhang et al., “Towards building the federatedgpt: Federated instruction tuning,” in ICASSP, 2024

    2024

  42. [50]

    Mix-of-show: Decentralized low-rank adaptation for multi-concept customization of diffusion models,

    Y . Gu, X. Wang, J. Z. Wu, Y . Shi, Y . Chen et al. , “Mix-of-show: Decentralized low-rank adaptation for multi-concept customization of diffusion models,” NeurIPS, 2024

    2024

  43. [51]

    Improving lora in privacy-preserving federated learning,

    Y . Sun, Z. Li, Y . Li, and B. Ding, “Improving lora in privacy-preserving federated learning,” ICLR, 2024

    2024

  44. [52]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li et al., “Lora: Low-rank adaptation of large language models,” in ICLR, 2022

    2022

  45. [53]

    Prefix-tuning: Optimizing continuous prompts for generation,

    X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” in ACL, 2021

    2021

  46. [54]

    The power of scale for parameter-efficient prompt tuning,

    B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” in EMNLP, 2021

    2021

  47. [55]

    Adaptive budget allocation for parameter-efficient fine-tuning,

    Q. Zhang, M. Chen, A. Bukharin, P. He, Y . Cheng et al. , “Adaptive budget allocation for parameter-efficient fine-tuning,” in ICLR, 2023

    2023

  48. [56]

    Vera: Vector-based random matrix adaptation,

    D. J. Kopiczko, T. Blankevoort, and Y . M. Asano, “Vera: Vector-based random matrix adaptation,” in ICLR, 2024

    2024

  49. [57]

    Fedpara: Low-rank hadamard product for communication-efficient federated learning,

    N. Hyeon-Woo, M. Ye-Bin, and T.-H. Oh, “Fedpara: Low-rank hadamard product for communication-efficient federated learning,” arXiv preprint arXiv:2108.06098, 2021

    2021 arXiv

  50. [58]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang et al., “Lora: Low-rank adaptation of large language models,” in ICLR, 2022

    2022

  51. [59]

    Supervised contrastive learning,

    P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y . Tian, P. Isola et al. , “Supervised contrastive learning,” in NeurIPS, 2020

    2020

  52. [60]

    Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison,

    J. Irvin, P. Rajpurkar, M. Ko, Y . Yu, S. Ciurea-Ilcus, C. Chute, H. Mark- lund, B. Haghgoo, R. Ball, K. Shpanskaya et al. , “Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison,” in Proceedings of the AAAI conference on artificial intellig...

    2019

  53. [61]

    Unlocking the potential of prompt-tuning in bridging generalized and personalized federated learning,

    W. Deng, C. Thrampoulidis, and X. Li, “Unlocking the potential of prompt-tuning in bridging generalized and personalized federated learning,” in CVPR, 2024

    2024

  54. [62]

    On bridging generic and personalized federated learning for image classification,

    H.-Y . Chen and W.-L. Chao, “On bridging generic and personalized federated learning for image classification,” in ICLR, 2022

    2022

  55. [63]

    Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter,

    V . Sanh, L. Debut, J. Chaumond, and T. Wolf, “Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter,” arXiv preprint arXiv:1910.01108, 2019

    1910 arXiv

  56. [64]

    Preparing a collection of radiology examinations for distribution and retrieval,

    D. Demner-Fushman, M. D. Kohli, M. B. Rosenman, S. E. Shooshan, L. Rodriguez et al. , “Preparing a collection of radiology examinations for distribution and retrieval,” J. Am. Med. Inform. Assoc., vol. 23, no. 2, pp. 304–310, 2016

    2016

  57. [65]

    Modeling and enhancing low- quality retinal fundus images,

    Z. Shen, H. Fu, J. Shen, and L. Shao, “Modeling and enhancing low- quality retinal fundus images,” IEEE Trans. Med. Imaging. , vol. 40, no. 3, pp. 996–1006, 2020

    2020

  58. [66]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in ACL, 2002

    2002

  59. [67]

    Cider: Consensus- based image description evaluation,

    R. Vedantam, C. Lawrence Zitnick, and D. Parikh, “Cider: Consensus- based image description evaluation,” in CVPR, 2015

    2015

  60. [68]

    Improving chest x-ray report generation by leveraging warm starting,

    A. Nicolson, J. Dowling, and B. Koopman, “Improving chest x-ray report generation by leveraging warm starting,” Artif. Intell. Med. , vol. 144, p. 102633, 2023

    2023

  61. [69]

    Combining automatic labelers and expert annotations for accurate radiology report labeling using bert,

    A. Smit, S. Jain, P. Rajpurkar, A. Pareek, A. Y . Ng, and M. P. Lungren, “Combining automatic labelers and expert annotations for accurate radiology report labeling using bert,” in EMNLP, 2020

    2020

  62. [70]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in CVPR, 2016

    2016

  63. [71]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” ICLR, 2019

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.