Pith. sign in

REVIEW 8 cited by

pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.13283 v2 pith:JXDB46G6 submitted 2023-10-20 cs.LG cs.DC

pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuning

Liping Yi , Han Yu , Gang Wang , Xiaoguang Liu , Xiaoxiao Li This is my paper
classification cs.LG cs.DC
keywords federatedlearningmodelpfedloraadapterloramodel-heterogeneouspersonalized
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

Federated learning (FL) is an emerging machine learning paradigm in which a central server coordinates multiple participants (clients) collaboratively to train on decentralized data. In practice, FL often faces statistical, system, and model heterogeneities, which inspires the field of Model-Heterogeneous Personalized Federated Learning (MHPFL). With the increased interest in adopting large language models (LLMs) in FL, the existing MHPFL methods cannot achieve acceptable computational and communication costs, while maintaining satisfactory model performance. To bridge this gap, we propose a novel and efficient model-heterogeneous personalized Federated learning framework based on LoRA tuning (pFedLoRA). Inspired by the popular LoRA method for fine-tuning pre-trained LLMs with a low-rank model (a.k.a., an adapter), we design a homogeneous small adapter to facilitate federated client's heterogeneous local model training with our proposed iterative training for global-local knowledge exchange. The homogeneous small local adapters are aggregated on the FL server to generate a global adapter. We theoretically prove the convergence of pFedLoRA. Extensive experiments on two benchmark datasets demonstrate that pFedLoRA outperforms six state-of-the-art baselines, beating the best method by 1.35% in test accuracy, 11.81 times computation overhead reduction and 7.41 times communication cost saving.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Concordia: Self-Improving Synthetic Tables for Federated LLMs

    cs.LG 2026-05 unverdicted novelty 7.0

    Concordia aligns synthetic table generation with federated validation utility via client-side utility scorers and group-relative policy optimization to improve LLM adaptation on non-IID tabular tasks.

  2. Adaptive Selection of LoRA Components in Privacy-Preserving Federated Learning

    cs.LG 2026-05 unverdicted novelty 7.0

    AS-LoRA adaptively chooses which LoRA factor to update per layer and round using a curvature-aware second-order score, eliminating reconstruction error floors and improving performance in DP federated learning.

  3. Shift-Dependent Asymmetry: Orthogonal Inverse Low-Rank Adaptation for Federated Medical Segmentation

    cs.CV 2026-06 unverdicted novelty 5.0

    Introduces IAT with module-specific personalization and orthogonality regularization to handle appearance and supervision shifts in federated medical segmentation.

  4. FedSDR: Federated Self-Distillation with Rectification

    cs.LG 2026-05 unverdicted novelty 5.0

    FedSDR augments federated self-distillation with dual LoRA streams (local smoothing and global rectification) to produce globally aligned, factually faithful models under statistical heterogeneity.

  5. Concordia: Self-Improving Synthetic Tables for Federated LLMs

    cs.LG 2026-05 unverdicted novelty 5.0

    Concordia aligns synthetic table generation with federated validation utility via client-level LoRA training, utility scorers, and outer GRPO refinement to boost performance over static synthetic baselines.

  6. FedProxy: Federated Fine-Tuning of LLMs via Proxy SLMs and Heterogeneity-Aware Fusion

    cs.LG 2026-04 unverdicted novelty 5.0

    FedProxy replaces weak adapters with a proxy SLM for federated LLM fine-tuning, outperforming prior methods and approaching centralized performance via compression, heterogeneity-aware aggregation, and training-free fusion.

  7. FediLoRA: Practical Federated Fine-Tuning of Foundation Models Under Missing-Modality Constraints

    cs.LG 2025-09 unverdicted novelty 4.0

    FediLoRA is a lightweight federated LoRA aggregation method that jointly mitigates missing modalities and heterogeneous ranks in collaborative fine-tuning of foundation models.

  8. A Survey on Foundation Models for Personalized Federated Intelligence

    cs.AI 2025-05 unverdicted novelty 3.0

    The survey introduces personalized federated intelligence (PFI) as a framework integrating federated learning and foundation models to support privacy-aware personalization of AI models.