Pith. sign in

REVIEW 20 cited by

When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directions

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.15546 v3 pith:GHLH2HBG submitted 2023-06-27 cs.LG cs.AIcs.DC

Weiming Zhuang , Chen Chen , Jingtao Li , Chaochao Chen , Yaochu Jin , Lingjuan Lyu This is my paper · ORCID
classification cs.LGcs.AIcs.DC
keywords datachallengesfoundationfuturelearningcollaborativedevelopmentdirections
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The intersection of Foundation Model (FM) and Federated Learning (FL) presents a unique opportunity to unlock new possibilities for real-world applications. On the one hand, FL, as a collaborative learning paradigm, help address challenges in FM development by expanding data availability, enabling computation sharing, facilitating the collaborative development of FMs, tackling continuous data update, avoiding FM monopoly, response delay and FM service down. On the other hand, FM, equipped with pre-trained knowledge and exceptional performance, can serve as a robust starting point for FL. It can also generate synthetic data to enrich data diversity and enhance overall performance of FL. Meanwhile, FM unlocks new sharing paradigm and multi-task and multi-modality capabilities for FL. By examining the interplay between FL and FM, this paper presents the motivations, challenges, and future directions of empowering FL with FM and empowering FM with FL. We hope that this work provides a good foundation to inspire future research efforts to drive advancements in both fields.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 20 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Personalized Federated Learning via Variance-Aware Nonparametric Empirical Bayes

    stat.ML 2026-08 conditional novelty 7.0 of 10

    VANEB generalizes nonparametric empirical Bayes to parameter-dependent noise and uses it to personalize federated models by shrinking local estimates toward a learned population prior.

  2. FM$^2$: Unified Federated Foundation Models for Heterogeneous Multimodal Medical Imaging

    cs.CV 2026-07 reject novelty 6.0 of 10

    FM² uses dual mixture-of-experts (per-class local, per-modality shared) with a proximal alignment regularizer to train federated medical imaging models across overlapped and disjoint modality settings, reporting consi...

  3. Flexible Personalized Split Federated Learning for On-Device Fine-Tuning of Foundation Models

    cs.DC 2025-08 conditional novelty 6.0 of 10

    FlexP-SFL fine-tunes foundation models on resource-constrained devices through personalized split learning without parameter aggregation, improving accuracy and cutting wall-clock time and communication versus federat...

  4. Assortment of Attention Heads: Accelerating Federated PEFT with Head Pruning and Strategic Client Selection

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A federated fine-tuning method prunes 90% of attention heads, weights updates by attention importance, and selects clients by loss gap, cutting communication 1.8x and training compute 3.9x with under 2% accuracy drop.

  5. FedHL: Federated Learning for Heterogeneous Low-Rank Adaptation via Unbiased Aggregation

    cs.LG 2025-05 reject novelty 6.0 of 10

    FedHL aggregates heterogeneous LoRA updates against a full-rank global baseline and claims O(1/sqrt T) convergence, with small gains on three LLM fine-tuning datasets.

  6. Look Back for More: Harnessing Historical Sequential Updates for Personalized Federated Adapter Tuning

    cs.LG 2025-01 conditional novelty 6.0 of 10

    pFedSeq trains a server-side Mamba sequence model on clients' historical adapter updates to generate personalized adapter calibrations, improving personalized federated adapter tuning.

  7. FedPIA -- Permuting and Integrating Adapters leveraging Wasserstein Barycenters for Finetuning Foundation Models in Multi-Modal Federated Learning

    cs.CV 2024-12 conditional novelty 6.0 of 10

    Permuting adapter neurons before averaging improves federated vision-language fine-tuning under heterogeneous medical clients compared to prior PEFT-FL baselines.

  8. Federated Foundation Models on Heterogeneous Time Series

    cs.LG 2024-12 conditional novelty 6.0 of 10

    FFTS is a federated pretraining framework with a timescale-aware mixture-of-experts module that trains a time series foundation model from scratch across heterogeneous, non-shared datasets.

  9. Foundational values for foundation models

    cs.CY 2026-08 conditional novelty 5.0 of 10

    A Socratic analysis of research values produces a network of reasons for using or abstaining from foundation models in medical imaging.

  10. Fed-HeLLo: Efficient Federated Foundation Model Fine-Tuning with Heterogeneous LoRA Allocation

    cs.LG 2025-06 conditional novelty 5.0 of 10

    Fed-HeLLo allocates different LoRA layers to clients of different resource levels using importance scores and geometric patterns, improving federated fine-tuning accuracy over random allocation baselines.

  11. Multi-Modal Multi-Task Federated Foundation Models for Next-Generation Extended Reality Systems: Towards Privacy-Preserving Distributed Intelligence in AR/VR/MR

    cs.LG 2025-06 conditional novelty 5.0 of 10

    The paper proposes M3T federated foundation models (FedFMs) as a privacy-preserving architecture for XR and codifies the key challenges as the SHIFT dimensions.

  12. Asymmetrical Reciprocity-based Federated Learning for Resolving Disparities in Medical Diagnosis

    cs.LG 2024-12 conditional novelty 5.0 of 10

    A cross-silo federated learning method uses one-time foundation-model API queries on public data and asymmetric dual knowledge distillation to improve small, data-poor medical clients.

  13. Protocol Learning, Decentralized Frontier Risk and the No-Off Problem

    cs.LG 2024-12 conditional novelty 5.0 of 10

    The paper introduces Protocol Learning, a decentralized, incentivized training paradigm, and argues it could reduce frontier risk even as it creates the No-Off Problem.

  14. FEDEXCHANGE: Bridging the Domain Gap in Federated Object Detection for Free

    cs.LG 2025-09 conditional novelty 4.0 of 10

    FEDEXCHANGE improves cross-domain federated object detection by server-side clustering and exchanging client decoder models, achieving higher mAP in some domains at no extra local compute.

  15. BetaWeb: Towards a Blockchain-enabled Trustworthy Agentic Web

    cs.MA 2025-08 unverdicted novelty 4.0 of 10

    BetaWeb promises a blockchain-enabled trustworthy agentic web, but the submitted manuscript body is a different mining-robot paper, leaving the proposal without supporting evidence.

  16. Scaling Decentralized Learning with FLock

    cs.LG 2025-07 reject novelty 4.0 of 10

    FLock claims the first secure decentralized fine-tuning of a 70B-class LLM, but the experiments omit the validator mechanism and compare against weak baselines.

  17. FedPhD: Federated Pruning with Hierarchical Learning of Diffusion Models

    cs.LG 2025-07 conditional novelty 4.0 of 10

    A hierarchical federated learning method with distribution-aware aggregation and structured pruning trains diffusion models under non-IID data with lower communication cost.

  18. Towards Group Fairness with Multiple Sensitive Attributes in Federated Foundation Models

    cs.LG 2025-06 conditional novelty 4.0 of 10

    EFF-DVP extends FF-DVP to multiple sensitive attributes with parallel demographic prompts and claims that larger causal effects of an attribute on the label predict smaller fairness improvements.

  19. Multifaceted User Modeling in Recommendation: A Federated Foundation Models Approach

    cs.IR 2024-12 conditional novelty 4.0 of 10

    MRFF, a federated sequential recommender with a group gating network and private user FFNs, improves CTR prediction over FedSASRec, FedHSTU, and FedLLaMA on three Kuai datasets.

  20. Federated Fine-Tuning of LLMs: Framework Comparison and Research Directions

    cs.LG 2025-01 conditional novelty 3.0 of 10

    A comparative analysis of three federated LLM fine-tuning frameworks shows FedLLMs achieve highest accuracy, KD-FedLLMs highest client computation, and Split-FedLLMs highest communication overhead in a GPT-2/Banking77...

Pith tools