Pith. sign in

REVIEW 10 cited by

Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2305.11414 v3 pith:AQ7RLOK7 submitted 2023-05-19 cs.LG cs.AIcs.CR

Sixing Yu , J. Pablo Muñoz , Ali Jannesari This is my paper · ORCID
classification cs.LGcs.AIcs.CR
keywords modelsdatafederatedlearningfoundationpotentialpre-trainingprivacy-preserving
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Foundation Models (FMs), such as LLaMA, BERT, GPT, ViT, and CLIP, have demonstrated remarkable success in a wide range of applications, driven by their ability to leverage vast amounts of data for pre-training. However, optimizing FMs often requires access to sensitive data, raising privacy concerns and limiting their applicability in many domains. In this paper, we propose the Federated Foundation Models (FFMs) paradigm, which combines the benefits of FMs and Federated Learning (FL) to enable privacy-preserving and collaborative learning across multiple end-users. We discuss the potential benefits and challenges of integrating FL into the lifespan of FMs, covering pre-training, fine-tuning, and application. We further outline potential future research avenues in FFM, including FFM pre-training, FFM fine-tuning, and federated prompt tuning, which allow the development of more personalized and context-aware models while ensuring data privacy. Moreover, we explore the possibility of continual/lifelong learning in FFMs, as increased computational power at the edge may unlock the potential for optimizing FMs using newly generated private data close to the data source. The proposed FFM concepts offer a flexible and scalable framework for training large language models in a privacy-preserving manner, setting the stage for subsequent advancements in both FM training and federated learning.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FedPIA -- Permuting and Integrating Adapters leveraging Wasserstein Barycenters for Finetuning Foundation Models in Multi-Modal Federated Learning

    cs.CV 2024-12 conditional novelty 6.0 of 10

    Permuting adapter neurons before averaging improves federated vision-language fine-tuning under heterogeneous medical clients compared to prior PEFT-FL baselines.

  2. F$^3$OCUS -- Federated Finetuning of Vision-Language Foundation Models with Optimal Client Layer Updating Strategy via Multi-objective Meta-Heuristics

    cs.CV 2024-11 conditional novelty 6.0 of 10

    F3OCUS combines per-client LNTK layer importance scores with server-side meta-heuristic optimization of layer diversity to improve federated fine-tuning of vision-language models for medical tasks, and releases the 70...

  3. Fed-HeLLo: Efficient Federated Foundation Model Fine-Tuning with Heterogeneous LoRA Allocation

    cs.LG 2025-06 conditional novelty 5.0 of 10

    Fed-HeLLo allocates different LoRA layers to clients of different resource levels using importance scores and geometric patterns, improving federated fine-tuning accuracy over random allocation baselines.

  4. Protocol Learning, Decentralized Frontier Risk and the No-Off Problem

    cs.LG 2024-12 conditional novelty 5.0 of 10

    The paper introduces Protocol Learning, a decentralized, incentivized training paradigm, and argues it could reduce frontier risk even as it creates the No-Off Problem.

  5. FedPhD: Federated Pruning with Hierarchical Learning of Diffusion Models

    cs.LG 2025-07 conditional novelty 4.0 of 10

    A hierarchical federated learning method with distribution-aware aggregation and structured pruning trains diffusion models under non-IID data with lower communication cost.

  6. Zero-Trust Foundation Models: A New Paradigm for Secure and Collaborative Artificial Intelligence for Internet of Things

    cs.CR 2025-05 conditional novelty 4.0 of 10

    The paper defines the ZTFM concept, identifies four zero-trust principles, reviews enabling technologies and threats, and lays out open research challenges for AI-driven IoT security.

  7. Federated Adapter on Foundation Models: An Out-Of-Distribution Approach

    cs.LG 2025-05 reject novelty 4.0 of 10

    FedOA regularizes personalized adapters toward the global model in feature space to improve OOD generalization in federated foundation models, but the proof is incomplete and the empirical gains are modest.

  8. Multifaceted User Modeling in Recommendation: A Federated Foundation Models Approach

    cs.IR 2024-12 conditional novelty 4.0 of 10

    MRFF, a federated sequential recommender with a group gating network and private user FFNs, improves CTR prediction over FedSASRec, FedHSTU, and FedLLaMA on three Kuai datasets.

  9. Personalized Federated Learning via Dual-Prompt Optimization and Cross Fusion

    cs.LG 2025-06 conditional novelty 3.0 of 10

    pFedDC combines global and local text and vision prompts with cross-attention fusion to personalize federated CLIP models under label and domain shift.

  10. A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning

    cs.LG 2025-04 conditional novelty 3.0 of 10

    A review that sorts recent federated-learning PEFT approaches into additive, selective, and reparameterized (LoRA-style) families and maps them onto NLP and vision applications.

Pith tools