REVIEW 5 cited by
Advances and Open Challenges in Federated Foundation Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The integration of Foundation Models (FMs) with Federated Learning (FL) presents a transformative paradigm in Artificial Intelligence (AI). This integration offers enhanced capabilities, while addressing concerns of privacy, data decentralization and computational efficiency. This paper provides a comprehensive survey of the emerging field of Federated Foundation Models (FedFM), elucidating their synergistic relationship and exploring novel methodologies, challenges, and future directions that the FL research field needs to focus on in order to thrive in the age of FMs. A systematic multi-tiered taxonomy is proposed, categorizing existing FedFM approaches for model training, aggregation, trustworthiness, and incentivization. Key challenges, including how to enable FL to deal with high complexity of computational demands, privacy considerations, contribution evaluation, and communication efficiency, are thoroughly discussed. Moreover, this paper explores the intricate challenges of communication, scalability and security inherent in training/fine-tuning FMs via FL. It highlights the potential of quantum computing to revolutionize the processes of training, inference, optimization and security. This survey also introduces the implementation requirement of FedFM and some practical FedFM applications. It highlights lessons learned with a clear understanding of our findings for FedFM. Finally, this survey not only provides insights into the current state and challenges of FedFM, but also offers a blueprint for future research directions, emphasizing the need for developing trustworthy solutions. It serves as a foundational guide for researchers and practitioners interested in contributing to this interdisciplinary and rapidly advancing field.
Forward citations
Cited by 5 Pith papers
-
Flexible Personalized Split Federated Learning for On-Device Fine-Tuning of Foundation Models
FlexP-SFL fine-tunes foundation models on resource-constrained devices through personalized split learning without parameter aggregation, improving accuracy and cutting wall-clock time and communication versus federat...
-
FedHL: Federated Learning for Heterogeneous Low-Rank Adaptation via Unbiased Aggregation
FedHL aggregates heterogeneous LoRA updates against a full-rank global baseline and claims O(1/sqrt T) convergence, with small gains on three LLM fine-tuning datasets.
-
LAARA: Layer-Aware Adaptive Rank Allocation for Parameter-Efficient Fine-Tuning
LAARA allocates LoRA ranks per layer from diagonal Fisher (gradient-based) estimates, reporting improved accuracy with fewer trainable parameters on GLUE and MathInstruct.
-
Foundational Models and Federated Learning: Survey, Taxonomy, Challenges and Practical Insights
A survey that maps methods for combining foundation models with federated learning into a training-customization-deployment taxonomy with practical ratings.
-
Towards Group Fairness with Multiple Sensitive Attributes in Federated Foundation Models
EFF-DVP extends FF-DVP to multiple sensitive attributes with parallel demographic prompts and claims that larger causal effects of an attribute on the label predict smaller fairness improvements.
Discussion (0). Sign in to comment.