Pith. sign in

REVIEW 10 cited by

FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.10049 v1 pith:WGBUCCVU submitted 2023-10-16 cs.LG cs.AI

FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Models

Tao Fan , Yan Kang , Guoqiang Ma , Weijing Chen , Wenbin Wei , Lixin Fan , Qiang Yang This is my paper
classification cs.LG cs.AI
keywords fate-llmlargellmslanguagemodelstrainingfederatedfedllm
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

Large Language Models (LLMs), such as ChatGPT, LLaMA, GLM, and PaLM, have exhibited remarkable performances across various tasks in recent years. However, LLMs face two main challenges in real-world applications. One challenge is that training LLMs consumes vast computing resources, preventing LLMs from being adopted by small and medium-sized enterprises with limited computing resources. Another is that training LLM requires a large amount of high-quality data, which are often scattered among enterprises. To address these challenges, we propose FATE-LLM, an industrial-grade federated learning framework for large language models. FATE-LLM (1) facilitates federated learning for large language models (coined FedLLM); (2) promotes efficient training of FedLLM using parameter-efficient fine-tuning methods; (3) protects the intellectual property of LLMs; (4) preserves data privacy during training and inference through privacy-preserving mechanisms. We release the code of FATE-LLM at https://github.com/FederatedAI/FATE-LLM to facilitate the research of FedLLM and enable a broad range of industrial applications.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. TIGER: Inverting Transformer Gradients via Embedding-Subspace Distance Optimization

    cs.CR 2026-06 unverdicted novelty 7.0

    TIGER turns the low-rank attention gradient subspace into a differentiable objective for continuous embedding optimization, improving reconstruction quality and robustness over prior discrete token tests especially un...

  2. Concordia: Self-Improving Synthetic Tables for Federated LLMs

    cs.LG 2026-05 unverdicted novelty 7.0

    Concordia aligns synthetic table generation with federated validation utility via client-side utility scorers and group-relative policy optimization to improve LLM adaptation on non-IID tabular tasks.

  3. FedAttr: Towards Privacy-preserving Client-Level Attribution in Federated LLM Fine-tuning

    cs.CR 2026-05 unverdicted novelty 7.0

    FedAttr enables privacy-preserving client-level attribution of watermarked data in federated LLM fine-tuning via paired-subset differencing, differential scoring, and Stouffer combination, achieving 100% TPR and 0% FP...

  4. HermesHFL: Incentive-Compatible Hierarchical Federated Unlearning for Dynamic LLM Fine-Tuning

    cs.CE 2026-07 conditional novelty 6.0

    HermesHFL plus Neogen jointly optimize incentives, client–edge association, and gradient-ascent unlearning so hierarchical LoRA fine-tuning remains useful after clients leave and rejoin.

  5. FedSmoothLoRA: Toward Smoother and Faster Convergence in Federated Low-Rank Adaptation

    cs.CV 2026-05 unverdicted novelty 6.0

    FedSmoothLoRA improves federated LoRA fine-tuning by constructing local initializations from a round-matching matrix for cross-round continuity and a gradient-aligned matrix for client-specific guidance, yielding fast...

  6. Shift-Dependent Asymmetry: Orthogonal Inverse Low-Rank Adaptation for Federated Medical Segmentation

    cs.CV 2026-06 unverdicted novelty 5.0

    Introduces IAT with module-specific personalization and orthogonality regularization to handle appearance and supervision shifts in federated medical segmentation.

  7. Concordia: Self-Improving Synthetic Tables for Federated LLMs

    cs.LG 2026-05 unverdicted novelty 5.0

    Concordia aligns synthetic table generation with federated validation utility via client-level LoRA training, utility scorers, and outer GRPO refinement to boost performance over static synthetic baselines.

  8. FedProxy: Federated Fine-Tuning of LLMs via Proxy SLMs and Heterogeneity-Aware Fusion

    cs.LG 2026-04 unverdicted novelty 5.0

    FedProxy replaces weak adapters with a proxy SLM for federated LLM fine-tuning, outperforming prior methods and approaching centralized performance via compression, heterogeneity-aware aggregation, and training-free fusion.

  9. AI Safety Landscape for Large Language Models: Taxonomy, State-of-the-art, and Future Directions

    cs.AI 2024-08 unverdicted novelty 4.0

    The paper introduces a taxonomy of AI safety for LLMs organized into Trustworthy AI, Responsible AI, and Safe AI perspectives, accompanied by a review of state-of-the-art methods, challenges, and future directions.

  10. Will LLMs Scaling Hit the Wall? Breaking Barriers via Distributed Resources on Massive Edge Devices

    cs.DC 2025-03 unverdicted novelty 2.0

    Position paper claiming that distributed training across massive edge devices can overcome data depletion and centralized compute monopolies in LLM scaling.