Pith. sign in

REVIEW 51 cited by

On the Convergence of FedAvg on Non-IID Data

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1907.02189 v4 pith:BL7FECYS submitted 2019-07-04 stat.ML cs.LGmath.OC

Xiang Li , Kaixuan Huang , Wenhao Yang , Shusen Wang , Zhihua Zhang This is my paper · ORCID
classification stat.MLcs.LGmath.OC
keywords convergencedatafedavgratedevicedeviceslearningnon-iid
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
abstract

Federated learning enables a large amount of edge computing devices to jointly learn a model without data sharing. As a leading algorithm in this setting, Federated Averaging (\texttt{FedAvg}) runs Stochastic Gradient Descent (SGD) in parallel on a small subset of the total devices and averages the sequences only once in a while. Despite its simplicity, it lacks theoretical guarantees under realistic settings. In this paper, we analyze the convergence of \texttt{FedAvg} on non-iid data and establish a convergence rate of $\mathcal{O}(\frac{1}{T})$ for strongly convex and smooth problems, where $T$ is the number of SGDs. Importantly, our bound demonstrates a trade-off between communication-efficiency and convergence rate. As user devices may be disconnected from the server, we relax the assumption of full device participation to partial device participation and study different averaging schemes; low device participation rate can be achieved without severely slowing down the learning. Our results indicate that heterogeneity of data slows down the convergence, which matches empirical observations. Furthermore, we provide a necessary condition for \texttt{FedAvg} on non-iid data: the learning rate $\eta$ must decay, even if full-gradient is used; otherwise, the solution will be $\Omega (\eta)$ away from the optimal.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 51 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 1,012 citations worldwide. Full citation record

  1. FedMeNF: Privacy-Preserving Federated Meta-Learning for Neural Fields

    cs.LG 2025-08 conditional novelty 7.0 of 10

    MDIR detects LLM weight homology from embedding matrices alone using polar decomposition and permutation matching, achieving perfect AUC and accuracy on LeaFBench and reconstructing layer-level transformations.

  2. FedACT: Federated Adaptive Coordinate Trust Modulation for Robust Transformer Training under Data Heterogeneity

    cs.LG 2026-07 conditional novelty 6.5 of 10

    Global-aware coordinate trust modulation after corrected AdamW updates improves federated Transformer and LLM training under data heterogeneity over strong adaptive baselines.

  3. Robust Federated Learning Under Real-World Client Churn

    cs.LG 2026-07 conditional novelty 6.0 of 10

    FeLiX reduces wall-clock time-to-target accuracy in federated learning by up to 2.37x using lightweight availability tiers, fresh-utility client selection, and informativeness-aware aggregation without requiring oracu...

  4. Achieving Linear Speedup for Composite Federated Learning

    cs.LG 2026-02 conditional novelty 6.0 of 10

    FedNMap provably achieves communication complexity O(1/(nQ ε⁴)) for nonconvex composite federated learning with nonsmooth regularizers, the first such linear-speedup guarantee.

  5. Degree of Staleness-Aware Data Updating in Federated Learning

    cs.LG 2025-08 unverdicted novelty 6.0 of 10

    DUFL is a payment-based incentive mechanism that jointly balances data staleness and data volume via a Stackelberg game and derives a closed-form optimal client data update strategy.

  6. Federated Learning Enhanced by Feature Reconstruction for Semantic Communication Module Updates of Agents

    eess.SP 2025-08 conditional novelty 6.0 of 10

    FedSFR lets poorly connected clients upload compact encoder features, which the server uses in a feature-reconstruction step, improving the stability and efficiency of federated training for vector-quantized image sem...

  7. Towards Collaborative Fairness in Federated Learning Under Imbalanced Covariate Shift

    cs.LG 2025-07 conditional novelty 6.0 of 10

    FedAKD improves collaborative fairness under imbalanced covariate shift by distilling only each client's correctly predicted samples into the global model.

  8. Adaptive collaboration for online personalized distributed learning with heterogeneous clients

    stat.ML 2025-07 conditional novelty 6.0 of 10

    An adaptive gradient-similarity criterion dynamically selects collaboration partners in personalized federated learning, provably recovering the oracle-optimal sample complexity of All-for-one without knowing client h...

  9. FedCLAM: Client Adaptive Momentum with Foreground Intensity Matching for Federated Medical Image Segmentation

    eess.IV 2025-06 conditional novelty 6.0 of 10

    FedCLAM improves federated medical segmentation by weighting client updates with validation-loss progress and aligning predicted and ground-truth foreground intensities.

  10. PNCS:Power-Norm Cosine Similarity for Diverse Client Selection in Federated Learning

    cs.LG 2025-06 conditional novelty 6.0 of 10

    A client selection rule based on L4-norm cosine similarity and a rotation queue improves federated learning accuracy modestly over three baselines on CIFAR-10 and Fashion-MNIST.

  11. Mitigating Non-IID Drift in Zeroth-Order Federated LLM Fine-Tuning with Transferable Sparsity

    cs.LG 2025-06 reject novelty 6.0 of 10

    Sparse zeroth-order federated fine-tuning with shared seeds and GradIP-based early stopping matches or beats full-parameter ZO while using far less communication.

  12. Adaptive Federated LoRA in Heterogeneous Wireless Networks with Independent Sampling

    cs.LG 2025-05 reject novelty 6.0 of 10

    An adaptive federated LoRA scheduler jointly tunes client sampling probabilities and LoRA sketching ratios to minimize wall-clock fine-tuning time, with experiments reporting 2.8 to 4.2 times speedups over prior methods.

  13. Hybrid Batch Normalisation: Resolving the Dilemma of Batch Normalisation in Federated Learning

    cs.LG 2025-05 conditional novelty 6.0 of 10

    Hybrid Batch Normalisation improves federated learning accuracy by combining local batch statistics with global statistics, and derives those global statistics from the pre-update global model.

  14. DeCAF: Decentralized Consensus-And-Factorization for Low-Rank Adaptation of Foundation Models

    cs.LG 2025-05 reject novelty 6.0 of 10

    A truncated-SVD consensus step for decentralized LoRA is claimed to reach O(1/sqrt T) convergence, matching decentralized SGD, with supporting CLIP and LLAMA2-7B experiments.

  15. Exploit Gradient Skewness to Circumvent Byzantine Defenses for Federated Learning

    cs.LG 2025-02 conditional novelty 6.0 of 10

    A skew-aware Byzantine attack, STRIKE, exploits the tendency of honest non-IID gradients to form dense clusters away from their mean, hiding malicious gradients inside the cluster.

  16. Federated Deep Subspace Clustering

    cs.LG 2024-12 reject novelty 6.0 of 10

    A federated deep subspace clustering framework with a shared encoder and private self-expressive layer is claimed to outperform centralized baselines, but its loss function is ill-defined and its evaluation lacks fede...

  17. FedGAT: A Privacy-Preserving Federated Approximation Algorithm for Graph Attention Networks

    cs.LG 2024-12 reject novelty 6.0 of 10

    FedGAT uses a truncated Chebyshev polynomial approximation of the GAT attention score to enable federated GAT training with a single pre-communication round.

  18. Channel-Adaptive Robust Aggregation for Over-the-Air Federated Learning in Heterogeneous Networks

    cs.LG 2026-07 conditional novelty 5.0 of 10

    Adaptive channel- and readiness-triggered aggregation with dual-purpose precoding yields O(1/T) OTA-FL convergence and higher CIFAR accuracy under stragglers and noise than fixed-schedule baselines.

  19. Enhancing Model Privacy in Federated Learning with Random Masking and Quantization

    cs.LG 2025-08 reject novelty 5.0 of 10

    FedQSN hides part of the server model with random masks and quantizes the remainder to give clients a degraded proxy, reporting a large global-vs-proxy performance gap with modest loss in the final global model.

  20. When Secure Aggregation Falls Short: Achieving Long-Term Privacy in Asynchronous Federated Learning for LEO Satellite Networks

    cs.CR 2025-08 conditional novelty 5.0 of 10

    Long-term privacy leakage in asynchronous federated learning over LEO satellite networks is kept bounded by fixed jointly-visible satellite partitions used with secure aggregation.

  21. Discerning and quantifying high frequency activities in EEG under normal and epileptic conditions

    nlin.CD 2025-08 unverdicted novelty 5.0 of 10

    The authors claim that gamma-band EEG activity at roughly 50 Hz and 76 Hz shows distinct bi-stable dynamics in epilepsy, enabling 94-95% accuracy in separating normal from ictal states.

  22. On the Convergence and Complexity of Proximal Gradient and Accelerated Proximal Gradient Methods under Adaptive Gradient Estimation

    math.OC 2025-07 conditional novelty 5.0 of 10

    Adaptive gradient accuracy yields optimal iteration complexity for (accelerated) proximal gradient methods with biased estimates, and query complexity claims for unbiased estimates.

  23. Federated Learning for Commercial Image Sources

    cs.CV 2025-07 conditional novelty 5.0 of 10

    The authors present a new 31-class, 8-source image classification dataset for federated learning and show that Fed-Cyclic and Fed-Star beat FedAvg and RingFed on it.

  24. SFedKD: Sequential Federated Learning with Discrepancy-Aware Multi-Teacher Knowledge Distillation

    cs.LG 2025-07 conditional novelty 5.0 of 10

    SFedKD uses discrepancy-weighted multi-teacher knowledge distillation and greedy teacher selection to reduce catastrophic forgetting in sequential federated learning.

  25. Cooperative Gradient Coding

    cs.DC 2025-07 reject novelty 5.0 of 10

    A gradient coding scheme for federated learning that avoids dataset duplication and adds a fallback decoder that recovers the model from incomplete messages.

  26. Avoid Forgetting by Preserving Global Knowledge Gradients in Federated Learning with Non-IID Data

    cs.LG 2025-05 conditional novelty 5.0 of 10

    FedProj combines client-side gradient projection onto a global-knowledge loss with server-side ensemble distillation and outperforms existing federated learning methods on non-IID image and NLP benchmarks.

  27. Federated Learning Strategies for Coordinated Beamforming in Multicell ISAC

    eess.SP 2025-01 conditional novelty 5.0 of 10

    Two federated learning frameworks for multicell ISAC beamforming are proposed: a VFL approach with a central server and an HFL approach with a fully distributed leakage-penalty loss function.

  28. Distributed Quasi-Newton Method for Fair and Fast Federated Learning

    cs.LG 2025-01 reject novelty 5.0 of 10

    DQN-Fed updates a global model in a direction that makes every client's loss decrease at a rate tied to its local quasi-Newton step, with claimed linear-quadratic convergence.

  29. pFedWN: A Personalized Federated Learning Framework for D2D Wireless Networks with Heterogeneous Data

    cs.LG 2025-01 reject novelty 5.0 of 10

    pFedWN combines channel-aware neighbor selection with an EM-based model weighting step to personalize federated learning over server-free D2D wireless networks.

  30. AutoDFL: A Scalable and Automated Reputation-Aware Decentralized Federated Learning

    cs.DC 2025-01 conditional novelty 5.0 of 10

    AutoDFL couples zk-rollups with a decentralized oracle network to scale blockchain-based federated learning and automate reputation scoring, reporting a projected 3000 TPS and up to 20x gas reduction.

  31. Federated Dropout: Convergence Analysis and Resource Allocation

    cs.LG 2024-12 conditional novelty 5.0 of 10

    Federated Dropout's convergence slowdown is tied to a gamma/(1-gamma) gradient-variance term, and dropout rate and bandwidth can be jointly optimized to shorten training latency.

  32. Accelerating Energy-Efficient Federated Learning in Cell-Free Networks with Adaptive Quantization

    cs.LG 2024-12 conditional novelty 5.0 of 10

    An adaptive element-wise quantization scheme and power allocation method for federated learning over cell-free massive MIMO improves test accuracy by up to 7-19% under equal energy and latency budgets.

  33. Enhancing Federated Graph Learning via Adaptive Fusion of Structural and Node Characteristics

    cs.LG 2024-12 reject novelty 5.0 of 10

    FedGCF fuses clustered structural models and selected node-feature models with a bandit-tuned ratio, claiming accuracy and communication improvements in federated graph classification, though its test-set-based tuning...

  34. Exploiting Label Skewness for Spiking Neural Networks in Federated Learning

    cs.LG 2024-12 conditional novelty 5.0 of 10

    FedLEC combines per-client logit calibration with missing-label knowledge distillation to improve federated spiking neural network accuracy under label skew, claiming about 11.59% average gains over eight baselines.

  35. Code Review Automation Via Multi-task Federated LLM -- An Empirical Study

    cs.SE 2024-12 conditional novelty 5.0 of 10

    Federated cumulative fine-tuning of LLaMA-3 with a separate classifier is the best multi-task setup for code review automation, but gains are small and sequential training causes catastrophic forgetting.

  36. Benchmarking Federated Learning for Semantic Datasets: Federated Scene Graph Generation

    cs.CV 2024-12 conditional novelty 5.0 of 10

    A clustering-based process for creating federated learning benchmarks with controllable semantic heterogeneity, demonstrated on panoptic scene graph generation and CelebA.

  37. Mind the Cost of Scaffold! Benign Clients May Even Become Accomplices of Backdoor Attack

    cs.LG 2024-11 conditional novelty 5.0 of 10

    BadSFL is a backdoor attack that exploits Scaffold's control variate to make benign clients amplify and preserve a planted backdoor in non-IID federated learning.

  38. How to Defend Against Large-scale Model Poisoning Attacks in Federated Learning: A Vertical Solution

    cs.LG 2024-11 conditional novelty 5.0 of 10

    VERT defends federated learning against large-scale model poisoning by selecting, in each round, the users whose gradients best match an autoregressive prediction from each user's own history.

  39. FedLBW: A Loss-Based Weighting Strategy for Federated Learning on Non-IID Data in Wireless Networks

    cs.AI 2026-08 conditional novelty 4.0 of 10

    A federated server that weights client updates by the inverse of each model's validation loss on a small proxy dataset outperforms data-size weighting, but its convergence proof assumes the weight concentration that t...

  40. Decentralized Federated Learning by Partial Message Exchange

    cs.LG 2026-03 reject novelty 4.0 of 10

    PaME combines random coordinate exchange with a growing-penalty schedule, claiming linear convergence under two mild assumptions, but its key parameter condition is never satisfied by its own experiments and the limit...

  41. Variational Gaussian Mixture Manifold Models for Client-Specific Federated Personalization

    cs.LG 2025-09 conditional novelty 4.0 of 10

    VGM2 personalizes federated learning by exchanging compact Bayesian summaries of same-class and different-class distance distributions instead of model weights.

  42. A Systematic Survey of Model Extraction Attacks and Defenses: State-of-the-Art and Perspectives

    cs.CR 2025-08 conditional novelty 4.0 of 10

    The paper classifies model extraction attacks and defenses into attack, defense, and computing environment categories and surveys their current state.

  43. PPFL-RDSN: Privacy-Preserving Federated Learning-based Residual Dense Spatial Networks for Encrypted Lossy Image Reconstruction

    cs.LG 2025-06 reject novelty 4.0 of 10

    A federated, privacy-preserving RDSN framework for encrypted image reconstruction whose local differential privacy mechanism is not actually differentially private because it releases low-frequency DCT coefficients wi...

  44. Communication Efficient Adaptive Model-Driven Quantum Federated Learning

    cs.LG 2025-06 conditional novelty 4.0 of 10

    mdQFL cuts quantum federated learning communication cost by training one device per cluster, while reporting maintained server accuracy and improved local accuracy.

  45. RoadFed: A Multimodal Federated Learning System for Improving Road Safety

    cs.CE 2025-02 reject novelty 4.0 of 10

    A multimodal federated learning system with quantization and local differential privacy is reported to detect road hazards at 96.42% accuracy with 0.035 s latency and up to 1000x lower communication cost than baselines.

  46. Incentive-Compatible Federated Learning with Stackelberg Game Modeling

    cs.LG 2025-01 reject novelty 4.0 of 10

    FLamma claims to balance fairness and accuracy in federated learning via a Stackelberg game with an adaptive decay factor, but the theory has derivation errors and the experiments use fixed local epochs.

  47. Fair Distributed Machine Learning with Imbalanced Data as a Stackelberg Evolutionary Game

    cs.LG 2024-12 reject novelty 4.0 of 10

    The paper proposes DSWM and ASWM, two Stackelberg-game-inspired algorithms that reweight node contributions in federated learning, and reports modest AUC gains for small-data nodes on three MedMNIST datasets.

  48. Federated Learning and RAG Integration: A Scalable Approach for Medical Large Language Models

    cs.CL 2024-12 conditional novelty 4.0 of 10

    In the authors' experiments, federated fine-tuned medical LLMs answered questions more factually and semantically similar to ground truth when a retrieval step was added, although no error bars or code accompany the results.

  49. Overview of AI and Communication for 6G Network: Fundamentals, Challenges, and Future Research Opportunities

    cs.NI 2024-12 conditional novelty 3.0 of 10

    This overview paper structures the convergence of AI and 6G into three stages and proposes a Quality of AI Service framework for measuring AI services in future networks.

  50. FedMetaMed: Federated Meta-Learning for Personalized Medication in Distributed Healthcare Systems

    cs.AI 2024-12 reject novelty 3.0 of 10

    FedMetaMed is claimed to reach 87% average F1 and 90% AUC across four dermoscopy clients, outperforming FedBN, FedAvg, and MetaFed in the reported tables.

  51. Encoded Spatial Attribute in Multi-Tier Federated Learning

    cs.LG 2025-01 reject novelty 2.0 of 10

    An under-specified N-tier federated learning design with spatial encoding reports 75.62% and 89.52% accuracy on two geospatial datasets, but the encoding mechanism is never defined.

Pith tools