REVIEW 14 cited by
Federated Optimization in Heterogeneous Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Federated Optimization in Heterogeneous Networks
read the original abstract
Federated Learning is a distributed learning paradigm with two key challenges that differentiate it from traditional distributed optimization: (1) significant variability in terms of the systems characteristics on each device in the network (systems heterogeneity), and (2) non-identically distributed data across the network (statistical heterogeneity). In this work, we introduce a framework, FedProx, to tackle heterogeneity in federated networks. FedProx can be viewed as a generalization and re-parametrization of FedAvg, the current state-of-the-art method for federated learning. While this re-parameterization makes only minor modifications to the method itself, these modifications have important ramifications both in theory and in practice. Theoretically, we provide convergence guarantees for our framework when learning over data from non-identical distributions (statistical heterogeneity), and while adhering to device-level systems constraints by allowing each participating device to perform a variable amount of work (systems heterogeneity). Practically, we demonstrate that FedProx allows for more robust convergence than FedAvg across a suite of realistic federated datasets. In particular, in highly heterogeneous settings, FedProx demonstrates significantly more stable and accurate convergence behavior relative to FedAvg---improving absolute test accuracy by 22% on average.
Forward citations
Cited by 14 Pith papers
-
TallyTrain: Communication-Efficient Federated Distillation
TallyTrain is a hard-label distillation protocol for federated learning that uses argmax transmission and optional sparse merges to match soft-label performance at up to 1000x lower communication cost.
-
FedFFT: Taming Client Drift in Federated SAM via Spectral Perturbation Filtering
Low-frequency components of client-side SAM perturbations carry most inter-client disagreement; high-pass filtering them yields more consistent federated updates and higher accuracy under non-IID data.
-
Federated Lightweight Fine-Tuning
A federated fine-tuning method transmits only 1,280 latent floats per round and reaches near-FedAvg accuracy by exploiting the exact averaging identity of affine mapping networks.
-
Exploring Cross-Client Memorization of Training Data in Large Language Models for Federated Learning
Framework quantifies intra- and inter-client memorization in FL LLMs, finding higher intra-client memorization influenced by decoding strategies, prefix length, and FL algorithms.
-
Adaptive Federated Optimization
Proposes federated adaptive optimizers (FedAdagrad, FedAdam, FedYogi) with convergence analysis for non-convex objectives under data heterogeneity and reports empirical gains over FedAvg.
-
FedProIn: Mitigating Client Drift for Learnable Prototypes in Federated Medical Imaging
FedProIn mitigates client drift in federated medical imaging by combining multiple learnable prototypes, feature-divergence and prototype-contrastive losses, and normalized influence aggregation, outperforming baselin...
-
Echelon: Auditable Aggregate-Only Language-Model Adaptation Across Privacy Boundaries
Echelon enables auditable aggregate-only adaptation of language models across privacy boundaries by training locally and sharing only boundary-level aggregates, achieving competitive performance in 1B LoRA experiments.
-
Choose Wisely and Privately: Proactive Client Selection for Fair and Efficient Federated Learning
Proactive client selection in federated learning via differentially private mutual information and simulated annealing to optimize Potential Federation Loss for utility and fairness.
-
Choose Wisely and Privately: Proactive Client Selection for Fair and Efficient Federated Learning
Proposes proactive client selection via differentially private mutual information and Potential Federation Loss optimized by simulated annealing to achieve faster, fairer, and more accurate federated models than unifo...
-
Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
Non-identical data distributions degrade federated averaging accuracy on visual classification, but server momentum raises CIFAR-10 accuracy from 30.1% to 76.9% in the most skewed regimes.
-
XAI-SOH-FL: Enhancing SOH-FL with Adaptive Aggregation and Explainable AI for Intrusion Detection in Heterogeneous IoT
XAI-SOH-FL extends SOH-FL with adaptive gamma via Bayesian optimization and SHAP interpretability, reporting 94.12% accuracy and 0.92 F1 on CICIDS2017 while converging faster than baseline.
-
Evaluating Federated Learning approaches for mammography under breast density heterogeneity
FedAvg matches centralized training accuracy on mammography data split by breast density heterogeneity, showing standard FL can handle this clinical variation without special fixes.
-
The Impact of Federated Learning on Distributed Remote Sensing Archives
FedProx outperforms FedAvg for deeper models under data heterogeneity, BSP reaches near-centralized accuracy at high communication cost, and LeNet gives the best accuracy-communication trade-off on the UC Merced dataset.
-
Will LLMs Scaling Hit the Wall? Breaking Barriers via Distributed Resources on Massive Edge Devices
Position paper claiming that distributed training across massive edge devices can overcome data depletion and centralized compute monopolies in LLM scaling.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.