REVIEW 14 cited by
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
read the original abstract
Federated Averaging (FedAvg) has emerged as the algorithm of choice for federated learning due to its simplicity and low communication cost. However, in spite of recent research efforts, its performance is not fully understood. We obtain tight convergence rates for FedAvg and prove that it suffers from `client-drift' when the data is heterogeneous (non-iid), resulting in unstable and slow convergence. As a solution, we propose a new algorithm (SCAFFOLD) which uses control variates (variance reduction) to correct for the `client-drift' in its local updates. We prove that SCAFFOLD requires significantly fewer communication rounds and is not affected by data heterogeneity or client sampling. Further, we show that (for quadratics) SCAFFOLD can take advantage of similarity in the client's data yielding even faster convergence. The latter is the first result to quantify the usefulness of local-steps in distributed optimization.
Forward citations
Cited by 14 Pith papers
-
TallyTrain: Communication-Efficient Federated Distillation
TallyTrain is a hard-label distillation protocol for federated learning that uses argmax transmission and optional sparse merges to match soft-label performance at up to 1000x lower communication cost.
-
FedFFT: Taming Client Drift in Federated SAM via Spectral Perturbation Filtering
Low-frequency components of client-side SAM perturbations carry most inter-client disagreement; high-pass filtering them yields more consistent federated updates and higher accuracy under non-IID data.
-
Sarus: Privacy-Preserving Multi-Vendor Perception Fusion via Homomorphic Encryption
Sarus is an HE-based framework that fuses vendors' Gaussian-moment detection summaries in encrypted form, with linear-scaling server fusion and near-identical output to plaintext fusion.
-
Federated Lightweight Fine-Tuning
A federated fine-tuning method transmits only 1,280 latent floats per round and reaches near-FedAvg accuracy by exploiting the exact averaging identity of affine mapping networks.
-
SGD at the Edge of Stability: Stochastic Stabilization with Large Learning Rates
SGD on multiclass cross-entropy loss alternates between curvature-driven oscillations and stable regimes but self-stabilizes to enable best-iterate convergence with large learning rates for linear and two-layer models.
-
FedEHR-Gen: Federated Synthetic Time-Series EHR Generation via Latent Space Alignment and Distribution-Aware Aggregation
FedEHR-Gen is a federated two-stage autoencoder plus TCVAE system that aligns latent spaces via layer-wise matching and uses distribution-aware aggregation to produce synthetic EHR time-series data matching centralize...
-
Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity
Rescaled ASGD recovers convergence to the true global objective by rescaling worker stepsizes proportional to computation times, matching the known time lower bound in the leading term under non-convex smoothness and ...
-
Adaptive Federated Optimization
Proposes federated adaptive optimizers (FedAdagrad, FedAdam, FedYogi) with convergence analysis for non-convex objectives under data heterogeneity and reports empirical gains over FedAvg.
-
Enhancing Federated Quadruplet Learning: Stochastic Client Selection and Embedding Stability Analysis
FedQuad uses quadruplet constraints and stochastic client selection in federated learning to reduce representation misalignment and improve generalization on heterogeneous data.
-
Subspace Optimization for Efficient Federated Learning under Heterogeneous Data
SSF enables efficient federated learning under heterogeneous data by optimizing in a low-dimensional subspace with projected corrections and backfill updates, achieving a non-asymptotic convergence rate of order O~(1/...
-
Aggregating Low Rank Adapters in Federated Fine-tuning
Proposes and benchmarks a new aggregation technique for LoRA adapters in federated fine-tuning against existing methods on GLUE tasks.
-
FoggyTrust: Robust Federated Learning with Hierarchical Trust Networks
FoggyTrust is a hierarchical extension of FLTrust that localizes trust computation to fog nodes and combines it with heterogeneity-aware optimizers, reporting over 50% gains on CIFAR-10 under Krum and Trim attacks.
-
FedReLa: Imbalanced Federated Learning via Re-Labeling
FedReLa applies feature-dependent re-labeling to correct biased global decision boundaries in imbalanced federated learning without requiring knowledge of the global class distribution.
-
DP-FedLoRA: Privacy-Enhanced Federated Fine-Tuning for On-Device Large Language Models
DP-FedLoRA clips and adds Gaussian noise to per-client LoRA matrices in federated LLM fine-tuning, claiming unbiased updates and bounded variance, but the privacy calibration and experiments have significant gaps.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.