Proposes a zero-overhead sensing-native over-the-air FL system that uses gradient autocorrelation for distance estimation and gradient statistics for cooperative localization, with a statistics-aware power and beamforming co-design.
hub
On the convergence of FedA vg on non-IID data
25 Pith papers cite this work. Polarity classification is still indexing.
25
Pith papers citing it
abstract
Federated learning enables a large amount of edge computing devices to jointly learn a model without data sharing. As a leading algorithm in this setting, Federated Averaging (\texttt{FedAvg}) runs Stochastic Gradient Descent (SGD) in parallel on a small subset of the total devices and averages the sequences only once in a while. Despite its simplicity, it lacks theoretical guarantees under realistic settings. In this paper, we analyze the convergence of \texttt{FedAvg} on non-iid data and establish a convergence rate of $\mathcal{O}(\frac{1}{T})$ for strongly convex and smooth problems, where $T$ is the number of SGDs. Importantly, our bound demonstrates a trade-off between communication-efficiency and convergence rate. As user devices may be disconnected from the server, we relax the assumption of full device participation to partial device participation and study different averaging schemes; low device participation rate can be achieved without severely slowing down the learning. Our results indicate that heterogeneity of data slows down the convergence, which matches empirical observations. Furthermore, we provide a necessary condition for \texttt{FedAvg} on non-iid data: the learning rate $\eta$ must decay, even if full-gradient is used; otherwise, the solution will be $\Omega (\eta)$ away from the optimal.
hub tools
citation-role summary
background 4
citation-polarity summary
roles
background 4polarities
background 4representative citing papers
First convergence analysis of DPO under federated and decentralized training, characterizing rates via client drift, communication frequency, preference heterogeneity, and graph spectral connectivity.
Semantic consensus on model outputs for public prompts enables federated LLM fine-tuning that matches parameter-aggregation baselines with orders-of-magnitude lower communication.
FeLiX reduces wall-clock time-to-target accuracy in federated learning by up to 2.37x using lightweight availability tiers, fresh-utility client selection, and informativeness-aware aggregation without requiring oracular knowledge of client availability.
Derives tractable optimal fair multi-class classifier and supplies in-processing and post-processing algorithms that converge to the accuracy-fairness Pareto frontier.
A device-partitioning bandwidth allocation policy for federated learning over IIoT networks that provably reduces total training time compared to any non-partitioning scheme.
Nonlinear kernel integration (NKI) with graph regularization enables accurate alignment of nonlinearly obfuscated decentralized data representations, outperforming linear methods on image classification.
Introduces FedHybrid and FedNewton for DP federated M-estimation, with finite-sample MSE bounds, minimax lower bound, and evaluations on vision datasets.
Rescaled ASGD recovers convergence to the true global objective by rescaling worker stepsizes proportional to computation times, matching the known time lower bound in the leading term under non-convex smoothness and bounded heterogeneity.
FedVSSAM mitigates flatness incompatibility in SAM-based federated learning by consistently using a variance-suppressed adjusted direction for local perturbation, descent, and global updates, with non-convex convergence guarantees.
ChainFed achieves memory-efficient private LLM fine-tuning on edge devices through sequential layer-by-layer adapter training with dynamic co-tuning, perceptive optimization, and adaptive starting point selection, improving accuracy by up to 46.46%.
Proposes federated adaptive optimizers (FedAdagrad, FedAdam, FedYogi) with convergence analysis for non-convex objectives under data heterogeneity and reports empirical gains over FedAvg.
QSplitFL is a DQN framework that selects split points in split federated learning from hardware metrics with a decayed loss-drop reward and committee voting, reporting faster convergence and higher accuracy than baselines on image classification tasks.
FIRMA introduces Fibonacci ring aggregation protocols for server-free federated learning that maintain private heads and achieve higher accuracy than FedAvg under label skew across multiple benchmarks and heterogeneity regimes.
FedGMI applies VAEs as density estimators in federated learning to infer mixture proportions of shared distributions for structured personalization under data heterogeneity.
FedPLT assigns client-specific model layers for training and matches or beats full-model federated learning accuracy with 71-82 percent fewer trainable parameters per client.
AdaBFL introduces a novel three-layer adaptive aggregation mechanism for Byzantine-robust federated learning that counters complex attacks, provides non-convex non-iid convergence guarantees, and shows superior performance in experiments.
SplitFT adapts cut-layer selection and reduces LoRA rank per client in federated split learning to improve efficiency and performance when fine-tuning LLMs on heterogeneous devices and data.
Relay-assisted partial aggregation with device grouping, relay selection, and SPCA-based power optimization reduces energy consumption by 2-6x and outage probability to 10^-6 in IIoT federated learning.
Non-identical data distributions degrade federated averaging accuracy on visual classification, but server momentum raises CIFAR-10 accuracy from 30.1% to 76.9% in the most skewed regimes.
FedBB addresses inter-case, inter-class, and inter-client imbalances in federated learning via Positive Negative Balanced loss and Client Balanced Reweighting, outperforming baselines on X-ray and natural image datasets while using limited statistics for privacy.
FedMTFI clusters heterogeneous clients, trains cluster prototypes, and applies multi-teacher distillation with SHAP to improve accuracy over standard FL in non-IID settings.
A closed-form FL convergence upper bound incorporating sensing SNR, dataset size, and transmission reliability enables joint optimization of sensing power, snapshots, and communication power in ISAC systems.
DP-FLogTinyLLM combines federated learning, differential privacy, and LoRA-tuned tiny LLMs to match centralized log anomaly detection performance on Thunderbird and BGL datasets while preserving privacy.
citing papers explorer
-
Sensing-Native Over-the-Air Federated Learning
Proposes a zero-overhead sensing-native over-the-air FL system that uses gradient autocorrelation for distance estimation and gradient statistics for cooperative localization, with a statistics-aware power and beamforming co-design.
-
Distributed Direct Preference Optimization
First convergence analysis of DPO under federated and decentralized training, characterizing rates via client drift, communication frequency, preference heterogeneity, and graph spectral connectivity.
-
Beyond Parameter Aggregation: Semantic Consensus for Federated Fine-Tuning of LLMs
Semantic consensus on model outputs for public prompts enables federated LLM fine-tuning that matches parameter-aggregation baselines with orders-of-magnitude lower communication.
-
Robust Federated Learning Under Real-World Client Churn
FeLiX reduces wall-clock time-to-target accuracy in federated learning by up to 2.37x using lightweight availability tiers, fresh-utility client selection, and informativeness-aware aggregation without requiring oracular knowledge of client availability.
-
Demystifying the Optimal Fair Classifier in Multi-Class Classification
Derives tractable optimal fair multi-class classifier and supplies in-processing and post-processing algorithms that converge to the accuracy-fairness Pareto frontier.
-
Bandwidth Allocation with Device Partitioning for Federated Learning over Industrial IoT networks
A device-partitioning bandwidth allocation policy for federated learning over IIoT networks that provably reduces total training time compared to any non-partitioning scheme.
-
Nonlinear Data Integration via Kernel Methods for Data Collaboration Analysis
Nonlinear kernel integration (NKI) with graph regularization enables accurate alignment of nonlinearly obfuscated decentralized data representations, outperforming linear methods on image classification.
-
Statistical Limits and Efficient Algorithms for Differentially Private Federated Learning
Introduces FedHybrid and FedNewton for DP federated M-estimation, with finite-sample MSE bounds, minimax lower bound, and evaluations on vision datasets.
-
Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity
Rescaled ASGD recovers convergence to the true global objective by rescaling worker stepsizes proportional to computation times, matching the known time lower bound in the leading term under non-convex smoothness and bounded heterogeneity.
-
FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learning
FedVSSAM mitigates flatness incompatibility in SAM-based federated learning by consistently using a variance-suppressed adjusted direction for local perturbation, descent, and global updates, with non-convex convergence guarantees.
-
Beyond End-to-End: Dynamic Chain Optimization for Private LLM Adaptation on the Edge
ChainFed achieves memory-efficient private LLM fine-tuning on edge devices through sequential layer-by-layer adapter training with dynamic co-tuning, perceptive optimization, and adaptive starting point selection, improving accuracy by up to 46.46%.
-
Adaptive Federated Optimization
Proposes federated adaptive optimizers (FedAdagrad, FedAdam, FedYogi) with convergence analysis for non-convex objectives under data heterogeneity and reports empirical gains over FedAvg.
-
QSplitFL: Capability Aware Deep Q-Learning for Optimal Split Point Selection in Split Federated Learning
QSplitFL is a DQN framework that selects split points in split federated learning from hardware metrics with a decayed loss-drop reward and committee voting, reporting faster convergence and higher accuracy than baselines on image classification tasks.
-
FIRMA: FIbonacci Ring Model Aggregation for Privacy-preserving Federated Learning
FIRMA introduces Fibonacci ring aggregation protocols for server-free federated learning that maintain private heads and achieve higher accuracy than FedAvg under label skew across multiple benchmarks and heterogeneity regimes.
-
FedGMI: Generative Model-Driven Federated Learning for Probabilistic Mixture Inference
FedGMI applies VAEs as density estimators in federated learning to infer mixture proportions of shared distributions for structured personalization under data heterogeneity.
-
FedPLT: Scalable, Resource-Efficient, and Heterogeneity-Aware Federated Learning via Partial Layer Training
FedPLT assigns client-specific model layers for training and matches or beats full-model federated learning accuracy with 71-82 percent fewer trainable parameters per client.
-
AdaBFL: Multi-Layer Defensive Adaptive Aggregation for Bzantine-Robust Federated Learning
AdaBFL introduces a novel three-layer adaptive aggregation mechanism for Byzantine-robust federated learning that counters complex attacks, provides non-convex non-iid convergence guarantees, and shows superior performance in experiments.
-
SplitFT: An Adaptive Federated Split Learning System For LLMs Fine-Tuning
SplitFT adapts cut-layer selection and reduces LoRA rank per client in federated split learning to improve efficiency and performance when fine-tuning LLMs on heterogeneous devices and data.
-
Energy-Efficient Federated Learning with Relay-Assisted Aggregation in IIoT Networks
Relay-assisted partial aggregation with device grouping, relay selection, and SPCA-based power optimization reduces energy consumption by 2-6x and outage probability to 10^-6 in IIoT federated learning.
-
Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
Non-identical data distributions degrade federated averaging accuracy on visual classification, but server momentum raises CIFAR-10 accuracy from 30.1% to 76.9% in the most skewed regimes.
-
Multi-Level Analyzation of Imbalance to Resolve Non-IID-Ness in Federated Learning
FedBB addresses inter-case, inter-class, and inter-client imbalances in federated learning via Positive Negative Balanced loss and Client Balanced Reweighting, outperforming baselines on X-ray and natural image datasets while using limited statistics for privacy.
-
FedMTFI: Feature Importance Based Optimized Multi Teacher Knowledge Distillation in Heterogeneous Federated Learning Environment
FedMTFI clusters heterogeneous clients, trains cluster prototypes, and applies multi-teacher distillation with SHAP to improve accuracy over standard FL in non-IID settings.
-
ISAC for AI: A Trade-off Framework Across Data Acquisition and Transfer in Federated Learning
A closed-form FL convergence upper bound incorporating sensing SNR, dataset size, and transmission reliability enables joint optimization of sensing power, snapshots, and communication power in ISAC systems.
-
DP-FlogTinyLLM: Differentially private federated log anomaly detection using Tiny LLMs
DP-FLogTinyLLM combines federated learning, differential privacy, and LoRA-tuned tiny LLMs to match centralized log anomaly detection performance on Thunderbird and BGL datasets while preserving privacy.
-
From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learning
A data-centric survey of federated learning that ranks non-IID data traits by influence on convergence, links splitting protocols to real phenomena, and examines data-related defenses under clean and adversarial conditions.