DMBA maintains attack success rates above 80% for all backdoors in a distributed multi-target FL setting where baselines drop below 50%.
hub
Learning Differentially Private Recurrent Language Models
13 Pith papers cite this work. Polarity classification is still indexing.
13
Pith papers citing it
abstract
We demonstrate that it is possible to train large recurrent language models with user-level differential privacy guarantees with only a negligible cost in predictive accuracy. Our work builds on recent advances in the training of deep networks on user-partitioned data and privacy accounting for stochastic gradient descent. In particular, we add user-level privacy protection to the federated averaging algorithm, which makes "large step" updates from user-level data. Our work demonstrates that given a dataset with a sufficiently large number of users (a requirement easily met by even small internet-scale datasets), achieving differential privacy comes at the cost of increased computation, rather than in decreased utility as in most prior work. We find that our private LSTM language models are quantitatively and qualitatively similar to un-noised models when trained on a large dataset.
hub tools
citation-role summary
background 3
citation-polarity summary
roles
background 3representative citing papers
Introduces FedHybrid and FedNewton for DP federated M-estimation, with finite-sample MSE bounds, minimax lower bound, and evaluations on vision datasets.
Nexa learns a response-conditioned policy that starts with parallel agent execution and adds at most one round of sequential message passing via a predicted sparse DAG, strictly subsuming pure parallel mode.
Hybrid QFL cuts quantum transmissions from 3TNMP to {3t + 2(T-t)}NMP over T rounds while preserving near-centralized convergence and improving depolarizing-noise resilience via decentralized aggregation and Steane-code QEC.
Proposes federated adaptive optimizers (FedAdagrad, FedAdam, FedYogi) with convergence analysis for non-convex objectives under data heterogeneity and reports empirical gains over FedAvg.
FedSteer constructs a gradient subspace from cached client updates, projects active gradients to obtain coordinates, and reuses those coordinates on the drifted subspace to correct extreme staleness in federated learning.
Replaces determinant growth with generalized Rayleigh quotient for rare switching in private linear bandits to control worst-direction volume despite non-monotonic design matrices from noise.
CPPDD is a new consensus-based protocol for privacy-preserving multi-client data sharing that achieves unanimous-release confidentiality, linear scalability, and high-probability malicious deviation detection.
DP-LAC provides a new adaptive clipping technique for DP-SGD in federated LLM fine-tuning that improves accuracy by 6.6% on average without consuming additional privacy budget or requiring new hyperparameters.
Adaptive bit-length schedulers plus Laplacian DP in non-IID FL reduce communicated data by up to 52.64% on MNIST and 45% on CIFAR-10 while keeping competitive accuracy and privacy.
Abstract-only report: theoretical comparison finds MIM more robust than CL to non-IID data in D-SSL and robustness scales with connectivity; MAR loss proposed as practical application.
The fitness difference between DP and non-private convex ML models is inversely proportional to training dataset size squared and privacy budget squared.
A survey organizing knowledge distillation techniques for addressing privacy, heterogeneity, communication, and personalization challenges in federated learning.
citing papers explorer
-
Adaptive Federated Optimization
Proposes federated adaptive optimizers (FedAdagrad, FedAdam, FedYogi) with convergence analysis for non-convex objectives under data heterogeneity and reports empirical gains over FedAvg.
-
FedSteer: Taming Extreme Gradient Staleness in Federated Learning with Corrective Projections and Caching
FedSteer constructs a gradient subspace from cached client updates, projects active gradients to obtain coordinates, and reuses those coordinates on the drifted subspace to correct extreme staleness in federated learning.
-
When Determinants Are Not Enough: Private Rare Switching
Replaces determinant growth with generalized Rayleigh quotient for rare switching in private linear bandits to control worst-direction volume despite non-monotonic design matrices from noise.
-
DP-LAC: Lightweight Adaptive Clipping for Differentially Private Federated Fine-tuning of Language Models
DP-LAC provides a new adaptive clipping technique for DP-SGD in federated LLM fine-tuning that improves accuracy by 6.6% on average without consuming additional privacy budget or requiring new hyperparameters.
-
Understanding the Robustness of Distributed Self-Supervised Learning Frameworks Against Non-IID Data
Abstract-only report: theoretical comparison finds MIM more robust than CL to non-IID data in D-SSL and robustness scales with connectivity; MAR loss proposed as practical application.
-
Knowledge Distillation in Federated Learning: a Survey on Long Lasting Challenges and New Solutions
A survey organizing knowledge distillation techniques for addressing privacy, heterogeneity, communication, and personalization challenges in federated learning.