Using the previous posterior as the next prior in federated SGLD training cut iterations to 85% accuracy by about 50% over three days of radar data, with improved calibration.
Decentralized Stochastic Gradient Langevin Dynamics and Hamiltonian Monte Carlo
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Stochastic gradient Langevin dynamics (SGLD) and stochastic gradient Hamiltonian Monte Carlo (SGHMC) are two popular Markov Chain Monte Carlo (MCMC) algorithms for Bayesian inference that can scale to large datasets, allowing to sample from the posterior distribution of the parameters of a statistical model given the input data and the prior distribution over the model parameters. However, these algorithms do not apply to the decentralized learning setting, when a network of agents are working collaboratively to learn the parameters of a statistical model without sharing their individual data due to privacy reasons or communication constraints. We study two algorithms: Decentralized SGLD (DE-SGLD) and Decentralized SGHMC (DE-SGHMC) which are adaptations of SGLD and SGHMC methods that allow scaleable Bayesian inference in the decentralized setting for large datasets. We show that when the posterior distribution is strongly log-concave and smooth, the iterates of these algorithms converge linearly to a neighborhood of the target distribution in the 2-Wasserstein distance if their parameters are selected appropriately. We illustrate the efficiency of our algorithms on decentralized Bayesian linear regression and Bayesian logistic regression problems.
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Bayesian Federated Learning for Continual Training
Using the previous posterior as the next prior in federated SGLD training cut iterations to 85% accuracy by about 50% over three days of radar data, with improved calibration.