Decentralized SGD achieves high-probability convergence with order-optimal rates and linear speedup under standard cost assumptions matching those for MSE convergence.
A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Norm
7 Pith papers cite this work, alongside 75 external citations. Polarity classification is still indexing.
7
Pith papers citing it
75
external citations · Pith
abstract
In this note, we derive concentration inequalities for random vectors with subGaussian norm (a generalization of both subGaussian random vectors and norm bounded random vectors), which are tight up to logarithmic factors.
verdicts
UNVERDICTED 7representative citing papers
Introduces hybrid noise and novel coupling analysis to achieve the first convergent hidden-state DP bound for zeroth-order optimization.
The paper establishes that the optimal excess risk for ε-unlearning is the usual statistical error plus an unlearning penalty that interpolates between retraining-from-scratch and an exponentially smaller term as ε/d grows, with matching bounds for mean estimation.
Establishes maximal concentration bounds for stochastic approximation under heavy-tailed Markovian noise, with tails ranging from sub-Gaussian to heavier than Weibull depending on step sizes and contractivity properties, plus a truncation argument for unbounded noise.
New analysis without global strong convexity yields tight scaling laws: NS error ~Θ(kd/n²) and NS-IF difference ~Θ((k+d)√(kd)/n²) for well-behaved logistic regressions.
GL-LowPopArt is a Catoni-style two-stage estimator for generalized low-rank trace regression that attains state-of-the-art bounds and nearly instance-wise minimax optimality up to the Hessian condition number.
A survey of Lyapunov techniques using generalized Moreau envelopes as universal functions for non-asymptotic mean-square convergence analysis of stochastic iterative algorithms under contractive operators.
citing papers explorer
-
High-Probability Convergence Guarantees of Decentralized SGD
Decentralized SGD achieves high-probability convergence with order-optimal rates and linear speedup under standard cost assumptions matching those for MSE convergence.
-
Privacy Amplification in Differentially Private Zeroth-Order Optimization with Hidden States
Introduces hybrid noise and novel coupling analysis to achieve the first convergent hidden-state DP bound for zeroth-order optimization.
-
Near-Optimal Pure Machine Unlearning for Smooth Strongly Convex Losses
The paper establishes that the optimal excess risk for ε-unlearning is the usual statistical error plus an unlearning penalty that interpolates between retraining-from-scratch and an exponentially smaller term as ε/d grows, with matching bounds for mean estimation.
-
Concentration of General Stochastic Approximation Under Heavy-Tailed Markovian Noise
Establishes maximal concentration bounds for stochastic approximation under heavy-tailed Markovian noise, with tails ranging from sub-Gaussian to heavier than Weibull depending on step sizes and contractivity properties, plus a truncation argument for unbounded noise.
-
On the Accuracy of Newton Step and Influence Function Data Attributions
New analysis without global strong convexity yields tight scaling laws: NS error ~Θ(kd/n²) and NS-IF difference ~Θ((k+d)√(kd)/n²) for well-behaved logistic regressions.
-
GL-LowPopArt: A Nearly Instance-Wise Minimax-Optimal Estimator for Generalized Low-Rank Trace Regression
GL-LowPopArt is a Catoni-style two-stage estimator for generalized low-rank trace regression that attains state-of-the-art bounds and nearly instance-wise minimax optimality up to the Hessian condition number.
-
Non-Asymptotic Convergence of Stochastic Iterative Algorithms: A Lyapunov Framework
A survey of Lyapunov techniques using generalized Moreau envelopes as universal functions for non-asymptotic mean-square convergence analysis of stochastic iterative algorithms under contractive operators.