REVIEW 4 cited by
Analysis of nonsmooth stochastic approximation: the differential inclusion approach
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
In this paper we address the convergence of stochastic approximation when the functions to be minimized are not convex and nonsmooth. We show that the "mean-limit" approach to the convergence which leads, for smooth problems, to the ODE approach can be adapted to the non-smooth case. The limiting dynamical system may be shown to be, under appropriate assumption, a differential inclusion. Our results expand earlier works in this direction by Benaim et al. (2005) and provide a general framework for proving convergence for unconstrained and constrained stochastic approximation problems, with either explicit or implicit updates. In particular, our results allow us to establish the convergence of stochastic subgradient and proximal stochastic gradient descent algorithms arising in a large class of deep learning and high-dimensional statistical inference with sparsity inducing penalties.
Forward citations
Cited by 4 Pith papers
-
Testing Approximate Stationarity Concepts for Piecewise Affine Functions
Testing approximate stationarity of piecewise affine functions is NP-hard; an exact Clarke subdifferential sum rule holds under a new polytope compatibility condition; near-approximate stationarity can be certified in...
-
The late-stage training dynamics of (stochastic) subgradient descent on homogeneous neural networks
Normalized stochastic subgradient descent iterates converge, after perfect classification, to critical points of the normalized margin for homogeneous neural networks.
-
Stationary Robust Mean-Field Games under Model Mismatches
Develops infinite-horizon stationary robust mean-field games incorporating distributional uncertainty, proves equilibrium existence via fixed-point on contractive Bellman operator, gives convergent algorithm, and deri...
-
Convergence of projected stochastic approximation algorithm
A rigorous proof shows that Robbins-Monro stochastic approximation with projections onto a hyperrectangle converges to stationary points of the associated projected ODE, filling a gap in Kushner and Yin's classic text.
Discussion (0). Continue with ORCID to comment.