Pith. sign in

REVIEW 4 minor 47 references

This paper proves that constant-stepsize SGD at a flat minimum has stationary error of order alpha^(1/m) and a generally non-Gaussian diffusion limit, with Markovian noise entering only through the asymptotic covariance.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 21:18 UTC pith:G4V5IPKC

load-bearing objection This paper proves the α^(1/m) non-Gaussian scaling limit for flat-minima SGD rigorously, and the proof is solid enough to deserve a serious referee.

arxiv 2607.16384 v1 pith:G4V5IPKC submitted 2026-07-17 cs.LG math.OCmath.PRstat.ML

Scaling Limits of Constant-Stepsize SGD at Flat Minima

Jingyi Zhang , Cheng Mao , Debankur Mukherjee This is my paper
classification cs.LG math.OCmath.PRstat.ML MSC 62L2060H1060J05
keywords stochastic gradient descentconstant stepsizeflat minimainvariant lawscaling limitMarkovian noisenon-Gaussian stationary distributiongeometric ergodicity
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

For a constant step size alpha, SGD does not converge to the minimizer; it settles into a stationary distribution whose width describes the algorithm's long-run error. The paper shows that when the objective has an m-flat minimum and subquadratic tails, that width is of order alpha^(1/m), not the familiar sqrt(alpha) of strongly convex problems. It proves geometric convergence to the stationary law with contraction factor 1 - c alpha^(m-1), and identifies the small-step limit: the rescaled iterate alpha^{-1/m} X converges to the stationary law of dY = -h_0(Y)dt + Sigma^{1/2}dB, with h_0 the limiting drift at the minimizer and Sigma the long-run covariance of the Markovian noise. For m=2 this recovers the Gaussian Ornstein-Uhlenbeck limit; for m>2 the limit is non-Gaussian. The result matters because flat minima arise naturally in quantile, CVaR, and robust objectives, where it gives a concrete prediction for how SGD noise scales and what shape it takes.

Core claim

For every sufficiently small step size alpha, the augmented SGD chain has a unique invariant law, and the paper proves geometric convergence to it in an alpha-dependent Wasserstein metric, with contraction rate 1 - c alpha^(m-1). The main discovery is the small-step scaling limit: after centering at the minimizer and rescaling by alpha^{-1/m}, the invariant law converges weakly to the unique stationary distribution of the stochastic differential equation dY_t = -h_0(Y_t)dt + Sigma^{1/2} dB_t, where h_0 is the limiting homogeneous drift near the flat minimizer and Sigma is the Green-Kubo asymptotic covariance of the centered noise at the minimizer. For m=2 this is the classical Gaussian Ornst

What carries the argument

The central object is the augmented Markov chain (X_n, xi_n) and its invariant law pi_alpha. The argument is carried by an alpha-dependent Lyapunov-weight metric: a weight V_alpha = T_beta + delta_alpha omega combines an exponential tail weight T_beta for subquadratic tails with a compactly supported near-minimizer weight omega whose amplitude delta_alpha is tuned so that the directional contraction kernel gives a factor 1 - c alpha^{m-1}. A Poisson-equation decomposition of the Markovian noise turns dependent increments into a martingale difference plus an asymptotically negligible telescoping term, which identifies the diffusion covariance Sigma as the Green-Kubo sum of lagged autocovarian

Load-bearing premise

The proof requires the mean gradient noise at each point to oppose the deterministic drift by less than a 100% share: a single constant theta<1 bounds how much the conditional noise mean can cancel the restoring force, and if that fails the averaged Jacobian can lose its positive curvature and the whole contraction argument collapses.

What would settle it

Run constant-stepsize SGD on H(x)=x^4/4 (m=4) with symmetric noise of bounded variance at the minimizer, and measure the stationary standard deviation of X for several alpha near 0. If it does not scale like alpha^{1/4} — equivalently, if alpha^{-1/4}X_infty does not converge to the stationary law of dY = -Y^3 dt + sqrt(Sigma) dB, whose density is proportional to exp(-|z|^4/(4 Sigma)) — then the scaling-limit theorem is false.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Stationary SGD error at an m-flat minimum scales as alpha^(1/m), so for m>2 the error decays more slowly than the strongly convex sqrt(alpha) rate.
  • The limiting stationary law is the invariant law of a nonlinear diffusion; for m>2 it is not Gaussian, with shape set by the homogeneous drift h_0 and the covariance Sigma.
  • Markov dependence in the gradient noise does not change the scaling exponent; it enters only through the long-run covariance Sigma computed from lagged correlations.
  • Geometric ergodicity with rate 1 - c alpha^(m-1) means the mixing time grows like alpha^{-(m-1)} as the step shrinks near a flat minimum.
  • For coordinate-separable objectives with unequal flatness exponents, under a common alpha^{-1/m_max} scaling only the flattest coordinates (m_i = m_max) have nondegenerate limits.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If this extends to deep-learning loss landscapes with locally flat minima, a practitioner could estimate m from the log-log slope of stationary SGD error versus step size and then predict the noise distribution's non-Gaussian shape.
  • The scale alpha^(1/m) implies that to halve the stationary error near a flat minimum, the step size must be divided by 2^m; for m=4 this is a factor of 16, which may explain why very small steps are needed in flat regions.
  • The generator argument suggests a multi-time-scale averaging theory for genuinely anisotropic objectives, where different coordinate groups evolve on different effective time scales alpha^{-1/m_i}; this is the natural next step beyond the separable case.
  • Because the contraction rate is 1 - c alpha^(m-1), methods that rely on fast mixing, such as coupling-based bias corrections, will degrade polynomially in alpha near flat minima.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 4 minor

Summary. The paper analyzes constant-stepsize SGD with Markovian noise for convex objectives whose minimizer may be flatter than quadratic. Under explicit assumptions on the objective (local flatness exponent m, tail exponent β) and on the update (contractive driving chain, co-coercive stochastic gradient, nondegenerate noise), it proves existence, uniqueness, and geometric ergodicity of the augmented invariant law with contraction factor 1−cα^{m−1} (Theorem 2.7). It then derives the small-stepsize scaling limit: after the α^{−1/m} rescaling, the invariant law converges weakly to the stationary law of the nonlinear SDE dY_t=−h0(Y_t)dt+Σ^{1/2}dB_t, where h0 is the limiting drift and Σ is a Green–Kubo covariance (Theorem 2.13). For m=2 this recovers the classical Gaussian/Ornstein–Uhlenbeck limit; for m>2 the scale is α^{1/m} and the limit is generally non-Gaussian. Coordinate-separable extensions with unequal flatness exponents are also given (Corollaries 2.15 and 2.17).

Significance. If the results hold, they provide the first rigorous invariant-law scaling limit for flat minima, confirming and extending prior numerical and conjectural work. The proof chain is detailed and internally consistent: geometric ergodicity is built from a directional contraction lemma (Lemma 4.3/4.4), and the scaling limit is identified through tightness, a Poisson-equation decomposition of Markovian noise, and a stationary generator equation (Lemmas 4.7, 4.9, 4.10). Important strengths are that the covariance Σ is derived, not fitted; that the assumptions are stated explicitly; and that the numerical experiments directly check the predicted α^{1/m} scaling and the Green–Kubo constant. The main caveat is scope: Assumption 2.5(N7), combined with Lemma 4.2, forces the map h to be globally Lipschitz, so canonical polynomial flat objectives such as H(x)=||x||^4 do not satisfy the full hypotheses. This narrows the advertised class of flat minima but does not invalidate the conditional theorems.

minor comments (4)
  1. [Assumption 2.5(N7) and Lemma 4.2] The abstract and introduction state the results for 'convex objectives H with flat minima', but Assumption 2.5(N7)(2.6), through Lemma 4.2, implies that h=∇H is globally Lipschitz. Consequently H(x)=||x||^m/m for m>2, which satisfies the local-flatness condition (H2), cannot satisfy the full assumptions. Please add an explicit remark stating this consequence of N7 and, if possible, give a concrete smooth objective with m>2, globally Lipschitz h, and subquadratic/quadratic tails that does satisfy all assumptions. The conditional theorems appear correct; this is a scope-clarity issue.
  2. [Section 3.1] The finite-state median example is explicitly described as illustrative because the pinball loss is nonsmooth, and the robust/logistic examples in Section 3.2 have m=2. Thus the paper does not provide a direct statistical application of the m>2 theorem. The authors should either add a smooth m>2 example satisfying the hypotheses or state more prominently in the conclusion that the m>2 regime is covered only under the additional global smoothness/co-coercivity conditions.
  3. [Lemma B.9] In the covariance calculation, the terms containing h(X_α) are dropped after the sentence 'The terms in the quadratic expansion containing h_α are negligible'. A one-sentence justification using (B.9) (E||h(X)||^2 ≤ Cα and sup E||g||^2 < ∞) would make the step easier to follow for readers.
  4. [Remark 2.4] The examples listed in Remark 2.4 illustrate condition (H2) only. Since the full theorems require much more (especially N7 and N8), I suggest explicitly stating that those examples are not claimed to satisfy the whole set of assumptions, to avoid misleading readers about the applicability of Theorem 2.13.

Circularity Check

0 steps flagged

No significant circularity: the scaling limit, covariance, and ergodic contraction are derived from explicit hypotheses rather than fitted to the target result.

full rationale

The central claims are self-contained derivations from stated assumptions. Theorem 2.13's limiting drift h0 is defined by Assumption 2.10(H4) as the local homogeneous limit of ∇H, and the covariance Σ is obtained through the Poisson-equation/Green–Kubo computation in Lemma 4.6 and equation (2.11), not calibrated to a target. Theorem 2.7's geometric ergodicity is proved under the explicit Assumptions 2.3 and 2.5, with the delicate co-coercivity/mean-perturbation condition (N7) stated as a hypothesis and verified for the logistic example; several authors also note that the nonsmooth quantile example is illustrative rather than a direct application. The self-citations to prior work by the authors appear only in the related-literature discussion and in the recognition of the subquadratic m=2 case, not as load-bearing uniqueness theorems or ansatz justifications. Numerical experiments either compute exact finite-state invariant laws or display predicted scalings; they do not feed fitted values back into the proofs. No equation in the proof chain reduces to a previously fitted quantity or to a self-citation, so the appropriate finding is no significant circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 8 axioms · 0 invented entities

Central claims rest entirely on the stated structural assumptions and standard tools. No free parameters are fitted, no new entities are postulated.

axioms (8)
  • domain assumption Assumptions 2.3(H1)-(H3): H is C^2 convex with minimizer 0, local flatness ∇²H(x) ⪰ c_in ||x||^(m−2) I_d and ||h(x)|| ≤ C_in||x||^(m−1) near 0, and tail coercivity with exponent β∈[1,2].
    Defines the objective class; used throughout for the near-minimizer bounds (Lemma A.1, Lemma B.1) and tail return estimates (Lemma A.5).
  • domain assumption Assumptions 2.5(N1)-(N7): contractive driving chain, Lipschitz noise, reference-point integrability, tail dissipativity, exponential integrability, noise nondegeneracy at minimizer, and co-coercivity with mean-perturbation bound.
    Defines the noise and update class. (N7) is used in Lemma 4.2 to obtain samplewise nonexpansiveness and averaged contraction of the SGD Jacobian.
  • domain assumption Assumption 2.10(H4): local gradient expansion ∇H(x)=∇H0(x)+o(||x||^(m−1)) with H0 convex, C^2, homogeneous of degree m.
    Guarantees the limiting drift h0 exists and is locally uniformly approximated by h_alpha.
  • domain assumption Assumption 2.11(N8): stationary centering ∫g(x,ξ)πΞ(dξ)=0 for all x and finite second moments at the minimizer.
    Needed so the Green-Kubo series (2.11) converges and the diffusion covariance is well-defined.
  • standard math Banach fixed-point theorem and completeness of (P1,d) (Proposition 4.1).
    Converts the one-step Wasserstein contraction into existence, uniqueness and geometric convergence of the invariant law.
  • standard math Krylov-Bogoliubov existence, Echeverria criterion, and Itô formula for the limiting diffusion.
    Used in Lemma 4.10 to prove existence and uniqueness of the invariant law of the SDE.
  • standard math Baillon-Haddad theorem for gradients of smooth convex functions.
    Used in Lemma 4.2 to derive ∥h(x)∥² ≤ L/(1−θ)⟨x,h(x)⟩ and uniform Lipschitz bounds.
  • standard math Poisson equation theory for Markov chains (Lemma B.3, Lemma 4.6).
    Used to decompose Markovian noise into a martingale-difference plus telescoping term, defining Σ.

pith-pipeline@v1.3.0-alltime-deepseek · 10 in / 21908 out tokens · 170925 ms · 2026-08-01T21:18:16.443522+00:00 · methodology

0 comments
read the original abstract

For stochastic gradient descent (SGD) with a constant stepsize $\alpha$, the invariant law of the iterates, centered at a minimizer, describes the behavior of the algorithm over long time horizons. In the strongly convex case, this invariant law has the familiar $\sqrt{\alpha}$ scaling and a Gaussian limit as $\alpha\downarrow 0$. We show that this behavior changes fundamentally for convex objectives $H$ with flat minima and (sub)quadratic tails. More specifically, we study SGD with Markovian noise generated by a contractive driving chain. For every sufficiently small constant stepsize $\alpha$, we prove existence, uniqueness, and geometric convergence to an augmented invariant law in a Wasserstein distance induced by an $\alpha$-dependent metric. When the minimizer $x_\star$ has local flatness exponent $m\ge2$, meaning that $\nabla^2 H(x)\asymp \lVert x-x_\star\rVert^{m-2} I_d$ as $x\to x_\star$, we obtain a contraction bound with factor $1-c\alpha^{m-1}$, where $c>0$ is a constant. This recovers the factor $1-c\alpha$ in the quadratic case $m=2$. We then analyze the small-stepsize scaling limit. We show that the invariant law concentrates on the scale $\alpha^{1/m}$ and that the rescaled iterates converge weakly to the stationary distribution of the stochastic differential equation $$ dY_t=-h_0(Y_t)\,dt+\Sigma^{1/2}\,dB_t , $$ where $h_0$ is the limiting drift at the minimizer and $\Sigma$ denotes the asymptotic covariance. This recovers the Gaussian limit when $m=2$ and gives generally non-Gaussian stationary limits in the flat case $m>2$. Finally, we give corresponding results for coordinate-separable objectives with unequal flatness exponents.

Figures

Figures reproduced from arXiv: 2607.16384 by Cheng Mao, Debankur Mukherjee, Jingyi Zhang.

Figure 1
Figure 1. Figure 1: SGD for median estimation. Panel (a) shows the L2 error for several crossing orders m; the fitted slopes match the theoretical exponent 1/m. Panel (b) checks the predicted local moment scale: in this model α −1E|X (α) ∞ | m → Σ = 1/8. Panel (c) shows the convergence of α −1/4X (α) ∞ for m = 4 to the quartic invariant density of the limiting diffusion, together with a Gaussian of the same variance for compa… view at source ↗
Figure 2
Figure 2. Figure 2: Two consequences of the scaling limit of invariant laws. Panel (a) uses a Markovian sign stream and verifies that the fourth moment constant in the m = 4 quantile example is governed by the asymptotic covariance rather than by the marginal variance. Panel (b) considers a separable two-coordinate model with exponents m1 = 2 and m2 = 4; under the common α −1/4 scale, the m1 = 2 coordinate converges to zero w… view at source ↗
Figure 3
Figure 3. Figure 3: Robust location estimation with generalized Charbonnier losses. Panel (a) verifies the population tail Hβ(x) ≍ |x| β . Panel (b) verifies the Gaussian scaling limit of X (α) ∞ / √ α for the pseudo-Huber loss with β = 1 because it remains locally quadratic. Panel (c) shows that the decay of the normalized state EXt/x0, starting from a large x0 > 0, depends on the tail exponent. Panel (d) normalizes the same… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references · 8 canonical work pages

  1. [1]

    Allmeier and N

    S. Allmeier and N. Gast. Computing the bias of constant-step stochastic approximation with Markovian noise. InAdvances in Neural Information Processing Systems, volume 37, pages 137873–137902, 2024. doi: 10.52202/079017-4379

    work page doi:10.52202/079017-4379 2024

  2. [2]

    Bach and ´E

    F. Bach and ´E. Moulines. Non-strongly-convex smooth stochastic approximation with conver- gence rate O(1/n). InAdvances in Neural Information Processing Systems, volume 26, pages 773–781, 2013

    2013

  3. [3]

    A. D. Barbour. Stein’s method for diffusion approximations.Probab. Theory Related Fields, 84 (3):297–322, 1990. doi: 10.1007/BF01197887

    doi:10.1007/bf01197887 1990

  4. [4]

    J. T. Barron. A general and adaptive robust loss function. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4331–4339, 2019. doi: 10.1109/ CVPR.2019.00446

    arXiv 2019

  5. [5]

    P. L. Bartlett, M. I. Jordan, and J. D. McAuliffe. Convexity, classification, and risk bounds.J. Amer. Statist. Assoc., 101(473):138–156, 2006. doi: 10.1198/016214505000000907

    doi:10.1198/016214505000000907 2006

  6. [6]

    H. H. Bauschke and P. L. Combettes. The Baillon–Haddad theorem revisited.J. Convex Anal., 17(3–4):781–787, 2010

    2010

  7. [7]

    Benveniste, M

    A. Benveniste, M. M´ etivier, and P. Priouret.Adaptive Algorithms and Stochastic Approx- imations, volume 22 ofApplications of Mathematics. Springer-Verlag, Berlin, 1990. doi: 10.1007/978-3-642-75894-2

    doi:10.1007/978-3-642-75894-2 1990

  8. [8]

    V. S. Borkar. Stochastic approximation with two time scales.Systems Control Lett., 29(5): 291–294, 1997. doi: 10.1016/S0167-6911(97)90015-3. 49

    doi:10.1016/s0167-6911(97)90015-3 1997

  9. [9]

    V. S. Borkar.Stochastic Approximation: A Dynamical Systems Viewpoint, volume 48 of Texts and Readings in Mathematics. Hindustan Book Agency, Gurgaon, 2008. doi: 10.1007/ 978-93-86279-38-5

    2008

  10. [10]

    Braverman, J

    A. Braverman, J. G. Dai, and J. Feng. Stein’s method for steady-state diffusion approximations: An introduction through the Erlang-A and Erlang-C models.Stoch. Syst., 6(2):301–366, 2016. doi: 10.1214/15-SSY212

    work page doi:10.1214/15-ssy212 2016

  11. [11]

    Charbonnier, L

    P. Charbonnier, L. Blanc-F´ eraud, G. Aubert, and M. Barlaud. Deterministic edge-preserving regularization in computed imaging.IEEE Trans. Image Process., 6(2):298–311, 1997. doi: 10.1109/83.551699

    doi:10.1109/83.551699 1997

  12. [12]

    Z. Chen, S. Mou, and S. T. Maguluri. Stationary behavior of constant stepsize SGD type algorithms: An asymptotic characterization.Proc. ACM Meas. Anal. Comput. Syst., 6(1): 19:1–19:24, 2022. doi: 10.1145/3508039

    work page doi:10.1145/3508039 2022

  13. [13]

    Dieuleveut, A

    A. Dieuleveut, A. Durmus, and F. Bach. Bridging the gap between constant step size stochastic gradient descent and Markov chains.Ann. Statist., 48(3):1348–1382, 2020. doi: 10.1214/ 19-AOS1850

    2020

  14. [14]

    Eberle, A

    A. Eberle, A. Guillin, and R. Zimmer. Quantitative Harris-type theorems for diffusions and McKean–Vlasov processes.Trans. Amer. Math. Soc., 371(10):7135–7173, 2019. doi: 10.1090/tran/7576

    doi:10.1090/tran/7576 2019

  15. [15]

    S. N. Ethier and T. G. Kurtz.Markov Processes: Characterization and Convergence. Wiley Series in Probability and Mathematical Statistics. John Wiley & Sons, New York, 1986

    1986

  16. [16]

    V. Fabian. On asymptotic normality in stochastic approximation.Ann. Math. Statist., 39(4): 1327–1332, 1968. doi: 10.1214/aoms/1177698258

    arXiv 1968

  17. [17]

    N. Gast. Expected values estimated via mean-field approximation are 1 /N-accurate.Proc. ACM Meas. Anal. Comput. Syst., 1(1):17:1–17:26, 2017. doi: 10.1145/3084454

    work page doi:10.1145/3084454 2017

  18. [18]

    Hadavi, W

    H. Hadavi, W. Mou, S. Samsonov, and H.-T. Wai. Revisiting the constant stepsize stochastic approximation with decision-dependent Markovian noise, 2026. arXiv:2604.13378

    Pith/arXiv arXiv 2026

  19. [19]

    P. J. Huber. Robust estimation of a location parameter.Ann. Math. Statist., 35(1):73–101,

  20. [20]

    D. Huo, Y. Chen, and Q. Xie. Bias and extrapolation in Markovian linear stochastic approxi- mation with constant stepsizes.Math. Oper. Res., 2026. doi: 10.1287/moor.2024.0471. Articles in Advance

    arXiv 2026

  21. [21]

    D. L. Huo, Y. Zhang, Y. Chen, and Q. Xie. The collusion of memory and nonlinearity in stochastic approximation with constant stepsize. InAdvances in Neural Information Processing Systems, volume 37, pages 21699–21762, 2024. doi: 10.52202/079017-0684

    work page doi:10.52202/079017-0684 2024

  22. [22]

    Kang and T

    H.-W. Kang and T. G. Kurtz. Separation of time-scales and model reduction for stochastic reaction networks.Ann. Appl. Probab., 23(2):529–583, 2013. doi: 10.1214/12-AAP841

    work page doi:10.1214/12-aap841 2013

  23. [23]

    R. Z. Khasminskii. On the principle of averaging the Itˆ o’s stochastic differential equations. Kybernetika, 4(3):260–279, 1968. 50

    1968

  24. [25]

    K. Knight. Limiting distributions for L1 regression estimators under general conditions.Ann. Statist., 26(2):755–770, 1998. doi: 10.1214/aos/1028144858

    arXiv 1998

  25. [26]

    Koenker and G

    R. Koenker and G. Bassett, Jr. Regression quantiles.Econometrica, 46(1):33–50, 1978. doi: 10.2307/1913643

    doi:10.2307/1913643 1978

  26. [27]

    H. J. Kushner and G. G. Yin.Stochastic Approximation and Recursive Algorithms and Applications, volume 35 ofApplications of Mathematics. Springer, New York, second edition,

  27. [28]

    L. Ljung. Analysis of recursive stochastic algorithms.IEEE Trans. Automat. Control, 22(4): 551–575, 1977. doi: 10.1109/TAC.1977.1101561

    arXiv 1977

  28. [29]

    Mandt, M

    S. Mandt, M. D. Hoffman, and D. M. Blei. Stochastic gradient descent as approximate Bayesian inference.J. Mach. Learn. Res., 18(134):1–35, 2017

    2017

  29. [30]

    Merad and S

    I. Merad and S. Ga ¨ ıffas. Convergence and concentration properties of constant step-size SGD through Markov chains.Electron. J. Stat., 19(2):5843–5894, 2025. doi: 10.1214/25-EJS2471

    doi:10.1214/25-ejs2471 2025

  30. [31]

    S. P. Meyn and R. L. Tweedie.Markov Chains and Stochastic Stability. Cambridge Mathematical Library. Cambridge University Press, Cambridge, second edition, 2009. doi: 10.1017/CBO9780511626630

    doi:10.1017/cbo9780511626630 2009

  31. [32]

    Moulines and F

    ´E. Moulines and F. R. Bach. Non-asymptotic analysis of stochastic approximation algorithms for machine learning. InAdvances in Neural Information Processing Systems, volume 24, pages 451–459, 2011

    2011

  32. [33]

    Nemirovski, A

    A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro. Robust stochastic approximation approach to stochastic programming.SIAM J. Optim., 19(4):1574–1609, 2009. doi: 10.1137/070704277

    doi:10.1137/070704277 2009

  33. [34]

    Pardoux and A

    ´E. Pardoux and A. Y. Veretennikov. On the Poisson equation and diffusion approximation. I. Ann. Probab., 29(3):1061–1085, 2001. doi: 10.1214/aop/1015345596

    arXiv 2001

  34. [35]

    Pardoux and A

    ´E. Pardoux and A. Y. Veretennikov. On Poisson equation and diffusion approximation. II. Ann. Probab., 31(3):1166–1192, 2003. doi: 10.1214/aop/1055425774

    arXiv 2003

  35. [36]

    Pardoux and A

    ´E. Pardoux and A. Y. Veretennikov. On the Poisson equation and diffusion approximation. III. Ann. Probab., 33(3):1111–1133, 2005. doi: 10.1214/009117905000000062

    work page doi:10.1214/009117905000000062 2005

  36. [37]

    G. C. Pflug. Stochastic minimization with constant step-size: Asymptotic laws.SIAM J. Control Optim., 24(4):655–666, 1986. doi: 10.1137/0324039

    doi:10.1137/0324039 1986

  37. [38]

    B. T. Polyak and A. B. Juditsky. Acceleration of stochastic approximation by averaging.SIAM J. Control Optim., 30(4):838–855, 1992. doi: 10.1137/0330046

    doi:10.1137/0330046 1992

  38. [39]

    Y. Qu, J. Blanchet, and P. W. Glynn. Computable bounds on convergence of Markov chains in Wasserstein distance via contractive drift.Ann. Appl. Probab., 35(4):2678–2715, 2025. doi: 10.1214/25-AAP2184

    work page doi:10.1214/25-aap2184 2025

  39. [40]

    Robbins and S

    H. Robbins and S. Monro. A stochastic approximation method.Ann. Math. Statist., 22(3): 400–407, 1951. doi: 10.1214/aoms/1177729586. 51

    arXiv 1951

  40. [41]

    R. T. Rockafellar and S. Uryasev. Optimization of Conditional Value-at-Risk.J. Risk, 2(3): 21–41, 2000. doi: 10.21314/JOR.2000.038

    doi:10.21314/jor.2000.038 2000

  41. [42]

    D. Ruppert. Efficient estimations from a slowly convergent Robbins–Monro process. Technical Report 781, School of Operations Research and Industrial Engineering, Cornell University, Ithaca, NY, 1988

    1988

  42. [43]

    Z. Wang, Y. Wang, I. Narang, F. Wang, Y. Wang, and S. T. Maguluri. Steady-state behavior of constant-stepsize stochastic approximation: Gaussian approximation and tail bounds, 2026. arXiv:2602.13960

    arXiv 2026

  43. [44]

    Z. Wei, J. Li, Z. Lou, and W. B. Wu. Gaussian approximation and concentration of constant learning-rate stochastic gradient descent. InAdvances in Neural Information Processing Systems, volume 38, 2025

    2025

  44. [45]

    L. Yu, K. Balasubramanian, S. Volgushev, and M. A. Erdogdu. An analysis of constant step size SGD in the non-convex regime: Asymptotic normality and bias. InAdvances in Neural Information Processing Systems, volume 34, pages 4234–4248, 2021

    2021

  45. [46]

    Zhang, D

    Y. Zhang, D. L. Huo, Y. Chen, and Q. Xie. Prelimit coupling and steady-state convergence of constant-stepsize nonsmooth contractive SA.ACM SIGMETRICS Perform. Eval. Rev., 52(1): 35–36, 2024. doi: 10.1145/3673660.3655076

    arXiv 2024

  46. [47]

    Zhang, D

    Y. Zhang, D. L. Huo, Y. Chen, and Q. Xie. A piecewise Lyapunov analysis of sub-quadratic SGD: Applications to robust and quantile regression.ACM SIGMETRICS Perform. Eval. Rev., 53(1):85–87, 2025. doi: 10.1145/3744970.3727269. Full version: arXiv:2504.08178. 52

    arXiv 2025

  47. [1964]

    doi: 10.1214/aoms/1177703732

    arXiv