RELTA-SGLD: Relative-Growth Localized Taming for Nonconvex Stochastic-Gradient Langevin Learning

Yiwei Zhou; Ziheng Chen

A taming denominator that switches on only beyond a calibrated norm threshold, with strength set by the ratio of stochastic-gradient noise to inward drift, is claimed to stabilize superlinear SGLD while keeping stationary error O(λ) in W1 a

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 12:26 UTC pith:DZHVD2ZK

load-bearing objection A genuinely new taming idea—Q/I-calibrated λ-scale denominator—with credible first-order W1/W2 rates inside the stated radial-leading class; the pilot-run threshold s0 is the main unanalyzed layer, but the paper deserves a serious referee.

arxiv 2607.19544 v1 pith:DZHVD2ZK submitted 2026-07-21 stat.ML cs.LGmath.OC

RELTA-SGLD: Relative-Growth Localized Taming for Nonconvex Stochastic-Gradient Langevin Learning

Yiwei Zhou , Ziheng Chen This is my paper

classification stat.ML cs.LGmath.OC MSC 60J2265C3060H3568T07

keywords stochastic gradient Langevin dynamicstamed Langevin algorithmsnonconvex optimizationWasserstein stationary accuracyLyapunov stabilityrelative-growth taminglocalized tamingexcess risk complexity

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces RELTA-SGLD, a tamed stochastic-gradient Langevin algorithm whose denominator activates only beyond a norm threshold and whose strength is set by the ratio of the stochastic-gradient quadratic size to the available inward drift, rather than by the absolute size of the gradient. It claims that for nonconvex objectives with radial-leading superlinear stochastic-gradient oracles, the chain has bounded polynomial moments and its stationary law is O(λ) close to the target in W1, and O(λ) in W2 when the drift is a gradient (O(λ^{1/2}) in general). If correct, this yields finite-time excess-risk complexity of order ε^{-1} up to temperature bias, and it avoids the over-taming of globally active square-root-scale denominators: learning-region updates are virtually unchanged, and the far-tail return does not vanish as λ→0. The practical upshot is a stabilization method that intervenes when and where instability actually threatens, shown on Fashion-MNIST to beat untamed and conventional tamed SGLD while matching a tuned adaptive optimizer.

Core claim

The central claim is that stability and stationary accuracy of explicit SGLD with superlinearly growing stochastic-gradient oracles can be achieved by a denominator Dλ(θ)=√(1+(Kλ[‖θ‖^{2r}−s0]_+)^2), localized by threshold s0 and calibrated by the relative growth of the quadratic oracle term Q to the inward drift I. Under the radial-leading model H(θ,u)=G(θ,u)+ηθ‖θ‖^{2r} with ‖G‖≤K(u)(1+‖θ‖^q), q<2r+1, the exterior Q/I ratio is asymptotically ‖θ‖^{2r}, so λ‖θ‖^{2r} is the correct tail scale; the quadratic denominator interpolates between untamed learning (where the score is zero) and tail taming (D≈KλS). The paper proves polynomial Foster–Lyapunov stability, a unique invariant law with moment

What carries the argument

The relative-growth (Q/I) denominator Dλ(θ)=√(1+(Kλ[‖θ‖^{2r}−s0]_+)^2), where [·]_+ is the positive part, s0 is a norm threshold from a pilot run, and K is an absorption coefficient. The machinery is the one-step Lyapunov balance RλV2−V2 = −2λI/Dλ + λ²Q/Dλ² + 2λβ^{-1}d; the denominator is chosen so that λ²Q/Dλ² is controlled by λI/Dλ via Q(θ)≤C_{QI}I(θ)S_{s0}(θ), giving Dλ≳λS in the tail. The quadratic form keeps Dλ−1 of order λ² where taming is inactive, preserving the learning update and a nonvanishing far-tail return, and the threshold localizes taming to the tail. The proof also uses a Gaussian-mixture identity that upgrades the one-step stochastic-gradient coupling error from O(λ²) to O

Load-bearing premise

The load-bearing premise is that the stochastic-gradient oracle has the radial-leading structure H(θ,u)=G(θ,u)+ηθ‖θ‖^{2r} with sub-radial remainder ‖G‖≤K(u)(1+‖θ‖^q), q<2r+1, which guarantees the exterior Q/I ratio is asymptotically ‖θ‖^{2r}; if real stochastic gradients do not follow this radial form, the Q/I-calibrated denominator is not guaranteed to stabilize or to avoid over-taming, and the data-dependent threshold s0 adds an unquantified sensitivity.

What would settle it

Run RELTA-SGLD on a nonconvex problem whose stochastic-gradient oracle has a non-radial superlinear component—for instance, direction-dependent or quadratically growing noise—and test whether stationary W1/W2 errors still scale as λ rather than λ^{1/2} or worse; a loss of moment control or a log-log slope deteriorating toward 1/2 would falsify the claimed universality of the Q/I calibration. A complementary check is to compute the empirical Q(θ)/I(θ) ratio over the state space and see whether it stays bounded by a constant times [‖θ‖^{2r}−s0]_+; if Q/I grows faster in some directions, the deno

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

For nonconvex SGLD with radial-leading superlinear oracles satisfying the paper's assumptions, stationary accuracy improves to W1≤Cλ and, for gradient drifts, W2≤Cβλ; the general non-gradient bound is W2≤Cλ^{1/2}.
All fixed even polynomial moments are uniformly bounded in λ, and a unique invariant law exists with matching stationary moment bounds, so the chain is ergodic in a practical sense.
Finite-time excess risk satisfies Rn ≤ C_M e^{-cnλ} + C_M λ + C d log(1+β)/β, giving an ε^{-1} iteration complexity up to the unavoidable Gibbs bias.
The lighter localized denominator reduces relative attenuation of the learning update from O(√λ) to O(λ²) in the moderate region and keeps the far-tail return at linear order in ‖θ‖, so the scheme does not over-suppress ordinary training.
Samplewise nonsmooth stochastic-gradient oracles, including ReLU-type maps, are covered as long as the averaged drift is C², extending the W1 and gradient-case W2 rates to nonsmooth networks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The threshold s0 is set from a train-only pilot quantile; a natural testable extension is an online adaptive threshold that tracks the local Q/I balance, which the paper itself flags as future work.
The first-order W2 result relies on an LSI or one-sided squared-metric contraction of the target diffusion; for non-gradient drifts, verifying such conditions for specific SGLD applications (e.g., with momentum or preconditioning) would extend the sharp rate.
The Q/I relative-growth principle suggests taming strength could be tuned per parameter block or per layer rather than globally by norm, which may matter for heterogeneous network architectures.
The claim of nearly untamed ordinary training depends on the threshold being correctly placed: too small a threshold over-tames, too large fails to stabilize, so a sensitivity analysis of the quantile choice is a direct next step.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A genuinely new taming idea—Q/I-calibrated λ-scale denominator—with credible first-order W1/W2 rates inside the stated radial-leading class; the pilot-run threshold s0 is the main unanalyzed layer, but the paper deserves a serious referee.

read the letter

You should read this one if you track tamed Langevin methods. The new idea is the RELTA denominator: instead of TUSLA's √λ-scale absolute-growth taming, they set the tail strength from the ratio Q/I and localize it with a threshold. That is a real step forward, not a repackaging. The payoff is first-order stationary W1 and W2 accuracy for nonconvex SGLD with superlinear stochastic gradients, where comparable schemes give half-order W1 and quarter-order W2.

The theory is built carefully. Lemma 2.1 derives the exterior growth from the radial-leading oracle model, and the fixed-block decomposition in Section 4 is the strongest part: separating the stochastic-gradient channel from the mean-discretization channel, and using the Gaussian-mixture one-step defect plus conditionally centered coupling to get the extra λ. The proofs are detailed enough that I believe them. The authors are also honest about scope: Remark 4.14 states the non-gradient W2 first-order bound needs an additional one-sided contraction condition, and the potential-case W2 upgrade is justified through LSI/Talagrand machinery rather than magic.

Now the soft spots, in proportion. The theorems are conditional on the radial-leading oracle decomposition (2.3)–(2.4). That is an explicit assumption, so it is not a hidden flaw, but it is a real restriction: the Q/I calibration is only guaranteed for that radial class. If real stochastic gradients are anisotropic or otherwise not radial-leading, the scalar score [∥θ∥^{2r}−s0]_+ may not align with the actual Q/I balance. The paper says the denominator is a sufficient condition, and that is fair, but the practical reach is narrower than the abstract may suggest.

The bigger gap is the threshold s0. The theory treats it as fixed, but the algorithm sets it from a train-only pilot-run quantile. There is no sensitivity analysis and no high-probability treatment of that data-dependent choice. For a paper this careful, that is a conspicuous loose end.

Minor issues: the reproducibility package is referenced but not linked; the one-dimensional slope fits in Figure 1 and Table 4 have no error bars; and the comparison to "half-order/quarter-order" TUSLA rates is not pinned to specific theorem statements in the cited papers. None of these change my reading of the central argument.

Bottom line: this is a solid theory paper by authors who know the literature and do not oversell. It belongs at a good journal or conference; send it to a serious referee. The referee should ask them to analyze or at least discuss the pilot-threshold sensitivity and to pin down the comparison rates.

Referee Report

0 major / 5 minor

Summary. The paper introduces RELTA-SGLD, a tamed stochastic-gradient Langevin algorithm whose denominator is Dλ(θ)=√(1+(Kλ[∥θ∥^{2r}-s0]_+)^2). Under the radial-leading oracle decomposition H(θ,u)=G(θ,u)+ηθ∥θ∥^{2r} with ∥G∥≤K(u)(1+∥θ∥^q), q<2r+1, the authors prove polynomial Foster–Lyapunov stability, existence and uniqueness of the invariant law, and stationary Wasserstein bounds W1(π^SG_λ,π_β)=O(λ), W2=O(λ) in the gradient-drift case, and W2=O(λ^{1/2}) for general dissipative drifts. They also derive a finite-time excess-risk bound with Õ(ε^{-1}) iteration complexity (up to Gibbs bias). The experiments compare RELTA with TUSLA and untamed SGLD on Fashion-MNIST and verify the predicted λ-scale stationary W1/W2 slopes on a quartic double-well model.

Significance. If the results are correct, RELTA-SGLD improves on the √λ-scale TUSLA denominator in two ways: it reduces the relative attenuation of the learning update to O(λ²) before the far tail, and it avoids the vanishing O(√λ) far-tail return of TUSLA as λ→0. The proof mechanism is original and carefully explained: the denominator is calibrated from the exterior Q/I relative growth, and the first-order block estimates use a centered Gaussian-mixture one-step defect plus conditionally centered accumulation. The paper is honest about scope: Section 2.3 explicitly states that the Q/I balance is sufficient rather than necessary, and Section 6 lists nonradial calibrations and the non-gradient W2 case as future work. The theorems are conditional on the stated radial-leading oracle model, and the data-dependent pilot-run threshold s0 is not covered by the theory; these are limitations of applicability rather than internal inconsistencies.

minor comments (5)

[Abstract/§1] The abstract and introduction say the results hold for 'superlinearly growing stochastic-gradient oracles' without immediately stating the radial-leading decomposition (2.3)–(2.4). Since the entire Q/I calibration depends on this structure, please qualify the headline statements with the model class, or add a pointer to Assumption 1 at the first occurrence.
[§2.3/§5] The threshold s0 is set in practice from a train-only pilot-run quantile, but the theory treats s0 as a fixed parameter and the constants may depend on it. A short remark clarifying that the theorems apply conditionally on any fixed s0, and that the pilot-choice layer is a heuristic outside the formal guarantee, would prevent readers from overclaiming the practical coverage. A small sensitivity experiment on the choice of the quantile would also strengthen the empirical section.
[§1/References] Minor typographical issues: 'Key W ords' in the abstract, the mis-rendered 'R˝ockner–Wang' in Section 4.5, and the unusual formatting of 'e-TH εO POULA' in the introduction. Please normalize these.
[Lemma 4.5/Appendix A.2] The notation σλ and the Gaussian coordinate σG is easy to confuse. Define σλ explicitly as √(2β^{-1}λ) and use a different symbol (e.g., G) for the standard Gaussian variable throughout the proof of Lemma 4.5.
[Figure 1/Table 4] The pooled log–log slopes in Table 4 are computed from only the four smallest step sizes. Reporting per-seed slope variability or confidence intervals would make the W1/W2 slope separation from the √λ TUSLA baseline more convincing, especially since only 10 seeds are used.

Circularity Check

0 steps flagged

No load-bearing circularity: the rates are proved from the stated oracle model, self-citations are contextual, and no target quantity is used as an input.

full rationale

The claimed derivation chain is not circular. The denominator is designed from the one-step Lyapunov balance (2.2) and the exterior growth estimates of Lemma 2.1, which follow from the radial-leading model (2.3)-(2.4). The paper then proves Foster-Lyapunov stability (Theorems 3.1-3.2) and derives W1/W2 rates (Theorems 4.9, 4.12) from fixed-block comparison estimates and external contraction/functional-inequality results [11-14]. No theorem assumes its own conclusion, and no fitted parameter is renamed as a prediction: K and s0 are fixed hyperparameters in the theory, and the empirical rate measurements in Section 5 are verification rather than inputs to the proof. The self-citations [4,5] appear only as conceptual background ('our earlier work developed a local-taming viewpoint') and are not load-bearing in any proof. The pilot-run threshold calibration is an unanalyzed data-dependent implementation detail and therefore a scope caveat, not a circular step. Similarly, the radial-leading model limits applicability outside that class, but that is a conditionality issue rather than a definitional circularity. Given the minor non-load-bearing self-citations, the score is 2 rather than 0, but no circular step is established.

Axiom & Free-Parameter Ledger

2 free parameters · 8 axioms · 0 invented entities

The central claim rests on a radial-leading model of the stochastic gradient (2.3)-(2.4), Assumption 1 regularity/moment conditions, and standard external theorems from diffusion contraction and functional inequalities. The only fitted quantities are the taming coefficient K and the threshold s₀; both are hyperparameters with theory-prescribed constraints, but s₀ is calibrated from a pilot run. No new physical entities are introduced.

free parameters (2)

K (denominator coefficient) = 2 (Fashion-MNIST experiments)
Controls taming strength in the active tail and the far-tail return coefficient η/K. Theory requires K > max(C_rel/2, η/2); set to 2 in experiments, no tuning per seed.
s₀ (threshold; R₀^{2r}) = R₀ = empirical 0.8-quantile of pilot norm snapshots (Fashion-MNIST)
Localizes taming: below the threshold the denominator is inactive. Calibrated from a train-only untamed pilot run; theory treats s₀ as an input, and sensitivity is not analyzed.

axioms (8)

standard math EGZ multiplicative semimetric contraction theorem (Eberle, Guillin, Zimmer)
Invoked in Proposition 4.4 to obtain weighted contraction of the target diffusion (4.18).
standard math Cattiaux–Guillin–Wu LSI criterion
Used in Lemma 4.11 to establish a log-Sobolev inequality for the Gibbs measure from Lyapunov conditions and curvature lower bound.
standard math Otto–Villani Talagrand T2 and Röckner–Wang log-Harnack inequality
Used in Lemma 4.11 to turn LSI and one-sided monotonicity into W2 contraction toward πβ.
standard math Mattingly–Stuart–Higham Lyapunov–minorization ergodicity route
Underpins the existence/uniqueness of the invariant law in Section 3.3.
domain assumption Radial-leading oracle decomposition (2.3)–(2.4): H(θ,u)=G(θ,u)+ηθ||θ||^{2r}, ||G(θ,u)||≤K(u)(1+||θ||^q), q<2r+1
Required for Lemma 2.1 (exterior I/Q growth) and thus for the Q/I relative-growth scale that defines the RELTA denominator.
domain assumption Assumption 1(i): averaged drift g∈C² with ||∇g||≤C(1+||x||^{ℓ1}), ||∇²g||≤C(1+||x||^{ℓ2}), ℓ1<2r
Used in Lemma 4.2 (two-point geometry of the target drift) and Lemma 4.3 (taming consistency).
domain assumption Assumption 1(ii): E[||ζ(x,Z)||^{q0}|x]≤C(1+||x||^{M0}) for q0>4
Required for the Gaussian-mixture one-step defect (Lemma 4.5) to reach the O(λ³) squared scale.
domain assumption Potential structure h=∇U for Theorem 4.12 and Theorem 4.21
Needed for the explicit Gibbs form, LSI, and the first-order stationary W2 and excess-risk bounds.

pith-pipeline@v1.3.0-alltime-deepseek · 27444 in / 22828 out tokens · 182663 ms · 2026-08-01T12:26:08.206710+00:00 · methodology

0 comments

read the original abstract

We introduce RELTA-SGLD, a taming scheme that stabilizes superlinear stochastic-gradient updates while reducing unnecessary suppression of the original learning drift. A threshold determines where the taming turns on, while a relative-growth principle derived from the one-step Lyapunov stability condition determines the required taming strength. Together, they produce a lighter $\lambda$-scale denominator and preserve a nonvanishing far-tail return. As a consequence, we prove polynomial moment stability and first-order stationary accuracy in both $W_1$ and $W_2$ for nonconvex SGLD with superlinearly growing stochastic-gradient oracles, improving the corresponding half-order and quarter-order bounds for comparable stochastic-gradient tamed schemes. On Fashion-MNIST under active stabilization pressure, RELTA improves the mean learning metrics over both untamed SGLD and TUSLA and remains competitive with a tuned AdamW reference. In an ordinary-training regime, its lighter localized denominator reduces unnecessary perturbation of the original update and maintains nearly untamed learning dynamics.

Figures

Figures reproduced from arXiv: 2607.19544 by Yiwei Zhou, Ziheng Chen.

discussion (0)

Reference graph

Works this paper leans on

26 extracted references · 5 linked inside Pith

[1]

Hutzenthaler, A

M. Hutzenthaler, A. Jentzen, and P. E. Kloeden. Strong and weak divergence in finite time of Euler’s method for stochastic differential equations with non-globally Lipschitz continuous coefficients.Proceedings of the Royal Society A, 467:1563–1576, 2011

2011
[2]

S. Sabanis. A note on tamed Euler approximations.Electronic Communications in Probability, 18:1–10, 2013

2013
[3]

Brosse, A

N. Brosse, A. Durmus, E. Moulines, and S. Sabanis. The tamed unadjusted Langevin algorithm. Stochastic Processes and their Applications, 129(10):3638–3663, 2019

2019
[4]

Zhou and Z

Y. Zhou and Z. Chen. Deterministic envelopes for tamed SGLD: decoupling stochastic-gradient noise and localizing taming. arXiv:2606.05242, 2026

Pith/arXiv arXiv 2026
[5]

Zhou and Z

Y. Zhou and Z. Chen. Deterministic denominator design for localized tamed stochastic-gradient Langevin dynamics. arXiv:2606.10559, 2026

Pith/arXiv arXiv 2026
[6]

Lovas, I

A. Lovas, I. Lytras, M. R´ asonyi, and S. Sabanis. Taming neural networks with TUSLA: nonconvex learning via adaptive stochastic gradient Langevin algorithms.SIAM Journal on Mathematics of Data Science, 5(2):323–345, 2023

2023
[7]

D.-Y. Lim, A. Neufeld, S. Sabanis, and Y. Zhang. Non-asymptotic estimates for TUSLA algorithm for non-convex learning with applications to neural networks with ReLU activation function.IMA Journal of Numerical Analysis, 44(3):1464–1559, 2024

2024
[8]

Lim and S

D.-Y. Lim and S. Sabanis. Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks.Journal of Machine Learning Research, 25(53):1–52, 2024

2024
[9]

D.-Y. Lim, A. Neufeld, S. Sabanis, and Y. Zhang. Langevin dynamics based algorithm e-THεO POULA for stochastic optimization problems with discontinuous stochastic gradient. Mathematics of Operations Research, 50(3):2333–2374, 2025

2025
[10]

Neufeld, M

A. Neufeld, M. Ng Cheng En, and Y. Zhang. Non-asymptotic convergence bounds for modified tamed unadjusted Langevin algorithm in non-convex setting.Journal of Mathematical Analysis and Applications, 543(1):128892, 2025. 38

2025
[11]

Cattiaux, A

P. Cattiaux, A. Guillin, and L. Wu. A note on Talagrand’s transportation inequality and logarithmic Sobolev inequality.Probability Theory and Related Fields, 148(1–2):285–304, 2010

2010
[12]

Otto and C

F. Otto and C. Villani. Generalization of an inequality by Talagrand and links with the logarithmic Sobolev inequality.Journal of Functional Analysis, 173(2):361–400, 2000

2000
[13]

R¨ ockner and F.-Y

M. R¨ ockner and F.-Y. Wang. Log-Harnack inequality for stochastic differential equations in Hilbert spaces and its consequences.Infinite Dimensional Analysis, Quantum Probability and Related Topics, 13(1):27–37, 2010

2010
[14]

Eberle, A

A. Eberle, A. Guillin, and R. Zimmer. Quantitative Harris-type theorems for diffusions and McKean–Vlasov processes.Transactions of the American Mathematical Society, 371(10):7135– 7173, 2019

2019
[15]

Lytras and P

I. Lytras and P. Mertikopoulos. Tamed Langevin sampling under weaker conditions. In Proceedings of the 28th International Conference on Artificial Intelligence and Statistics, volume 258 ofProceedings of Machine Learning Research, pages 847–855, 2025

2025
[16]

Lytras and S

I. Lytras and S. Sabanis. Taming under isoperimetry.Stochastic Processes and their Applica- tions, 188:104684, 2025

2025
[17]

Lytras, S

I. Lytras, S. Sabanis, and Y. Zhang. kTULA: A Langevin sampling algorithm with improved KL bounds under super-linear log-gradients. arXiv:2506.04878, 2025

Pith/arXiv arXiv 2025
[18]

Z. Ju, L. Li, and Y. Wang. A modified tamed scheme for stochastic differential equations with superlinear drifts. arXiv:2507.09475, 2025

Pith/arXiv arXiv 2025
[19]

Lytras and A

I. Lytras and A. Ntousis. Error estimates for tamed Euler and randomized Euler schemes for SDEs with locally Lipschitz drift with applications to non-logconcave sampling and optimization. arXiv:2605.24937, 2026

Pith/arXiv arXiv 2026
[20]

Ren and F.-Y

P. Ren and F.-Y. Wang. Exponential convergence in entropy and Wasserstein for McKean– Vlasov SDEs.Nonlinear Analysis, 206:112259, 2021

2021
[21]

Raginsky, A

M. Raginsky, A. Rakhlin, and M. Telgarsky. Non-convex learning via stochastic gradient Langevin dynamics: a nonasymptotic analysis. InProceedings of the Conference on Learning Theory, 2017

2017
[22]

P. Xu, D. Zou, J. Chen, and Q. Gu. Global convergence of Langevin dynamics based algorithms for nonconvex optimization. InAdvances in Neural Information Processing Systems, pages 3122–3133, 2018

2018
[23]

Welling and Y

M. Welling and Y. W. Teh. Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the International Conference on Machine Learning, 2011

2011
[24]

S. J. Vollmer, K. C. Zygalakis, and Y. W. Teh. Exploration of the (non-)asymptotic bias and variance of stochastic gradient Langevin dynamics.Journal of Machine Learning Research, 17(159):1–48, 2016. 39

2016
[25]

J. C. Mattingly, A. M. Stuart, and D. J. Higham. Ergodicity for SDEs and approximations: locally Lipschitz vector fields and degenerate noise.Stochastic Processes and their Applications, 101(2):185–232, 2002

2002
[26]

Loshchilov and F

I. Loshchilov and F. Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2019. 40

2019

[1] [1]

Hutzenthaler, A

M. Hutzenthaler, A. Jentzen, and P. E. Kloeden. Strong and weak divergence in finite time of Euler’s method for stochastic differential equations with non-globally Lipschitz continuous coefficients.Proceedings of the Royal Society A, 467:1563–1576, 2011

2011

[2] [2]

S. Sabanis. A note on tamed Euler approximations.Electronic Communications in Probability, 18:1–10, 2013

2013

[3] [3]

Brosse, A

N. Brosse, A. Durmus, E. Moulines, and S. Sabanis. The tamed unadjusted Langevin algorithm. Stochastic Processes and their Applications, 129(10):3638–3663, 2019

2019

[4] [4]

Zhou and Z

Y. Zhou and Z. Chen. Deterministic envelopes for tamed SGLD: decoupling stochastic-gradient noise and localizing taming. arXiv:2606.05242, 2026

Pith/arXiv arXiv 2026

[5] [5]

Zhou and Z

Y. Zhou and Z. Chen. Deterministic denominator design for localized tamed stochastic-gradient Langevin dynamics. arXiv:2606.10559, 2026

Pith/arXiv arXiv 2026

[6] [6]

Lovas, I

A. Lovas, I. Lytras, M. R´ asonyi, and S. Sabanis. Taming neural networks with TUSLA: nonconvex learning via adaptive stochastic gradient Langevin algorithms.SIAM Journal on Mathematics of Data Science, 5(2):323–345, 2023

2023

[7] [7]

D.-Y. Lim, A. Neufeld, S. Sabanis, and Y. Zhang. Non-asymptotic estimates for TUSLA algorithm for non-convex learning with applications to neural networks with ReLU activation function.IMA Journal of Numerical Analysis, 44(3):1464–1559, 2024

2024

[8] [8]

Lim and S

D.-Y. Lim and S. Sabanis. Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks.Journal of Machine Learning Research, 25(53):1–52, 2024

2024

[9] [9]

D.-Y. Lim, A. Neufeld, S. Sabanis, and Y. Zhang. Langevin dynamics based algorithm e-THεO POULA for stochastic optimization problems with discontinuous stochastic gradient. Mathematics of Operations Research, 50(3):2333–2374, 2025

2025

[10] [10]

Neufeld, M

A. Neufeld, M. Ng Cheng En, and Y. Zhang. Non-asymptotic convergence bounds for modified tamed unadjusted Langevin algorithm in non-convex setting.Journal of Mathematical Analysis and Applications, 543(1):128892, 2025. 38

2025

[11] [11]

Cattiaux, A

P. Cattiaux, A. Guillin, and L. Wu. A note on Talagrand’s transportation inequality and logarithmic Sobolev inequality.Probability Theory and Related Fields, 148(1–2):285–304, 2010

2010

[12] [12]

Otto and C

F. Otto and C. Villani. Generalization of an inequality by Talagrand and links with the logarithmic Sobolev inequality.Journal of Functional Analysis, 173(2):361–400, 2000

2000

[13] [13]

R¨ ockner and F.-Y

M. R¨ ockner and F.-Y. Wang. Log-Harnack inequality for stochastic differential equations in Hilbert spaces and its consequences.Infinite Dimensional Analysis, Quantum Probability and Related Topics, 13(1):27–37, 2010

2010

[14] [14]

Eberle, A

A. Eberle, A. Guillin, and R. Zimmer. Quantitative Harris-type theorems for diffusions and McKean–Vlasov processes.Transactions of the American Mathematical Society, 371(10):7135– 7173, 2019

2019

[15] [15]

Lytras and P

I. Lytras and P. Mertikopoulos. Tamed Langevin sampling under weaker conditions. In Proceedings of the 28th International Conference on Artificial Intelligence and Statistics, volume 258 ofProceedings of Machine Learning Research, pages 847–855, 2025

2025

[16] [16]

Lytras and S

I. Lytras and S. Sabanis. Taming under isoperimetry.Stochastic Processes and their Applica- tions, 188:104684, 2025

2025

[17] [17]

Lytras, S

I. Lytras, S. Sabanis, and Y. Zhang. kTULA: A Langevin sampling algorithm with improved KL bounds under super-linear log-gradients. arXiv:2506.04878, 2025

Pith/arXiv arXiv 2025

[18] [18]

Z. Ju, L. Li, and Y. Wang. A modified tamed scheme for stochastic differential equations with superlinear drifts. arXiv:2507.09475, 2025

Pith/arXiv arXiv 2025

[19] [19]

Lytras and A

I. Lytras and A. Ntousis. Error estimates for tamed Euler and randomized Euler schemes for SDEs with locally Lipschitz drift with applications to non-logconcave sampling and optimization. arXiv:2605.24937, 2026

Pith/arXiv arXiv 2026

[20] [20]

Ren and F.-Y

P. Ren and F.-Y. Wang. Exponential convergence in entropy and Wasserstein for McKean– Vlasov SDEs.Nonlinear Analysis, 206:112259, 2021

2021

[21] [21]

Raginsky, A

M. Raginsky, A. Rakhlin, and M. Telgarsky. Non-convex learning via stochastic gradient Langevin dynamics: a nonasymptotic analysis. InProceedings of the Conference on Learning Theory, 2017

2017

[22] [22]

P. Xu, D. Zou, J. Chen, and Q. Gu. Global convergence of Langevin dynamics based algorithms for nonconvex optimization. InAdvances in Neural Information Processing Systems, pages 3122–3133, 2018

2018

[23] [23]

Welling and Y

M. Welling and Y. W. Teh. Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the International Conference on Machine Learning, 2011

2011

[24] [24]

S. J. Vollmer, K. C. Zygalakis, and Y. W. Teh. Exploration of the (non-)asymptotic bias and variance of stochastic gradient Langevin dynamics.Journal of Machine Learning Research, 17(159):1–48, 2016. 39

2016

[25] [25]

J. C. Mattingly, A. M. Stuart, and D. J. Higham. Ergodicity for SDEs and approximations: locally Lipschitz vector fields and degenerate noise.Stochastic Processes and their Applications, 101(2):185–232, 2002

2002

[26] [26]

Loshchilov and F

I. Loshchilov and F. Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2019. 40

2019