Pith. sign in

REVIEW 3 major objections 2 minor 57 references

Regime-Switching Langevin Monte Carlo Algorithms

T0 review · 3 major / 2 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes regime-switching Langevin samplers and proves an O(1/√ε) iteration bound for the frictional kinetic variant.

desk verdict Useful new random-stepsize LMC/KLMC results, but the FRS-KLMC O(1/sqrt(ε)) acceleration rests on a joint-vs-marginal Wasserstein error that likely kills it. read the letter →

arxiv 2509.00941 v1 pith:LBGUHF2M submitted 2025-08-31 stat.CO cs.LGmath.PRstat.ML

Xiaoyu Wang , Yingli Wang , Lingjiong Zhu This is my paper
classification stat.COcs.LGmath.PRstat.ML MSC 60J2265C0562F15
keywords regime-switchingLangevindynamicskineticMonteCarlo2-Wassersteinconvergencenon-asymptoticcomplexityrandomstepsizefrictionGibbssamplingMarkovchain
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces three sampling algorithms—regime-switching Langevin Monte Carlo (RS-LMC), regime-switching kinetic Langevin Monte Carlo (RS-KLMC), and frictional-regime-switching kinetic Langevin Monte Carlo (FRS-KLMC)—in which a Markov chain switches the stepsize or friction coefficient over time. It proves that each discrete algorithm converges to the Gibbs target π(x) ∝ e^{−f(x)} in 2-Wasserstein distance and gives explicit iteration complexities: O(ε^{−2} log(1/ε)), O(ε^{−1} log(1/ε)), and O(ε^{−1/2} log(1/ε)) respectively. The key selling point is the last bound, which would accelerate sampling relative to standard kinetic Langevin Monte Carlo. The analysis works by conditioning on the switching chain and applying synchronous coupling, then using Perron–Frobenius theory to control the expected contraction.

What carries the argument

The central objects are a finite-state continuous-time Markov chain (CTMC) with generator Q modulating the stepsize or friction coefficient, and the tilted transition operator T_θ = P(η)e^{−θΛ} whose Perron–Frobenius eigenvalue controls the contraction rate α. Conditioning on the switching chain reduces each step to a standard LMC/KLMC step with effective stepsize β_kη or friction γ_k; the synchronous-coupling error bounds from [DK19] and [DRD20] then apply one step at a time. For FRS-KLMC, a constant-friction contraction for W2 is applied piecewise on each switching interval. The spectral radius analysis of T_θ—using the matrix Q − mΛ (respectively Q − (m/γ)Λ and Q − 2mΛ_γ^{−1})—gives the e

What would settle it

Run FRS-KLMC on a strongly convex quadratic with the paper's friction regimes, starting from v0 ∼ N(0,I); estimate the 2-Wasserstein distance between the empirical velocity marginal and N(0,I) after a few hundred iterations. If it is not negligible while the position has not yet mixed, the process-error premise fails; a second check is to plot the empirical W2 position error against K and see whether it decays like K^{−1/2} (claimed) or only like K^{−1}.

Watch

Extended reading notes

Core claim

Central claim: a finite-state continuous-time Markov chain can randomize the stepsize (RS-LMC, RS-KLMC) or the friction (FRS-KLMC) without moving the target, which remains the Gibbs distribution π ∝ e^{−f}. The authors prove non-asymptotic 2-Wasserstein bounds for all three discretizations. Conditioning on the switching chain turns each step into a standard LMC/KLMC step with effective stepsize β_k η or effective friction γ_k; taking expectations over the chain yields a contraction governed by the spectral radius of the tilted operator T_θ = P(η) e^{−θΛ}. The headline bound is W2(ν_K, π) ≤ √2(1 − αη/2)^{K/2}W2(ν_0, π) + C_B η² for FRS-KLMC, which yields iteration complexity O(ε^{−1/2} log(1/

Load-bearing premise

The FRS-KLMC bound rests on the assumption that the velocity marginal stays exactly standard Gaussian throughout the iterations, even though the update rule (3.12) perturbs the velocity whenever the gradient is nonzero—if that marginal drifts, the O(1/√ε) rate is not established.

Editorial extensions

If this is right

  • If the FRS-KLMC bound is correct, reaching ε accuracy in 2-Wasserstein distance costs O(ε^{−1/2} log(1/ε)) iterations, versus O(ε^{−1}) for standard KLMC.
  • The RS-KLMC bound gives dimension dependence √d and condition-number dependence κ, improving on RS-LMC's d and κ².
  • Larger spectral gaps of Q and larger regime values β̄_i (or appropriate friction values γ̄_i) speed up convergence through a larger α; the numerical experiments confirm this on Bayesian linear regression.
  • Because the algorithms are still LMC/KLMC with random coefficients, they can be combined with stochastic gradients (the paper's experiments use SGLD/SGHMC versions) and inherit the same per-iteration cost as standard LMC/KLMC.
  • The theoretical guarantees are non-asymptotic and stated in W2, so they give concrete finite-step error bounds under the m-strongly-convex and M-smooth assumptions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Not proven in the paper, but the conditioning-on-the-CTMC technique is modular and could likely be transplanted to other coefficient-randomized samplers.
  • If the velocity-marginal gap in the FRS-KLMC proof is real, a repair may require re-randomizing the velocity after each friction switch or tracking the joint position–velocity Wasserstein distance; a natural test is to measure the velocity marginal in the Bayesian regression experiments.
  • The random-friction mechanism, not just the random stepsize, appears to drive the square-root acceleration; comparing FRS-KLMC with RS-KLMC on the same target would isolate the effect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The paper introduces regime-switching Langevin Monte Carlo (RS-LMC), regime-switching kinetic Langevin Monte Carlo (RS-KLMC), and frictional-regime-switching kinetic Langevin Monte Carlo (FRS-KLMC) algorithms. These can be viewed as LMC/KLMC with random stepsizes or random frictions, driven by a finite-state continuous-time Markov chain. The authors prove that the associated continuous-time processes have the Gibbs distribution as an invariant x-marginal, derive non-asymptotic 2-Wasserstein convergence bounds for the discretized algorithms via synchronous coupling and Perron-Frobenius analysis, and report iteration complexities of O(1/epsilon^2), O(1/epsilon), and O(1/sqrt(epsilon)) for the three algorithms. Numerical experiments on Bayesian linear and logistic regression compare the proposed samplers with classical LMC/KLMC in terms of MSE and classification accuracy.

Significance. If the results are correct, the paper provides the first non-asymptotic W2 guarantees for regime-switching Langevin samplers, with explicitly computed constants that depend on no fitted parameters. The RS-KLMC bound, if valid, would match the known O(1/epsilon) complexity of kinetic Langevin Monte Carlo while allowing randomized stepsizes, and the FRS-KLMC O(1/sqrt(epsilon)) claim would be a substantial acceleration. The proofs are largely self-contained and reuse only standard black-box estimates from Dalalyan-Karagulyan and Dalalyan-Riou-Durand. The numerical section supports the empirical usefulness of regime switching, though it does not directly test the W2 error bounds. The central FRS-KLMC acceleration claim, however, rests on an invalid identification of joint and marginal Wasserstein distances, and the current analysis also has fixable but load-bearing technical gaps in the tilted-matrix arguments.

major comments (3)
  1. [Appendix A.3 / Propositions 5, 11, 16] The process-error step identifies W2(mu P^X_{K eta}, mu' P^X_{K eta}) with W2(nu_K, pi) after setting mu = N(0,I_d) ⊗ nu_0 and mu' = N(0,I_d) ⊗ pi. This equality holds only if at step K the algorithm's velocity marginal is N(0,I_d) and independent of x_K. Update (3.12), v_{k+1} = e^{-gamma_k eta} v_k - psi_1(eta,gamma_k) grad f(x_k) + sqrt(2 gamma_k) xi, does not preserve standard Gaussian velocity: the conditional mean is -psi_1 grad f(x_k), which is nonzero away from stationarity. The DRD20 contraction quoted in (A.80) acts on joint laws, not on position marginals. Dropping the velocity discrepancy removes the kinetic velocity-accumulation error, which is O(eta) in W2; restoring it turns the claimed O(eta^2) bias of Theorem 17 into the standard O(eta) bias. The problem is sharp for N=1: FRS-KLMC is the constant-gamma KLMC of [DRD20], and Proposition 16 would give it an O(eta^2) W2 bias
  2. [Appendix A.3 / Propositions 5, 11, 16] The spectral-radius bound rho(T_{m eta}) <= 1 - alpha eta + C_M eta^2 is converted to 1 - alpha eta/2 by imposing eta <= alpha/(2 C_M). This condition is used in the proof but is absent from the statement of Proposition 5, and it is also not stated when the same argument is reused in Propositions 11 and 16. Without this step-size restriction, the factor (1 - alpha eta/2)^k in the central recursions is not justified. The condition should be added explicitly to the propositions and carried through the complexity corollaries.
  3. [Appendix A.3, Eq. (A.21)] The conditional expectation satisfies u_{K+1}(i) = exp(-theta beta_i) sum_j P_{ij}(eta) u_K(j), so the tilted transition matrix should be T_theta = Lambda_theta P(eta), where (Lambda_theta)_{ii} = exp(-theta beta_i). The proof instead defines (T_theta)_{ij} = P_{ij}(eta) exp(-theta beta_j), i.e. T_theta = P(eta) Lambda_theta. For square matrices, P D and D P have the same spectrum, so the spectral-radius computation may be repairable, but the Perron-Frobenius recursion and the perturbation expansion are written for the wrong operator and must be corrected.
minor comments (2)
  1. [Section 4.2] The logistic-regression experiments use stochastic gradients, while the theoretical guarantees in Theorems 6, 12, and 17 assume full gradients with strongly convex, smooth potentials. The experiments are therefore useful heuristics for the algorithms but do not test the W2 claims.
  2. [Throughout] There are typographical errors: 'Baysesian' in the introduction, 'regime-swithcing' in several figure captions, and 'metrix' in Section 4.1. In the proof of Corollary 7, W2(nu_{beta,0}, pi) should be W2(nu_0, pi).

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the rates are derived from external contraction bounds plus a self-contained Perron–Frobenius analysis; self-citations are not load-bearing.

full rationale

The paper's central derivation chain is not circular. The discrete-time analyses condition on the regime path and then invoke the external constant-stepsize contraction bounds of DK19 and DRD20; the regime-switching expectation is handled by a self-contained Perron–Frobenius analysis of the tilted transition matrix, with the decay rate α and constants C computed from the declared generator Q and regime values. The iteration complexities in Corollaries 7, 13, and 18 are algebraic consequences of the stated recursions: e.g., Corollary 18 combines the η^2 bias term of Theorem 17 with the contraction factor and chooses η = O(√ε), giving K = O(ε^{-1/2} log(1/ε)) by direct substitution, not by fitting or renaming an output as an input. The invariant-distribution theorems are verified directly through generator adjoint calculations. The only self-citations (GGHZ21, GIWZ24, GGZ20, GGZ22) appear as background, motivation, or experimental comparison and do not support any load-bearing step of the proofs; no uniqueness theorem or ansatz is imported from the authors' prior work. The skeptic's concern about FRS-KLMC—that the proof identifies the joint Wasserstein distance with the position marginal W2(ν_K, π) in Proposition 16—is a potential technical gap in the contraction argument, but it is not a circular reduction of the claim to its own inputs; it is a correctness issue about the validity of the bound, not a self-referential or definitional equivalence. Accordingly, no circular step is identified and the score is 0.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No fitted constants appear in the derivation; the bounds hold for any user-chosen regime set and generator. The only input assumptions are standard convexity and smoothness and CTMC irreducibility.

assumptions (5)
  • standard math f is m-strongly convex and M-smooth (Assumption 1)
    Used in every coupling step to control contraction; standard in LMC literature.
  • standard math Regime process beta(t) is an irreducible finite-state CTMC with generator Q (Assumption 2)
    Ensures unique stationary distribution psi and spectral gap used in the rates.
  • domain assumption For FRS results, min_i gamma_i >= max(sqrt(2), sqrt(M+m))
    Sufficient condition for contraction of kinetic Langevin coupling used in Theorem 15 and Proposition 16.
  • standard math Initial velocity V(0) ~ N(0,I) and beta(0) ~ psi in continuous-time theorems
    Boundary conditions for coupling arguments; standard in Langevin analysis.
  • standard math Synchronous coupling one-step bounds of DK19 and DRD20 hold
    The paper imports these bounds verbatim (Eq. A.20, A.54) to handle each fixed-regime step.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Regime-Switching Langevin Monte Carlo Algorithms." pith.science (2026). https://pith.science/paper/LBGUHF2M

@misc{pith2026250900941,
  author       = {Pith},
  title        = {Pith review of: Regime-Switching Langevin Monte Carlo Algorithms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LBGUHF2M}},
  note         = {Machine review of arXiv:2509.00941}
}
read the original abstract

Langevin Monte Carlo (LMC) algorithms are popular Markov Chain Monte Carlo (MCMC) methods to sample a target probability distribution, which arises in many applications in machine learning. Inspired by regime-switching stochastic differential equations in the probability literature, we propose and study regime-switching Langevin dynamics (RS-LD) and regime-switching kinetic Langevin dynamics (RS-KLD). Based on their discretizations, we introduce regime-switching Langevin Monte Carlo (RS-LMC) and regime-switching kinetic Langevin Monte Carlo (RS-KLMC) algorithms, which can also be viewed as LMC and KLMC algorithms with random stepsizes. We also propose frictional-regime-switching kinetic Langevin dynamics (FRS-KLD) and its associated algorithm frictional-regime-switching kinetic Langevin Monte Carlo (FRS-KLMC), which can also be viewed as the KLMC algorithm with random frictional coefficients. We provide their 2-Wasserstein non-asymptotic convergence guarantees to the target distribution, and analyze the iteration complexities. Numerical experiments using both synthetic and real data are provided to illustrate the efficiency of our proposed algorithms.

Figures

Figures reproduced from arXiv: 2509.00941 by the authors.

Figure 1
Figure 1. 0 10 20 30 40 50 Iterations 10 0 3 × 10 1 4 × 10 1 6 × 10 1 2 × 10 0 MSE MSE Convergence for RS-LMC and LMC RS-LMC Small Range with beta = [0.5,0.6,0.7,0.8,0.9] LMC with const beta = 1.0 0 10 20 30 40 50 Iterations 10 0 3 × 10 1 4 × 10 1 6 × 10 1 2 × 10 0 MSE MSE Convergence for RS-LMC and LMC RS-LMC Large Range with beta = [0.1,1.0,1.8,2.6,4.0] LMC with const beta = 1.0 [PITH_FULL_IMAGE:figures/full_fig_p017_1.png] view at source ↗
Figure 2
Figure 2. MSE for RS-KLMC and KLMC. We observe in [PITH_FULL_IMAGE:figures/full_fig_p018_2.png] view at source ↗
Figure 3
Figure 3. MSE for FRS-KLMC and KLMC. 4.2 Bayesian Logistic Regression In this section, we aim to test the performance of our algorithms in binary classification problems by considering the Bayesian logistic regression model on both synthetic and real data (Iris4 and MAGIC Gamma Telescope5 ). Suppose we have access to a dataset Z = {zj} n j=1 where zj = (Xj , yj ), Xj ∈ R d are the features and yj ∈ {0, 1} are the labels with … view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Comparisons within regime-switching algorithms over the synthetic data. [PITH_FULL_IMAGE:figures/full_fig_p020_4.png]
Figure 5
Figure 5. Figure 5: Comparisons between regime-switching (RS-SGLD, RS-SGHMC, FRS-SGHMC) and [PITH_FULL_IMAGE:figures/full_fig_p021_5.png]
Figure 6
Figure 6. Figure 6: Comparisons within regime-switching algorithms over the real data. [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]
Figure 7
Figure 7. Figure 7: Comparing RS-SGLD to SGLD. These plots demonstrate that RS-SGLD is more stable than SGLD for achieving the same accuracy, even with a small batch-size on both large and small sample sets. In the example using the Iris dataset (150 samples), SGLD exhibits unstable chang…
Figure 8
Figure 8. Figure 8: Comparing RS-SGHMC, FRS-SGHMC to SGHMC. also be viewed as the LMC and KLMC algorithms with random stepsizes. We also proposed frictional-regime-switching kinetic Langevin dynamics (FRS-KLD) and its associated algorithm frictional-regime-switching kinetic Langevin Monte…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 55 canonical work pages

  1. [1]

    An introduction to MCMC for machine learning

    Christophe Andrieu, Nando De Freitas, Arnaud Doucet, and Michael I Jordan. An introduction to MCMC for machine learning. Machine Learning , 50(1):5--43, 2003

    work page 2003

  2. [2]

    Basak, Arnab Bisi, and Mrinal K

    Gopal K. Basak, Arnab Bisi, and Mrinal K. Ghosh. Stability of a random diffusion with linear drift. Journal of Mathematical Analysis and Applications , 202(2):604--622, 1996

    work page 1996

  3. [3]

    Rate of convergence for ergodic continuous M arkov processes: L ypaunov versus P oincar\' e

    Dominique Bakry, Patrick Cattiaux, and Arnaud Guillin. Rate of convergence for ergodic continuous M arkov processes: L ypaunov versus P oincar\' e . Journal of Functional Analysis , 254:727--759, 2008

    work page 2008

  4. [4]

    On stochastic gradient L angevin dynamics with dependent data streams in the logconcave case

    Mathias Barkhagen, Ngoc Huy Chau, \' E ric Moulines, Miklos R\' a sonyi, Sotirios Sabanis, and Ying Zhang. On stochastic gradient L angevin dynamics with dependent data streams in the logconcave case. Bernoulli , 27(1):1--33, 2021

    work page 2021

  5. [5]

    Bartlett

    Xiang Cheng and Peter L. Bartlett. Convergence of L angevin MCMC in KL -divergence. In Proceedings of the 29th International Conference on Algorithmic Learning Theory (ALT) , volume 83, pages 186--211. PMLR, 2018

    work page 2018

  6. [6]

    Sharp Convergence Rates for L angevin Dynamics in the Nonconvex Setting

    Xiang Cheng , Niladri S Chatterji , Yasin Abbasi-Yadkori , Peter L Bartlett , and Michael I Jordan . Sharp Convergence Rates for L angevin Dynamics in the Nonconvex Setting . arXiv:1805.01648 , 2018

    arXiv 2018

  7. [7]

    Underdamped L angevin MCMC : A non-asymptotic analysis

    Xiang Cheng, Niladri S Chatterji, Peter L Bartlett, and Michael I Jordan. Underdamped L angevin MCMC : A non-asymptotic analysis. In Proceedings of the 31st Annual Conference on Learning Theory , volume 75, pages 300--323. PMLR, 2018

    work page 2018

  8. [8]

    Exponential ergodicity for M arkov processes with random switching

    Bertrand Cloez and Martin Hairer. Exponential ergodicity for M arkov processes with random switching. Bernoulli , 21(1):505--536, 2015

    work page 2015

Show all 57 references
  1. [9]

    Complexity of randomized algorithms for underdamped L angevin dynamics

    Yu Cao, Jianfeng Lu, and Lihan Wang. Complexity of randomized algorithms for underdamped L angevin dynamics. Communications in Mathematical Sciences , 19(7):1827--1853, 2021

    2021

  2. [10]

    On explicit L^ 2 -convergence rate estimate for underdamped L angevin dynamics

    Yu Cao, Jianfeng Lu, and Lihan Wang. On explicit L^ 2 -convergence rate estimate for underdamped L angevin dynamics. Archive for Rational Mechanics and Analysis , 247(90):1--34, 2023

    2023

  3. [11]

    On stochastic gradient L angevin dynamics with dependent data streams: the fully non-convex case

    Ngoc Huy Chau, \' E ric Moulines, Miklos R\' a sonyi, Sotirios Sabanis, and Ying Zhang. On stochastic gradient L angevin dynamics with dependent data streams: the fully non-convex case. SIAM Journal of Mathematics of Data Science , 3(3):959--986, 2021

    2021

  4. [12]

    Theoretical guarantees for approximate sampling from smooth and log-concave densities

    Arnak S Dalalyan. Theoretical guarantees for approximate sampling from smooth and log-concave densities. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 79(3):651--676, 2017

    2017

  5. [13]

    User-friendly guarantees for the L angevin M onte C arlo with inaccurate gradient

    Arnak S Dalalyan and Avetik Karagulyan. User-friendly guarantees for the L angevin M onte C arlo with inaccurate gradient. Stochastic Processes and their Applications , 129(12):5278--5311, 2019

    2019

  6. [14]

    Non-asymptotic convergence analysis for the U nadjusted L angevin A lgorithm

    Alain Durmus and Eric Moulines. Non-asymptotic convergence analysis for the U nadjusted L angevin A lgorithm. Annals of Applied Probability , 27(3):1551--1587, 2017

    2017

  7. [15]

    High-dimensional B ayesian inference via the U nadjusted L angevin A lgorithm

    Alain Durmus and Eric Moulines. High-dimensional B ayesian inference via the U nadjusted L angevin A lgorithm. Bernoulli , 25(4A):2854--2882, 2019

    2019

  8. [16]

    Hypocoercivity for linear kinetic equations conserving mass

    Jean Dolbeault, Cl\' e ment Mouhot, and Christian Schmeiser. Hypocoercivity for linear kinetic equations conserving mass. Transactions of the American Mathematical Society , 367:3807--3828, 2015

    2015

  9. [17]

    On sampling from a log-concave density using kinetic L angevin diffusions

    Arnak S Dalalyan and Lionel Riou-Durand. On sampling from a log-concave density using kinetic L angevin diffusions. Bernoulli , 26(3):1956--1988, 2020

    1956

  10. [18]

    Ermak and Helen Buckholz

    Donald L. Ermak and Helen Buckholz. Numerical integration of the L angevin equation: M onte C arlo simulation. Journal of Computational Physics , 35:169--182, 1980

    1980

  11. [19]

    Couplings and quantitative contraction rates for L angevin dynamics

    Andreas Eberle, Arnaud Guillin, and Raphael Zimmer. Couplings and quantitative contraction rates for L angevin dynamics. Annals of Probability , 47(4):1982--2010, 2019

    1982

  12. [20]

    Erdogdu, Rasa Hosseinzadeh, and S

    Murat A. Erdogdu, Rasa Hosseinzadeh, and S. Zhang, Matthew. Convergence analysis of L angevin M onte C arlo in chi-square and R \' e nyi divergence. In Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS) , volume 151, pages 8151...

    2022

  13. [21]

    Bayesian Data Analysis

    Andrew Gelman, John B Carlin, Hal S Stern, and Donald B Rubin. Bayesian Data Analysis . Chapman & Hall/CRC Press, 1995

    1995

  14. [22]

    Decentralized stochastic gradient L angevin dynamics and H amiltonian M onte C arlo

    Mert G \"u rb \"u zbalaban, Xuefeng Gao, Yuanhan Hu, and Lingjiong Zhu. Decentralized stochastic gradient L angevin dynamics and H amiltonian M onte C arlo. The Journal of Machine Learning Research , 22(1):10804--10872, 2021

    2021

  15. [23]

    Breaking reversibility accelerates L angevin dynamics for global non-convex optimization

    Xuefeng Gao, Mert G\" u rb\" u zbalaban, and Lingjiong Zhu. Breaking reversibility accelerates L angevin dynamics for global non-convex optimization. In Advances in Neural Information Processing Systems (NeurIPS) , volume 33, pages 17850--17862. Curran Associates, Inc., 2020

    2020

  16. [24]

    Global convergence of S tochastic G radient H amiltonian M onte C arlo for non-convex stochastic optimization: Non-asymptotic performance bounds and momentum-based acceleration

    Xuefeng Gao, Mert G\" u rb\" u zbalaban, and Lingjiong Zhu. Global convergence of S tochastic G radient H amiltonian M onte C arlo for non-convex stochastic optimization: Non-asymptotic performance bounds and momentum-based acceleration. Operations Research , 70:2931--2947, 2022

    2022

  17. [25]

    Cyclic and randomized stepsizes invoke heavier tails in SGD than constant stepsize

    Mert G\" u rb\" u zbalaban, Yuanhan Hu, Umut S ims e kli, and Lingjiong Zhu. Cyclic and randomized stepsizes invoke heavier tails in SGD than constant stepsize. Transactions on Machine Learning Research , 08:1--15, 2023

    2023

  18. [26]

    Generalized EXTRA decentralized stochastic gradient L angevin dynamics

    Mert G\" u rb\" u zbalaban, Rafiq Islam, Xiaoyu Wang, and Lingjiong Zhu. Generalized EXTRA decentralized stochastic gradient L angevin dynamics. arXiv:2412.01993 , 2024

    2024

  19. [27]

    Deep reinforcement learning using cyclical learning rates

    Ralf Gulde, Marc Tuscher, Akos Csiszar, Oliver Riedel, and Alexander Verl. Deep reinforcement learning using cyclical learning rates. In 2020 Third International Conference on Artificial Intelligence for Industries (AI4I) , pages 32--35. IEEE, 2020

    2020

  20. [28]

    Hopcroft, and Kilian Q

    Gao Huang, Yixuan Li, Geoff Pleiss, Zhuang Liu, John E. Hopcroft, and Kilian Q. Weinberger. Snapshot ensembles: Train 1, get M for free. In International Conference on Learning Representations , 2017

    2017

  21. [29]

    Isotropic hypoellipticity and trend to equilibrium for the F okker- P lanck equation with a high-degree potential

    Fr \'e d \'e ric H \'e rau and Francis Nier. Isotropic hypoellipticity and trend to equilibrium for the F okker- P lanck equation with a high-degree potential. Archive for Rational Mechanics and Analysis , 171(2):151--218, 2004

    2004

  22. [30]

    Steepest descent method with random step lengths

    Zden e k Kalousek. Steepest descent method with random step lengths. Foundations of Computational Mathematics , 17:359--422, 2017

    2017

  23. [31]

    Markov Chains and Mixing Times , volume 107

    David A Levin and Yuval Peres. Markov Chains and Mixing Times , volume 107. American Mathematical Soc., 2017

    2017

  24. [32]

    Chatterji, Xiang Cheng, Nicolas Flammarion, Peter L

    Yi-An Ma, Niladri S. Chatterji, Xiang Cheng, Nicolas Flammarion, Peter L. Bartlett, and Michael I. Jordan. Is there an analog of N esterov acceleration for gradient-based MCMC ? Bernoulli , 27(3):1942--1992, 2021

    1942

  25. [33]

    Mixing of H amiltonian M onte C arlo on strongly log-concave distributions 2: Numerical integrators

    Oren Mangoubi and Aaron Smith. Mixing of H amiltonian M onte C arlo on strongly log-concave distributions 2: Numerical integrators. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS) , volume 89, pages 586--595. PMLR, 2019

    2019

  26. [34]

    Mixing of H amiltonian M onte C arlo on strongly log-concave distributions: Continuous dynamics

    Oren Mangoubi and Aaron Smith. Mixing of H amiltonian M onte C arlo on strongly log-concave distributions: Continuous dynamics. Annals of Applied Probability , 31(5):2019--2045, 2021

    2019

  27. [35]

    Ergodicity for SDE s and approximations: locally L ipschitz vector fields and degenerate noise

    Jonathan C Mattingly, Andrew M Stuart, and Desmond J Higham. Ergodicity for SDE s and approximations: locally L ipschitz vector fields and degenerate noise. Stochastic Processes and their Applications , 101(2):185--232, 2002

    2002

  28. [36]

    Stochastic gradient descent with random learning rate

    Daniele Musso. Stochastic gradient descent with random learning rate. arXiv preprint arXiv:2003.06926 , 2020

    2003 arXiv

  29. [37]

    Stochastic Differential Equations with Markovian Switching

    Xuerong Mao and Chenggui Yuan. Stochastic Differential Equations with Markovian Switching . Imperial College Press, London, 2006

    2006

  30. [38]

    Stochastic Processes and Applications: Diffusion processes, the F okker- P lanck and L angevin Equations , volume 60

    Grigorios A Pavliotis. Stochastic Processes and Applications: Diffusion processes, the F okker- P lanck and L angevin Equations , volume 60. Springer, 2014

    2014

  31. [39]

    Non-convex learning via stochastic gradient L angevin dynamics: a nonasymptotic analysis

    Maxim Raginsky, Alexander Rakhlin, and Matus Telgarsky. Non-convex learning via stochastic gradient L angevin dynamics: a nonasymptotic analysis. In Conference on Learning Theory , volume 65, pages 1674--1703. PMLR, 2017

    2017

  32. [40]

    Some remarks and examples concerning the transience and recurrence of random diffusions

    Pinsky Ross and Michael Scheutzow. Some remarks and examples concerning the transience and recurrence of random diffusions. Annales de l'I.H.P. Probabilit\' e s et statistiques , 28(4):519--536, 1992

    1992

  33. [41]

    Spectral methods for L angevin dynamics and associated error estimates

    Julien Roussel and Gabriel Stoltz. Spectral methods for L angevin dynamics and associated error estimates. ESAIM: M2AN , 52(3):1051--1083, 2018

    2018

  34. [42]

    Criteria for transience and recurrence of regime-switching diffusion processes

    Jinghai Shao. Criteria for transience and recurrence of regime-switching diffusion processes. Electronic Journal of Probability , 20:1--15, 2015

    2015

  35. [43]

    Ergodicity of regime-switching diffusions in W asserstein distances

    Jinghai Shao. Ergodicity of regime-switching diffusions in W asserstein distances. Stochastic Processes and their Applications , 125(2):739--758, 2015

    2015

  36. [44]

    The randomized midpoint method for log-concave sampling

    Ruoqi Shen and Yin Tat Lee. The randomized midpoint method for log-concave sampling. In Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019

    2019

  37. [45]

    Leslie N. Smith. Cyclical learning rates for training neural networks. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) , pages 464--472, 2017

    2017

  38. [46]

    Exploring loss function topology with cyclical learning rates

    Leslie N Smith and Nicholay Topin. Exploring loss function topology with cyclical learning rates. arXiv preprint arXiv:1702.04283 , 2017

    2017 arXiv

  39. [47]

    Inverse problems: A B ayesian perspective

    Andrew M Stuart. Inverse problems: A B ayesian perspective. Acta Numerica , 19:451--559, 2010

    2010

  40. [48]

    Strong ergodicity of the regime-switching diffusions

    Jinghai Shao and Fubao Xi. Strong ergodicity of the regime-switching diffusions. Stochastic Processes and their Applications , 123(11):3903--3918, 2013

    2013

  41. [49]

    Stability and recurrence of regime-switching diffusion processes

    Jinghai Shao and Fubao Xi. Stability and recurrence of regime-switching diffusion processes. SIAM Journal on Control and Optimization , 52(6):3496–3516, 2014

    2014

  42. [50]

    Consistency and fluctuations for stochastic gradient L angevin dynamics

    Yee Whye Teh, Alexandre H Thiery, and Sebastian J Vollmer. Consistency and fluctuations for stochastic gradient L angevin dynamics. Journal of Machine Learning Research , 17(1):193--225, 2016

    2016

  43. [51]

    Hypocoercivity

    C\' e dric Villani. Hypocoercivity. Memoirs of the American Mathematical Society , 202(950):iv+141, 2009

    2009

  44. [52]

    Optimal Transport: Old and New

    C \'e dric Villani. Optimal Transport: Old and New . Springer, Berlin, 2009

    2009

  45. [53]

    An empirical study of cyclical learning rate on neural machine translation

    Weixuan Wang, Choon Meng Lee, Jianfeng Liu, Talha Colakoglu, and Wei Peng. An empirical study of cyclical learning rate on neural machine translation. Natural Language Engineering , 29(2):316--336, 2023

    2023

  46. [54]

    Global convergence of L angevin dynamics based algorithms for nonconvex optimization

    Pan Xu, Jinghui Chen, Difan Zou, and Quanquan Gu. Global convergence of L angevin dynamics based algorithms for nonconvex optimization. In Advances in Neural Information Processing Systems , volume 31, pages 3122--3133. Curran Associates, Inc., 2018

    2018

  47. [55]

    Hybrid Switching Diffusions: Properties and Applications , volume 63

    Gang George Yin and Chao Zhu. Hybrid Switching Diffusions: Properties and Applications , volume 63. Springer, 2010

    2010

  48. [56]

    Nonasymptotic estimates for S tochastic G radient L angevin D ynamics under local conditions in nonconvex optimization

    Ying Zhang, \" O mer Deniz Akyildiz, Theodoros Damoulas, and Sotirios Sabanis. Nonasymptotic estimates for S tochastic G radient L angevin D ynamics under local conditions in nonconvex optimization. Applied Mathematics & Optimization , 87:25, 2023

    2023

  49. [57]

    Cyclical stochasitc gradient MCMC for B ayesian deep learning

    Ruqi Zhang, Chunyuan Li, Jianyi Zhang, Changyou Chen, and Andrew Gordon Wilson. Cyclical stochasitc gradient MCMC for B ayesian deep learning. In International Conference on Learning Representations , 2020

    2020

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.