pith. sign in

arxiv: 2606.26881 · v1 · pith:DO5XMK3Dnew · submitted 2026-06-25 · 🧮 math.NA · cs.LG· cs.NA· stat.CO

Accelerated sampling using SamAdams variable timesteps and position-adaptive Langevin dynamics

Benedict Leimkuhler , Peter A. Whalley This is my paper

Pith reviewed 2026-06-26 04:16 UTC · model grok-4.3

classification 🧮 math.NA cs.LGcs.NAstat.CO
keywords adaptive timesteppingLangevin dynamicssampling methodsposition-adaptive frictionpalindromic integratorRosenbrock potentialMueller-Brown potential
0
0 comments X

The pith

SamAdams variable timesteps combined with position-adaptive Langevin dynamics accelerate sampling while preserving the canonical distribution.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces a sampling method that pairs variable timesteps shrinking automatically in stiff phase-space regions with friction directed only along the local force vector. The two devices are combined inside a palindromic integrator that needs just one force evaluation per step by exploiting the rank-one-plus-scalar form of the friction tensor. Tests on the Rosenbrock function, Mueller-Brown potential, thin entropic channels, and a Bayesian shrinkage-prior problem show faster mixing and large efficiency gains over fixed-step Langevin integration. A reader would care because these gains directly reduce the computational cost of exploring high-dimensional probability distributions that arise in statistics and molecular simulation.

Core claim

The SA-PAL scheme integrates SamAdams adaptive timestepping, which shrinks the effective step using a relaxed stiffness monitor, with position-adaptive Langevin dynamics that concentrates friction along the force direction while keeping the canonical distribution as the exact invariant measure; implemented via a palindromic integrator, the method improves mixing rates by 1.5-3 times on Rosenbrock and Mueller-Brown potentials and yields efficiency gains exceeding an order of magnitude on the remaining examples.

What carries the argument

SamAdams adaptive timestepping, which automatically shrinks the effective integration step in stiff regions using a relaxed stiffness monitor, together with position-adaptive Langevin dynamics, which concentrates friction along the local force direction while preserving the canonical distribution.

If this is right

  • The palindromic integrator requires only one force evaluation per iteration because of the rank-one-plus-scalar structure of the PAL friction tensor.
  • Mixing rates improve by factors between 1.5 and 3 relative to fixed-stepsize integration on the Rosenbrock and Mueller-Brown potentials.
  • Efficiency gains of more than an order of magnitude appear on the thin entropic channel and the Bayesian parameterisation problem with sparsity-inducing prior.
  • The canonical distribution remains exactly invariant under the combined dynamics.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same adaptive friction idea could be tested on molecular-dynamics systems whose stiffness varies strongly with conformation.
  • Variable-timestep schemes of this type might reduce the amount of manual step-size tuning required for reliable sampling.
  • Applying the method to larger Bayesian models with many parameters would test whether the efficiency gains persist at higher dimension.

Load-bearing premise

Position-adaptive Langevin dynamics concentrates friction along the local force direction while preserving the canonical distribution as the exact invariant measure.

What would settle it

A simulation in which the long-time distribution generated by the PAL dynamics deviates measurably from the canonical distribution on any of the tested potentials, or in which the combined SA-PAL integrator fails to improve mixing rates over fixed-stepsize integration on the Rosenbrock potential.

Figures

Figures reproduced from arXiv: 2606.26881 by Benedict Leimkuhler, Peter A. Whalley.

Figure 1
Figure 1. Figure 1: Rosenbrock friction–stepsize dilemma. (a, left) BAOAB stability boundary [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Thin channel at equal gradient budget (N = 2×105 ). Columns, left to right: BAOAB at γ = 0.5, γ = 2, γ = 10, and SA-PAL (α = 0.05, β = 5). Top: x1 vs gradient evaluations (wells at x1 = ±3, neck at x1 = 0). Bottom: trajectories on the channel potential contours. BAOAB barely crosses the neck at any friction, so its x1 autocorrelation time is not reliably estimable (see [PITH_FULL_IMAGE:figures/full_fig_p0… view at source ↗
Figure 3
Figure 3. Figure 3: Mueller–Brown (kT = 15). Top: trajectories on the three-well landscape (+ marks the minima) at equal gradient budget, for (left to right) BAOAB at γ = 1, γ = 10, γ = 40, force-monitor SA-PAL, and arc-length SA-PAL. Bottom: autocorrelation of x1, x2 and U (left to right) per gradient evaluation, each panel comparing all five methods (coloured as in the top row). The force-monitor SA-PAL row decorrelates eve… view at source ↗
Figure 4
Figure 4. Figure 4: Distribution of the SamAdams effective stepsize [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Reweighted, normalized autocorrelation functions for the position coordinates [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Per-chain estimates of ⟨x1⟩, ⟨x2⟩ and ⟨U⟩ (columns, left to right) from independent BAOAB, OBABO, SA-only and SA-PAL (F=force, A= arclength) chains at matched gradient budget, plotted against the exact target values from quadrature (red dashed lines). Rows: Rosenbrock, thin channel, Mueller–Brown. BAOAB, OBABO and SA-only are each shown at three frictions; box width is the run-to-run sampling spread, box p… view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of BAOAB, SA-only, and SA-PAL on the horseshoe regression problem [PITH_FULL_IMAGE:figures/full_fig_p019_7.png] view at source ↗
read the original abstract

We introduce an accelerated Langevin-based sampling method that is based on two complementary devices: \emph{SamAdams} adaptive timestepping, which automatically shrinks the effective integration step in stiff regions of phase space using a relaxed stiffness monitor, and \emph{position-adaptive Langevin} (PAL) dynamics, which concentrates friction along the local force direction while preserving the canonical distribution as the exact invariant measure. The resulting combined scheme (SA-PAL) is implemented in a palindromic integrator which requires only one force evaluation per iteration through suitable organisation of the integration steps and by exploiting the rank-one-plus-scalar structure of the PAL friction tensor. We test the method on various model problems: the Rosenbrock function, a thin entropic channel, the Mueller-Brown potential, and a Bayesian parameterisation problem with a sparsity-inducing shrinkage prior. On the Rosenbrock and Mueller-Brown potentials mixing rates are improved by 1.5-3 times compared to fixed stepsize integration. Efficiency gains of more than an order of magnitude are documented in the other examples.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces SamAdams adaptive timestepping combined with position-adaptive Langevin (PAL) dynamics for accelerated sampling. PAL concentrates friction along the local force direction while preserving the canonical distribution exactly as the invariant measure; the combined SA-PAL scheme is realized via a palindromic integrator exploiting the rank-one-plus-scalar friction structure (one force evaluation per step) and is tested on the Rosenbrock function, a thin entropic channel, the Mueller-Brown potential, and a Bayesian shrinkage-prior problem, reporting 1.5–3 imes mixing-rate improvements on the first two and efficiency gains exceeding an order of magnitude on the others.

Significance. If the exact invariance claim holds and the reported speed-ups are reproducible, the method would constitute a practically useful advance in adaptive Langevin sampling for stiff or anisotropic potentials, with direct relevance to molecular dynamics and Bayesian computation. The palindromic splitting and exploitation of the friction tensor’s low-rank structure are computationally attractive features.

major comments (2)
  1. [Abstract and PAL-dynamics construction] The assertion that PAL dynamics exactly preserves the canonical measure (abstract and the PAL-dynamics section) is load-bearing for every performance claim. The manuscript must supply the explicit Itô–Stratonovich conversion, the verification that the rank-one-plus-scalar friction tensor plus palindromic splitting introduces no spurious drift, and a statement of the precise conditions under which the stationary density remains exp(−V). Without this derivation the numerical speed-ups compare samplers whose invariant measures may differ.
  2. [Numerical experiments] § on numerical experiments: the 1.5–3 imes mixing-rate gains on Rosenbrock and Mueller-Brown and the >10 imes efficiency gains elsewhere are stated relative to fixed-stepsize integration, yet no table or figure reports the effective sample size, integrated autocorrelation time, or statistical uncertainty on these ratios. In addition, it is unclear whether the fixed-stepsize baseline employs the identical PAL friction or the standard isotropic Langevin dynamics.
minor comments (2)
  1. [SamAdams adaptive timestepping] Clarify the precise definition of the “relaxed stiffness monitor” used by SamAdams timestepping and state whether it introduces any additional bias when the timestep becomes position-dependent.
  2. All model potentials (Rosenbrock, Mueller-Brown, entropic channel) should be written explicitly with their functional forms and parameters in the main text rather than only in figure captions.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful and constructive report. The two major comments identify important points that we will address directly in revision. We provide point-by-point responses below.

read point-by-point responses

  1. Referee: [Abstract and PAL-dynamics construction] The assertion that PAL dynamics exactly preserves the canonical measure (abstract and the PAL-dynamics section) is load-bearing for every performance claim. The manuscript must supply the explicit Itô–Stratonovich conversion, the verification that the rank-one-plus-scalar friction tensor plus palindromic splitting introduces no spurious drift, and a statement of the precise conditions under which the stationary density remains exp(−V). Without this derivation the numerical speed-ups compare samplers whose invariant measures may differ.

    Authors: We agree that an explicit derivation is required to substantiate the invariance claim. In the revised manuscript we will add, in the PAL-dynamics section, the full Itô–Stratonovich conversion for the position-dependent friction, a direct verification that the rank-one-plus-scalar structure together with the palindromic integrator produces no additional drift terms, and a precise statement of the conditions (smoothness of V and the friction coefficients) under which the unique invariant measure is exactly exp(−V). revision: yes

  2. Referee: [Numerical experiments] § on numerical experiments: the 1.5–3 times mixing-rate gains on Rosenbrock and Mueller-Brown and the >10 times efficiency gains elsewhere are stated relative to fixed-stepsize integration, yet no table or figure reports the effective sample size, integrated autocorrelation time, or statistical uncertainty on these ratios. In addition, it is unclear whether the fixed-stepsize baseline employs the identical PAL friction or the standard isotropic Langevin dynamics.

    Authors: We will revise the numerical-experiments section to state explicitly that all fixed-stepsize comparisons use the identical PAL friction tensor (not isotropic Langevin). We will also add a table (or extended figure caption) that reports effective sample sizes, integrated autocorrelation times, and bootstrap or batch-means estimates of uncertainty on the reported speed-up ratios for each test problem. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation self-contained

full rationale

The abstract presents PAL dynamics as constructed to concentrate friction along the local force direction while preserving the canonical distribution exactly as invariant measure, with the combined SA-PAL scheme implemented via palindromic integrator exploiting the friction tensor structure. Performance claims (1.5-3x mixing improvement on Rosenbrock/Mueller-Brown, >10x efficiency elsewhere) are supported by direct empirical tests on specified model problems rather than by any reduction to fitted parameters or self-citations. No load-bearing step in the given text reduces by construction to its inputs, self-definition, or author-overlapping citations; the invariance property is stated as following from the dynamics design and is externally falsifiable via the reported sampling experiments. This is the normal case of a self-contained numerical methods paper.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available; no explicit free parameters, axioms, or invented entities are described in sufficient detail to populate the ledger.

pith-pipeline@v0.9.1-grok · 5724 in / 1048 out tokens · 50540 ms · 2026-06-26T04:16:14.330980+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 21 canonical work pages

  1. [1]

    Bou-Rabee and J

    N. Bou-Rabee and J. M. Sanz-Serna. Geometric integrators and the Hamiltonian Monte Carlo method.Acta Numerica, 27:113–206, 2018. doi:10.1017/S0962492917000101

    work page doi:10.1017/s0962492917000101 2018

  2. [2]

    Y. Cao, J. Lu, and L. Wang. On explicitL 2-convergence rate estimate for under- damped Langevin dynamics.Archive for Rational Mechanics and Analysis, 247(5):90, 2023. doi:10.1007/s00205-023-01922-4. 22

    work page doi:10.1007/s00205-023-01922-4 2023

  3. [3]

    C. M. Carvalho, N. G. Polson, and J. G. Scott. The horseshoe estimator for sparse signals. Biometrika, 97(2):465–480, 2010. doi:10.1093/biomet/asq017

    work page doi:10.1093/biomet/asq017 2010

  4. [4]

    M. Chak, N. Kantas, T. Lelièvre, and G. A. Pavliotis. Optimal friction matrix for under- damped Langevin sampling.ESAIM Math. Model. Numer. Anal., 57(6):3335–3371, 2023. doi:10.1051/m2an/2023083

    work page doi:10.1051/m2an/2023083 2023

  5. [5]

    J. M. Flegal and G. L. Jones. Batch means and spectral variance estimators in Markov chain Monte Carlo.Annals of Statistics, 38(2):1034–1070, 2010. doi:10.1214/09-AOS735

    work page doi:10.1214/09-aos735 2010

  6. [6]

    Girolami and B

    M. Girolami and B. Calderhead. Riemann manifold Langevin and Hamiltonian Monte Carlo methods.J. R. Stat. Soc. B, 73(2):123–214, 2011. doi:10.1111/j.1467-9868.2010.00765.x

    work page doi:10.1111/j.1467-9868.2010.00765.x 2011

  7. [7]

    Huang and B

    W. Huang and B. Leimkuhler. The adaptive Verlet method.SIAM Journal on Scientific Computing, 18(1):239–256, 1997. doi:10.1137/S1064827595284658

    work page doi:10.1137/s1064827595284658 1997

  8. [8]

    Jia and B

    Z. Jia and B. Leimkuhler. A projective thermostatting dynamics technique.Multiscale Model. Simul., 4(2):563–583, 2005. doi:10.1137/040603863

    work page doi:10.1137/040603863 2005

  9. [9]

    Jones and B

    A. Jones and B. Leimkuhler. Adaptive stochastic methods for sampling driven molecular systems.Journal of Chemical Physics, 135(8):084125, 2011. doi:10.1063/1.3626941

    work page doi:10.1063/1.3626941 2011

  10. [10]

    Karoni, B

    A. Karoni, B. Leimkuhler, and G. Stoltz. Friction-adaptive descent: A family of dynamics- based optimization methods.Journal of Computational Dynamics, 10(4):450–484, 2023. doi:10.3934/jcd.2023007

    work page doi:10.3934/jcd.2023007 2023

  11. [11]

    Karoni, R

    A. Karoni, R. Rajpal, B. Leimkuhler, and G. Stoltz. Adaptive momentum and nonlinear damping for neural network training.arXiv preprint arXiv:2602.00334, 2026

    arXiv 2026

  12. [12]

    Leimkuhler and C

    B. Leimkuhler and C. Matthews. Rational construction of stochastic numerical methods for molecular sampling.Appl. Math. Res. eXpress, 2013(1):34–56, 2013. doi:10.1093/amrx/abs010

    work page doi:10.1093/amrx/abs010 2013

  13. [13]

    Leimkuhler, D

    B. Leimkuhler, D. Paulin, and P. A. Whalley. Contraction and convergence rates for dis- cretizedkineticLangevindynamics.SIAM Journal on Numerical Analysis, 62(3):1226–1258,

  14. [14]

    doi:10.1137/23M1556289

    work page doi:10.1137/23m1556289

  15. [15]

    Leimkuhler, R

    B. Leimkuhler, R. Lohmann, and P. A. Whalley. A Langevin sampling algorithm inspired by the Adam optimizer.ACM Trans. Probab. Mach. Learn., 1, 2026. doi:10.1145/3806203

    work page doi:10.1145/3806203 2026

  16. [16]

    Lim and M

    K. Lim and M. Tao. Appropriate state-dependent friction coefficient accelerates kinetic Langevin dynamics.SIAM J. Appl. Math., 85(1):1–26, 2025. doi:10.1137/23M1625810

    work page doi:10.1137/23m1625810 2025

  17. [17]

    J. Lu. A sharp hypocoercive entropy decay estimate for underdamped Langevin dynamics. arXiv preprint arXiv:2605.01933, 2026

    Pith/arXiv arXiv 2026

  18. [18]

    Matthews, J

    C. Matthews, J. Weare, and B. Leimkuhler. Ensemble preconditioning for Markov chain Monte Carlo simulation.Stat. Comput., 28(2):277–290, 2018. doi:10.1007/s11222-017-9730- 1

    work page doi:10.1007/s11222-017-9730- 2018

  19. [19]

    Müller and L

    K. Müller and L. D. Brown. Location of saddle points and minimum energy paths by a constrained simplex optimization procedure.Theoret. Chim. Acta, 53(1):75–93, 1979. doi:10.1007/BF00547608

    work page doi:10.1007/bf00547608 1979

  20. [20]

    G. A. Pavliotis.Stochastic Processes and Applications: Diffusion Processes, the Fokker- Planck and Langevin Equations, volume 60 ofTexts in Applied Mathematics. Springer, New York, 2014. doi:10.1007/978-1-4939-1323-7. 23

    work page doi:10.1007/978-1-4939-1323-7 2014

  21. [21]

    V. Roy, A. Tan, and J. M. Flegal. Estimating standard errors for importance sam- pling estimators with multiple Markov chains.Statistica Sinica, 28(2):1079–1101, 2018. doi:10.5705/ss.202016.0003

    work page doi:10.5705/ss.202016.0003 2018

  22. [22]

    Sachs, B

    M. Sachs, B. Leimkuhler, and V. Danos. Langevin dynamics with variable coefficients and nonconservative forces: From stationary states to numerical methods.Entropy, 19(12):647,

  23. [23]

    doi:10.3390/e19120647

    work page doi:10.3390/e19120647

  24. [24]

    A. D. Sokal. Monte Carlo methods in statistical mechanics: Foundations and new al- gorithms. InFunctional Integration: Basics and Applications, pages 131–192. Springer, Boston, MA, 1997. doi:10.1007/978-1-4899-0319-8_6

    work page doi:10.1007/978-1-4899-0319-8_6 1997

  25. [25]

    K. F. Sundman. Memoir on the three-body problem.Acta Mathematica, 36(1):105–179, 1913

    1913

  26. [26]

    G. Vilmart. Weak second order multirevolution composition methods for highly oscillatory stochastic differential equations with additive or multiplicative noise.SIAM Journal on Scientific Computing, 36(4):A1770–A1796, 2014. doi:10.1137/130935331. 24

    work page doi:10.1137/130935331 2014