{"id":"a90d4a2c-1bf0-4759-a3f7-b5c57a3a6ab7","arxiv_id":"2607.14862","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"tSGHMC provably samples from strongly convex targets with superlinear, discontinuous stochastic gradients at a λ^{1/4} Wasserstein-2 rate.","lead":"A new 'tamed' variant of stochastic gradient Hamiltonian Monte Carlo (tSGHMC) keeps sampling stable when gradients grow faster than linearly, with a proven Wasserstein-2 convergence rate of λ^{1/4}. The result covers stochastic optimization problems with possibly discontinuous, superlinearly growing gradients, such as CVaR and newsvendor losses.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claimed λ^{1/4} rate is contradicted by Theorem 1's own bound: substituting γ=O(λ^{-1/4}) yields Ω(λ^{3/8}) terms.","rationale":"The reader's weakest assumption concerns Assumption 3 and the mismatch between the theorem's step-size restriction λ≤λmax,γ and the numerical experiments. That is a valid concern about empirical support, but it leaves the theoretical rate claim intact. The more load-bearing issue is that the paper's headline rate of 1/4 does not follow from its own Theorem 1. The bound in Eq. (17), with the authors' chosen γ=O(λ^{-1/4}), contains terms scaling as λ^{3/8}, which dominate the ε=λ^{1/4} term. This is not a matter of missing conditions or overly strong assumptions; it is an algebraic inconsistency in the central contribution. If the intended rate is λ^{1/4}, the theorem's error bound must be different (e.g., no λγ^{5/2} term). If the correct rate is λ^{3/8}, then it is worse than the λ^{1/5} rate of tKLMC, so the claimed improvement is reversed. The paper could potentially be revised by correcting the rate or by removing the improvement claim, but as written the main theoretical result is not supported. This warrants a REJECT verdict rather than a conditional acceptance.","tokens_in":62067,"tokens_out":9142,"duration_ms":69223,"concrete_test":"Perform a direct algebraic check: set γ = c λ^{-1/4} and ε = λ^{1/4} in Eq. (17) and in the intermediate bounds of Section 4.5 (Propositions 4 and 5). Compute the exponents of each term and verify the dominant scaling. If the dominant term is λ^{3/8} or λ^{1/2} rather than λ^{1/4}, the claimed rate is false. Also re-check Remark 7's assertion that √(λγ)=O(λ^{1/4}) for this γ; the correct exponent is 3/8.","verdict_should_be":"REJECT","load_bearing_attack":"Theorem 1 (Eq. 17) states W2(L(θλ_n), πβ) ≤ Ċ(√(λγ) + λγ^{5/2} + γ^{-1} + exp(...) + ε). In Remark 7 the authors choose ε=λ^{1/4} and γ=O(ε^{-1})=O(λ^{-1/4}). Substituting into Eq. (17): √(λγ) = Θ(λ^{3/8}), λγ^{5/2} = Θ(λ^{3/8}), and γ^{-1} = Θ(λ^{1/4}). Since λ^{3/8} dominates λ^{1/4} as λ→0, the right-hand side is Ω(λ^{3/8}), not O(λ^{1/4}). The proof's intermediate bounds in Propositions 4 and 5 contain terms λγ^2 and √(γS), which scale as λ^{1/2} and λ^{3/8} respectively for this γ; no cancellation can restore the advertised λ^{1/4}. Thus the central theoretical claim—a W2 rate of 1/4, improving on tKLMC's 1/5—is unsupported by the paper's own theorem. If the correct rate is 3/8 (or worse), it is slower than tKLMC's 1/5, reversing the claimed improvement. This is an internal mathematical inconsistency, independent of the empirical step-size issues the reader flagged.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes tSGHMC (Eqs. (9)-(10)), a tamed stochastic-gradient kinetic Langevin algorithm for sampling and stochastic optimization with superlinearly growing and possibly discontinuous stochastic gradients. Under Assumptions 2-3 (polynomial continuity in average and data-dependent strong convexity), Theorem 1 gives a Wasserstein-2 error bound with terms sqrt(lambda gamma)+lambda gamma^(5/2)+gamma^{-1}+exp(...)+epsilon, and Theorem 2 converts this into an expected-excess-risk bound. The paper's central advertised claim is that choosing epsilon=lambda^(1/4) and gamma=O(lambda^{-1/4}) yields a W2 rate of lambda^(1/4), improving on the lambda^(1/5) rate of tKLMC [33]. Numerical experiments on penalized logistic regression, a quartic artificial example, newsvendor, CVaR minimization, and ReLU regression compare tSGHMC with TUSLA.","tokens_in":1566,"tokens_out":1796,"duration_ms":77697,"significance":"If the advertised rate were correct, this would be a valuable contribution: it extends tamed kinetic Langevin sampling to stochastic gradients under relaxed smoothness/curvature assumptions, provides explicit constants and a careful Moreau-Yosida and auxiliary-process proof architecture, and gives a non-asymptotic excess-risk guarantee. The paper is unusually explicit about constants and technical conditions, which is commendable. However, the central quantitative claim is internally inconsistent: the paper's own parameter choice in Theorem 1 (Eq. (17)) yields an Omega(lambda^(3/8)) bound, not O(lambda^(1/4)). The numerical experiments also violate the theorem's step-size restriction (Eq. (16)), and the ReLU regression experiment does not verify Assumption 3. The proof framework may still contain useful tools for a corrected analysis, but as written the main claim of an improved rate is not supported.","major_comments":[{"comment":"Substituting gamma=O(lambda^{-1/4}) and epsilon=lambda^(1/4) into (17) gives sqrt(lambda gamma)=Theta(lambda^(3/8)), lambda gamma^(5/2)=Theta(lambda^(3/8)), and gamma^{-1}=Theta(lambda^(1/4)). The right-hand side of (17) is therefore Omega(lambda^(3/8)), not O(lambda^(1/4)); the displayed bound in Remark 7, W2 <= 5Cdot(lambda^(1/4)+exp(...)W2(...)), simply drops these leading terms. The sentence in Section 1.1 that 'lambda gamma=O(lambda^(3/4)) ... leading to ... sqrt(lambda gamma)=O(lambda^(1/4))' is internally inconsistent, since sqrt(lambda gamma)=O(lambda^(3/8)) if lambda gamma=O(lambda^(3/4)). At best, Theorem 1 yields a lambda^(3/8) rate for this tuning, which is slower than the lambda^(1/5) rate of [33], reversing the claimed improvement. No cancellation between Propositions 4 and 5 can restore the lambda^(1/4) rate, since for this gamma Proposition 4 contains terms Theta(lambda^(","section":"Eq. (17), Remark 7"},{"comment":"Every numerical experiment uses step sizes outside the theorem's validity domain lambda <= lambda_max,gamma. For example, in Section 3.1 (logistic regression, m=10^{-3}, gamma=1, r=0.5), q=12r=6 gives lambda_max,gamma = min(1/4, 3*10^{-3}/(16*6*5*2^4)) approximately 3.9*10^{-7}, while Figure 1 uses lambda up to 5*10^{-3}. In Section 3.3 lambda=0.05 with m=0.041, gamma=1, r=1.5 gives an even smaller lambda_max,gamma. Thus the RMSE plots and excess-risk trajectories cannot be cited as empirical validation of Theorem 1; they probe a regime in which the theorem is silent.","section":"Sections 3.1-3.4, Eq. (16)"},{"comment":"The ReLU nonlinear regression experiment sets m=10^{-10} but provides no argument that the objective (35) is m-strongly convex, or that E[A(X0)] has smallest eigenvalue at least m. For a one-hidden-layer ReLU network the objective is generically nonconvex; setting a tiny numerical value of m does not satisfy Assumption 3. Since m enters the taming coefficient (10), the step-size condition (16), and all moment/contraction estimates, Theorems 1 and 2 do not apply to this experiment as stated.","section":"Section 3.4.2, Assumption 3"}],"minor_comments":[{"comment":"Typo: 'vrious applications' should be 'various applications'.","section":"Section 3"},{"comment":"The notation '16C0e/m' is easy to misread; use exp(1) or a different symbol to distinguish the constant e from the smoothing parameter epsilon.","section":"Proposition 5 / Theorem 3 proof"},{"comment":"The step 'replace (n+1)T on both sides with n+1' is informal. Since lambda T >= 1/2, a short monotonicity/rounding argument would make the replacement rigorous.","section":"Proof of Theorem 1"}],"recommendation":"reject","confidential_remarks":"The main issue is a simple but load-bearing parametric error: the claimed lambda^(1/4) rate does not follow from the paper's own Theorem 1, and the actual implied rate is slower than the existing tKLMC rate it claims to improve. This is not a presentation issue. The systematic violation of lambda <= lambda_max,gamma in the numerics and the unverified strong convexity in the ReLU experiment further weaken the paper. I recommend rejection unless the authors can supply a genuinely lambda^(1/4) bound with a different parameter tuning, which would require a substantial revision of the proof."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, the tSGHMC algorithm is genuinely new: first tamed SGHMC variant, and the proof framework follows the established [33,38] template with careful moment estimates. Second, the paper's headline rate of 1/4 is not what Theorem 1 delivers. Plug gamma=O(lambda^-1/4), epsilon=lambda^1/4 into (17): the sqrt(lambda*gamma) term is Theta(lambda^3/8), as is lambda*gamma^(5/2). So the bound is Omega(lambda^3/8), not O(lambda^1/4). The stress-test note is right about that. But it then stumbles: lambda^3/8 is faster than tKLMC's lambda^1/5, not slower. So the central selling point—improvement over tKLMC—survives, but the stated rate is wrong and the proof needs to be rewritten to claim 3/8 (or whatever the actual best balance is). That is a real flaw, but not a fatal one. What the paper does well: the taming construction (10) is sensible, the assumptions (polynomial continuity in average, data-dependent strong convexity) are weaker than global Lipschitz, and the analysis of the auxiliary processes is serious. The excess-risk bound in Theorem 2 is a natural byproduct. The proofs are long but coherent. The soft spots: (i) every numerical experiment uses step sizes that violate the theorem's own condition lambda <= lambda_max,gamma. For logistic regression with m=10^-3, gamma=1, lambda_max is about 3.9e-7, but they run lambda up to 5e-3. That is four orders of magnitude outside the theory. The claim that numerics 'support' the theory is overreach. (ii) The ReLU network experiment uses m=10^-10 without verifying strong convexity; that example likely does not satisfy Assumption 3. (iii) The self-citation load is heavy but the cited works provide the right tools, not padding. Bottom line: this is a serious technical paper that deserves referee time, but it needs a corrected rate statement and a more careful separation between what is proved and what is simulated. If the rate is restated as 3/8, the paper becomes a solid contribution. I'd bring it to a reading group to sort out the rate issue.","headline":"New tamed SGHMC algorithm with real promise, but the advertised 1/4 Wasserstein rate does not follow from the paper's own theorem—the bound actually gives 3/8, which is still better than the 1/5 it improves on.","tokens_in":676,"tokens_out":865,"would_cite":true,"duration_ms":39831,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["60J22","65C05","90C15"],"pacs":[],"model":"deepseek-v4-flash","headline":"A tamed stochastic-gradient Hamiltonian Monte Carlo algorithm converges in Wasserstein-2 distance at rate λ^{1/4} for superlinearly growing, possibly discontinuous stochastic gradients, improving on the λ^{1/5} rate of its full-gradient tam","keywords":["tamed stochastic gradient Hamiltonian Monte Carlo","superlinear gradients","discontinuous stochastic gradients","Wasserstein-2 distance","non-asymptotic convergence","expected excess risk","data-dependent strong convexity"],"falsifier":"Run tSGHMC on a quartic potential u(θ)=|θ|^4 without a quadratic term, so E[A(X0)] has zero eigenvalue (m=0), and check whether W2(L(θλ_n),πβ) still decays; if it diverges or plateaus, Assumption 3 is essential. A milder check: in the newsvendor experiment (m=0.041, γ=1, r=1.5), λmax,γ is several orders of magnitude below the λ=0.05 used; test whether the log-log RMSE slope changes from the claimed λ^{1/4} once λ exceeds that threshold.","tokens_in":61930,"feed_emoji":"🎲","tokens_out":6460,"duration_ms":51843,"temperature":0.7,"pith_summary":"This paper introduces tSGHMC, a tamed, stochastic-gradient version of Hamiltonian Monte Carlo aimed at sampling and optimization problems where gradients grow faster than linearly and may be discontinuous. Under a polynomial continuity-in-average condition and a data-dependent strong convexity condition, it proves a non-asymptotic Wasserstein-2 bound whose dominant error term is λ^{1/4}, improving the λ^{1/5} rate previously established for tamed kinetic Langevin Monte Carlo with exact gradients. The taming coefficient keeps iterates bounded without adding a full-gradient computation, so the method retains the low per-iteration cost of stochastic gradients. The same bound yields an expected-excess-risk estimate, giving an end-to-end guarantee for using tSGHMC samples to solve stochastic optimization problems. Experiments on posterior sampling, newsvendor, CVaR, and ReLU regression tasks show it outperforms the first-order tamed stochastic Langevin algorithm in most settings.","feed_headline":"Tamed stochastic-gradient HMC converges at λ^1/4 rate","feed_subtitle":"Handling superlinear, discontinuous gradients with cheap stochastic updates; the rate beats full-gradient tamed samplers.","key_machinery":"The argument rests on the tamed stochastic gradient Hγ(θ,x) = mθ + (H(θ,x)−mθ)/(√(1+γ^{-1}|θ|^{4r})). This one coefficient simultaneously enforces a dissipativity condition, brings linear growth in θ, and keeps the deterministic drift close to the raw stochastic gradient H, which prevents the finite-time moment explosion that breaks plain Euler schemes. Around it the paper builds a chain: Moreau–Yosida regularization of u provides a globally Lipschitz, strongly convex surrogate; auxiliary continuous-time Langevin processes and a Lyapunov function control second and higher moments; and a contraction theorem for the regularized underdamped SDE supplies the exponential memory term in the bound.","core_discovery":"The central claim is Theorem 1: for every ε>0 and β>0, as long as the step size λ is below a threshold λmax,γ and the friction γ is at least a problem-dependent γmin, the Wasserstein-2 distance between the law of tSGHMC's position θλ_n and the target πβ is bounded by Ċ(√(λγ)+λγ^{5/2}+γ^{-1}+exp(−λm/(2γ)n)W2(L(θ0,ν0),Πβ)+ε). When ε is set to λ^{1/4} and γ is chosen of order ε^{-1}, the bound becomes order λ^{1/4}, a faster rate than the λ^{1/5} obtained for tKLMC in [33]; equivalently, achieving ε precision requires Õ(ε^{-5}) iterations. This is the first such non-asymptotic guarantee for a stochastic-gradient kinetic Langevin sampler under superlinear and discontinuous gradients.","pith_inferences":["The rate gain comes specifically from scaling γ with λ (γ=O(λ^{-1/4})); an editor-level reading suggests the same parameter trade-off could be transplanted to other tamed kinetic samplers, including nonconvex variants, whenever a contraction estimate is available.","The dependency of the taming coefficient on the strong-convexity constant m means the proof does not currently cover the ReLU-network experiment, where m=10^{-10} is set arbitrarily; an adaptive or estimated m would be needed to bring that experiment under the theorem.","A testable prediction: in the regime λ ≪ λmax,γ the empirical Wasserstein-2 error should scale as λ^{1/4} on log-log plots; if instead the slope matches λ^{1/5} for small λ, the claimed improvement would not materialize in practice.","The bound is dimension- and temperature-dependent through Ċ=O((d/β)^{3r+1/2}), so the method is likely most attractive in moderate dimension or when β is large enough for concentration; extending the analysis to a log-Sobolev setting could decouple the rate from dimension."],"forward_implications":["With ε=λ^{1/4} and γ=O(ε^{-1}), Theorem 1 gives an Õ(ε^{-5}) iteration complexity to reach ε accuracy in Wasserstein-2 distance, improving on the Õ(ε^{-6}) complexity listed for tKLMC.","Theorem 2 converts the same bound into an expected excess risk guarantee E[u(θλ_n)]−u(θ*) ≤ ... + O(d/(mβ)); the β^{-1} term can be driven down by raising inverse temperature.","Stochastic-gradient usage means per-iteration cost is independent of dataset size, unlike tKLMC that uses exact gradients each step.","The assumptions cover discontinuous stochastic gradients (e.g., indicators in newsvendor and CVaR) and online data, so the algorithm is applicable to nonsmooth risk minimization.","Numerically, tSGHMC attains smaller RMSE and smaller expected excess risk than the first-order tamed stochastic Langevin algorithm across the tested problems, with comparable performance in the remaining cases."],"fun_headline_variants":["tSGHMC beats tKLMC: λ^{1/4} convergence rate","Stochastic-gradient HMC tamed for superlinear gradients","First non-asymptotic bound for stochastic kinetic Langevin","tSGHMC: faster convergence under discontinuous gradients","Tamed HMC handles superlinear, discontinuous gradients"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The proof collapses if Assumption 3 fails: the objective must be strongly convex with a known positive lower bound m on the smallest eigenvalue of E[A(X0)], because the taming coefficient uses that exact m; if m is unknown, overestimated, or the objective has a flat direction, the dissipativity, contraction, and moment bounds all cease to hold.","fun_headline_variants_meta":{"raw":{"variants":["tSGHMC beats tKLMC: λ^{1/4} convergence rate","Stochastic-gradient HMC tamed for superlinear gradients","First non-asymptotic bound for stochastic kinetic Langevin","tSGHMC: faster convergence under discontinuous gradients","Tamed HMC handles superlinear, discontinuous gradients"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000167,"raw_usage":{"total_tokens":1106,"prompt_tokens":771,"completion_tokens":335,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":515,"completion_tokens_details":{"reasoning_tokens":260}},"tokens_in":515,"tokens_out":335,"duration_ms":3047,"temperature":1.0,"reasoning_tokens":260,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T00:50:49.632108+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run tSGHMC on a quartic potential u(θ)=|θ|^4 without a quadratic term, so E[A(X0)] has zero eigenvalue (m=0), and check whether W2(L(θλ_n),πβ) still decays; if it diverges or plateaus, Assumption 3 is essential. A milder check: in the newsvendor experiment (m=0.041, γ=1, r=1.5), λmax,γ is several orders of magnitude below the λ=0.05 used; test whether the log-log RMSE slope changes from the claimed λ^{1/4} once λ exceeds that threshold.","supporting_citations":[],"review_version":1}