Pith. sign in

REVIEW 3 major objections 5 minor 31 references

Path Integral Optimiser: Global Optimisation via Neural Schr\"odinger-F\"ollmer Diffusion

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a neural Schrödinger-Föllmer diffusion, trained to sample a loss's Boltzmann density, is a global optimiser with a three-term failure-probability bound that can be driven to zero.

desk verdict The method is a clever repackaging of PIS as an optimizer, but Theorem T4.6 is vacuous as stated: AA.1 and AA.2 cannot both hold for any sigma in (0,1). read the letter →

arxiv 2506.06815 v1 pith:6ZJJCGUC submitted 2025-06-07 cs.LG

Max McGuinness , Eirik Fladmark , Francisco Vargas This is my paper · ORCID
classification cs.LG MSC 60H1068T0765K10
keywords globaloptimisationSchrödinger-FöllmerprocesspathintegralsamplerBoltzmanndistributionneuraldriftapproximationdiffusionmodelstochasticcontrolannealingschedule
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces the Path Integral Optimiser (PIO), which reformulates global optimisation as a sampling problem: the loss function $V$ is converted into a Boltzmann density $\exp(-V/\sigma)$, and a neural Schrödinger-Föllmer diffusion is trained to produce samples from that density. The central theoretical claim is a bound on the probability that the optimiser's terminal point has loss above a threshold $\tau$, which splits into a temperature term, a neural-approximation term, and a time-discretisation term. If the bound is valid, then as the temperature, the drift-network error, and the step size all go to zero, the optimiser finds a global minimiser with probability approaching one. The paper reports that on benchmarks up to about 1.5 thousand parameters PIO matches or exceeds classical optimisers, while on a 15.9-thousand-parameter MNIST task it fails to escape an early local minimum.

What carries the argument

The central object is the Path Integral Optimiser, an EM-discretised neural Schrödinger-Föllmer process whose drift $b_\theta(x,t)$ is a time-conditioned Fourier MLP trained with the Path Integral Sampler loss $L(\theta) = \frac12 \int_0^1 \|b_\theta(X_t,t)\|^2 dt + \log P_{W_1}(X_1)/P(X_1)$. The argument is carried by the Boltzmann transform $P(\phi) = \exp(-V(\phi)/\sigma)$, which turns minimisation into sampling; scale invariance of the drift's $\nabla f/f$ ratio removes the normalising constant; and the PIS training objective turns the stochastic-control solution of the Schrödinger bridge into an ordinary neural-network optimisation problem. The theorem then decomposes the failure probability into three independently controllable errors: the Boltzmann-temperature gap, the drift-network approximation error, and the EM discretisation error.

What would settle it

Train PIO on a loss whose Boltzmann density has a known closed form, then measure the KL divergence or mean squared drift error between the learned network and the true Schrödinger-Föllmer drift as a function of network size and training steps; if the measured error stays bounded away from the assumed $\hat{\epsilon}$ even as $\sigma$ and $1/T$ shrink, the bound's prediction that failure probability goes to zero would not be observed on that problem.

Watch

Extended reading notes

Core claim

The paper's central claim is Theorem T4.6: under assumptions AA.1-AA.4, an Euler-Maruyama discretised neural Schrödinger-Föllmer process with $L'$-Lipschitz drift satisfies $$P_{\widehat{Y}_1}\bigl(V(X_1)>\tau\bigr) \le C_{\tau,\epsilon,n}\,\exp\!\bigl(-(\tau-\epsilon)/\$\sigma$\bigr) + \sqrt{2\hat{\epsilon}} + \sqrt{4L'^2\bigl(C_{L'}/T + 1/$T^{2}$\bigr)}.$$ The first term is inherited from the analytic Schrödinger-Föllmer optimisation bound, the second from a neural-network approximation of the density ratio, and the third from EM discretisation; the proof chains these with total-variation and Pinsker inequalities. The paper interprets the claim as saying that PIO is a $\tau$-global minimiser with probability at least $1 -$ the bound, and derives from it the parameter schedules $\sigma = O((\tau-\epsilon)/\log(1/\delta))$, $\hat{\epsilon} = O(\delta)$, $T = O(L'/\sqrt{\delta})$ for failure probability $\sqrt{\delta}$. On the empirical side, it reports that PIO matches or beats Adam, Adagrad, and SGD on Carrillo, Moons, and several classification tasks up to 1,247 parameters, but collapses on the 15.9k-parameter MNIST problem.

Load-bearing premise

The theorem assumes the trained drift network actually attains the small approximation error $\hat{\epsilon}$ promised by the neural-approximation assumptions, but the paper never measures that error; if Adam stops far above $\hat{\epsilon}$, the middle term of the bound cannot be driven toward zero.

Editorial extensions

If this is right

  • If Theorem T4.6 holds, global optimisation is achievable by a diffusion sampler with an explicit failure-probability certificate, not just a heuristic.
  • Corollary T4.7 gives concrete schedules: lower the Boltzmann temperature logarithmically with the desired confidence, keep the neural drift error linear in the confidence, and take enough discretisation steps.
  • A single trained drift yields an ensemble of candidate solutions for free, since every new stochastic trajectory is a new parameterisation.
  • The same bound applies to denoising-diffusion or Föllmer-like optimisers whose drift admits the same semigroup expressiveness, so the guarantee is not tied to this specific architecture.
  • The empirical failure on 15.9k-parameter MNIST shows that the current implementation does not yet realise the bound at scale; the paper calls for larger drift networks, ensembling, pre-training, and better parallelisation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment the paper does not run is to measure the achieved drift-approximation error $\hat{\epsilon}$ on synthetic losses with known Boltzmann densities; that number decides whether the middle term of the bound is credible in practice.
  • The logarithmic $\sigma$ schedule makes PIO a close cousin of simulated annealing, so a head-to-head with annealed Langevin dynamics on the same benchmarks would isolate what the Schrödinger-Föllmer drift adds.
  • If the bound carries to large problems, PIO-style warm-starting could seed SGD or Adam with diverse high-quality points, since the diffusion produces multiple distinct parameterisations rather than one trajectory.
  • The MNIST collapse hints that the bottleneck is drift expressiveness or training, not the optimisation formulation; testing PIO with a larger Fourier MLP or with pre-trained initialisation on the same 15.9k task would separate those two explanations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes PIO, a diffusion-based global optimizer that trains a neural Schrödinger–Föllmer drift to sample from the Boltzmann distribution of a loss function V. The theoretical core is Theorem T4.6, which bounds the probability that the terminal point of an EM-discretized neural SFP exceeds a threshold τ, with the bound composed of three terms: a Boltzmann concentration term exp(−(τ−ε)/σ), a neural approximation term √(2ε̂), and an EM-discretization term O(L′/√T). The paper also reports experiments on Carrillo, Moons, MNIST, and several UCI tasks, concluding that PIO matches or exceeds classical optimizers up to about 1,500 parameters and struggles on a 15.9k-parameter MNIST problem.

Significance. If Theorem T4.6 were valid, it would be a notable formal statement: a diffusion-based optimizer with explicit exponential concentration in σ, polynomial-size neural approximators, and explicit discretization error. The paper is transparent in deriving the chain of inequalities, states its assumptions explicitly, and releases code. However, the central theorem is vacuous as stated because its assumptions are mutually inconsistent, and the remaining proof chain relies on approximation guarantees that are not connected to the trained network. The empirical contribution is honest about the MNIST failure but is too preliminary and under-powered to establish a practical advantage. As a result, the formal significance claimed in the paper is not established.

major comments (3)
  1. [§A.2, Assumptions AA.1–AA.2; Definitions T2.2 and T2.5; Theorem T4.6] Assumptions AA.1 and AA.2 are incompatible for every σ∈(0,1). Assumption AA.1 states that V(x)=‖x‖²/2 outside a ball B_R. With the Boltzmann target of Definition T2.5, P(x)∝exp(−V(x)/σ), the density ratio f=dP/dN(0,I_n) in Definition T2.2 satisfies, for ‖x‖≥R, f(x) ∝ exp(−V(x)/σ+‖x‖²/2)=exp((σ−1)/(2σ)‖x‖²). For every σ∈(0,1) this decays to 0 as ‖x‖→∞. Assumption AA.2 requires a constant c∈(0,1] with f≥c everywhere, which is impossible when inf f=0. Hence no loss V satisfying AA.1 can satisfy AA.2 for any σ in the range allowed by Theorem T4.6. Since Remarks 4.0.1 and Corollary T4.7 explicitly rely on the limit σ→0 to concentrate the Boltzmann target on the global minimizer, the theorem's hypothesis set is empty in exactly the regime needed for the claimed global-optimization guarantee.
  2. [Theorem T4.6; Remarks 4.0.1] The simultaneous limit in Remarks 4.0.1 is not justified because the Lipschitz constant L′ of the drift network may depend on the approximation error ε̂. The theorem states that for each ε̂ there exist constants C_{τ,ε,n} and C_{L′}, with the bound containing √(4L′²(C_{L′}/T+1/T²)). As ε̂→0, one must in general choose a different, possibly more complex network, and its Lipschitz constant L′ can grow without bound. The remark treats L′ and C_{L′} as fixed while sending ε̂→0, but the theorem supplies no bound on L′ in terms of ε̂. Consequently the conclusion that the three error terms tend to 0 simultaneously is not established; the third term and the second term are coupled through the choice of approximating network.
  3. [Definition T2.4; Appendix A.2, Assumptions AA.3–AA.4; Section 5, Table 1] The theorem's √(2ε̂) term is conditional on the existence of a feedforward network approximating f and ∇f to sup-norm accuracy ε̂ (AA.3–AA.4). The implemented PIO trains its drift network by Adam on the Path Integral Sampler loss L(θ) of Definition T2.4. The paper does not show that minimizing L(θ) drives the sup-norm approximation error required by AA.3–AA.4 below ε̂, nor does it measure the achieved ε̂ on any benchmark. The statement in Remarks 4.0.1 that 'there exists a neural SFP drift for every ε̂>0' is an existence claim and does not imply that the Adam-trained network attains that error. Thus the √(2ε̂) term is not connected to the algorithm actually evaluated in Section 5; the MNIST results in Table 1 (PIO losses 2.46 and 2.42 vs. Adam losses 0.48 and 0.35) are consistent with this gap.
minor comments (5)
  1. [Theorem T4.6; Corollary T4.7; Remarks 4.0.1] The theorem bounds P(V(X₁)>τ), while Corollary T4.7 defines a τ-global minimizer by the event V(Y₁)≤τ+inf V(x). Unless one explicitly assumes inf V=0 (or shifts V, which is allowed because constants cancel in the drift), the event {V(X₁)>τ} is not the complement of τ-global optimality when inf V<0. A one-sentence normalization assumption would fix this.
  2. [§C.1] The text contains an unresolved cross-reference 'Figure??' in the discussion of the Narrow×32 sweep; this should be fixed before publication.
  3. [§2.5] The sentence 'Sampling-based optimization methods that don't require loss gradient information (PIO uses dL/dθ)' is confusing: PIO uses the gradient of the PIS loss L(θ) with respect to the network parameters θ, not the gradient of V, and the parenthetical appears to contradict the main clause. Please rephrase.
  4. [§5, Table 2] Table 2 reports test accuracy without standard deviations or seed counts, unlike Table 1; single-run entries such as Covertype PIO-Grad 0.813 vs. SGLD 0.656 are difficult to interpret without repeated runs or error bars.
  5. [§6] The conclusion that PIO 'matches or exceeds classical optimizers in tasks up to 1.5k parameters' is stronger than the evidence: on MNIST PIO is substantially worse, and the Carrillo and Moons advantages are not consistent across both sweep settings (Table 1) or are based on single runs (Table 2).

Circularity Check

0 steps flagged · score 2.0 of 10

No circularity by construction: Theorem T4.6 is an analytic bound over control parameters; one self-cited EM-discretization lemma is a lineage concern, while the main defect is an inconsistent assumption set, which is a correctness risk rather than circularity.

full rationale

The central bound (Theorem T4.6) is not a fitted prediction: sigma, epsilon-hat, and T enter as control parameters whose scaling is constrained by Corollary T4.7, and the three error terms come from Dai et al. (analytic bound), Tzen et al. (neural approximation), and Vargas et al. (EM discretization). No quantity is fitted to the benchmark losses and then renamed a prediction, so no fitted-input-called-prediction circularity is present. The only self-citation is the EM-discretization bound [27], where co-author F. Vargas is an author; it supports one of three error terms but is not derived from this paper's conclusion, and the other two terms are independently sourced, so it is a self-citation lineage/verification concern rather than a constructional circularity. The neural error epsilon-hat is assumed (AA.3-AA.4) and never measured in training, which is a missing empirical check, not a circular step. Separately, a serious consistency defect exists: under AA.1, the Boltzmann ratio f(x) = Z^{-1} exp(-V(x)/sigma + ||x||^2/2) equals Z^{-1} exp(-(1-sigma)/(2sigma)||x||^2) for ||x|| >= R, which tends to 0 for every sigma in (0,1), contradicting AA.2's requirement that f >= c > 0 everywhere. This makes Theorem T4.6 vacuous in the sigma<1 regime needed for optimization; this is an internal-consistency/correctness risk, not circularity-by-construction. Overall, the derivation chain does not reduce to its own inputs, so the circularity score is low.

Assumptions & free parameters 4 free parameters · 7 assumptions · 0 invented entities

The central bound of Theorem T4.6 is a chaining of three published inequalities (Dai et al. 3.5, Tzen et al. 3.1, Vargas et al. A6) via Pinsker's inequality; the paper contributes the packaging, the corollary's scaling conditions, and the empirical study. Everything quantitative is rented from prior work, and the key premise, that the Adam-trained PIO drift attains the Tzen approximation error, is unstated and unverified. No new entities are postulated.

free parameters (4)
  • σ (Boltzmann temperature) = tuned via BayesOpt, annealed by 50% on plateaus
    Controls the concentration of the target Boltzmann distribution. The theory requires σ→0 (Corollary T4.7), but in experiments σ is a tuned hyperparameter and the annealing schedule is a heuristic (Section 3).
  • bε (neural approximation error) = not measured
    The theorem's second term is √(2 bε); the paper assumes bε is small via AA.3-AA.4 but never measures the trained network's approximation error, so the bound cannot be instantiated in practice.
  • T (number of EM discretization steps) = 300 steps in Table 2; implicit elsewhere
    The theorem's third term shrinks as 1/T; experiments fix T=300 and no ablation on T is reported.
  • L' (Lipschitz constant of the drift net) = uncontrolled
    The discretization error term scales with L'^2, but no attempt is made to control or measure L' of the Fourier MLP, leaving the third term's size unknown.
assumptions (7)
  • domain assumption AA.1: V is twice continuously differentiable and V(x) = ||x||^2/2 outside a ball B_R (Dai et al. assumption).
    Invoked in Appendix A.2 to enable the Dai et al. analytic bound P_{X1}(V(X1) > τ) ≤ C exp(−(τ−ε)/σ). Many ML losses (ReLU networks) are not twice differentiable, and quadratic growth outside a ball is a strong tail assumption that is not checked for the benchmarks.
  • domain assumption AA.2: f = dP/dN(0,I) is differentiable, L-Lipschitz, and f ≥ c > 0 everywhere (Tzen et al.).
    The lower bound f ≥ c is violated by Boltzmann targets with small σ, which concentrate mass on narrow modes; the paper gives no check that the experiments satisfy this condition.
  • domain assumption AA.3: a polynomial-size feedforward net approximates f and ∇f to bε on the ball B_n(R) (Tzen et al.).
    The entire theoretical guarantee is conditional on this existence statement; the paper never measures the achieved approximation error of the trained PIO drift.
  • domain assumption AA.4: the drift net bv uses activations in {S, S', ReLU} and has polynomial size (Tzen et al.).
    Architectural assumption inherited from Tzen et al.; the paper's Fourier MLP is not analyzed for whether it satisfies this condition.
  • ad hoc to paper Adam training on the PIS loss (Definition T2.4) converges to the bε-approximate drift required by AA.3-AA.4 within the allowed step budget.
    Never stated or verified. The MNIST result (PIO 2.42 vs Adam 0.35, Table 1) is consistent with this assumption failing at 15.9k parameters. This is the theory-practice gap of the paper.
  • domain assumption The three ingredient bounds (Dai et al. 3.5, Tzen et al. 3.1, Vargas et al. A6) are correct and applicable under the stated assumptions.
    The appendix reproduces only the chaining argument, not the cited bounds; two of the three come from the same research lineage (Vargas is a co-author of [27] and [28]), so the present theorem is only as solid as those black-box inputs.
  • standard math Pinsker's inequality and the total-variation bound (Definitions TA.8 and TA.9).
    Standard inequalities used in the chaining step (Equation 4) that are not in question.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Path Integral Optimiser: Global Optimisation via Neural Schr\"odinger-F\"ollmer Diffusion." pith.science (2026). https://pith.science/paper/6ZJJCGUC

@misc{pith2026250606815,
  author       = {Pith},
  title        = {Pith review of: Path Integral Optimiser: Global Optimisation via Neural Schr\"odinger-F\"ollmer Diffusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6ZJJCGUC}},
  note         = {Machine review of arXiv:2506.06815}
}
read the original abstract

We present an early investigation into the use of neural diffusion processes for global optimisation, focusing on Zhang et al.'s Path Integral Sampler. One can use the Boltzmann distribution to formulate optimization as solving a Schr\"odinger bridge sampling problem, then apply Girsanov's theorem with a simple (single-point) prior to frame it in stochastic control terms, and compute the solution's integral terms via a neural approximation (a Fourier MLP). We provide theoretical bounds for this optimiser, results on toy optimisation tasks, and a summary of the stochastic theory motivating the model. Ultimately, we found the optimiser to display promising per-step performance at optimisation tasks between 2 and 1,247 dimensions, but struggle to explore higher-dimensional spaces when faced with a 15.9k parameter model, indicating a need for work on adaptation in such environments.

Figures

Figures reproduced from arXiv: 2506.06815 by the authors.

Figure 1
Figure 1. The Path Integral Optimiser is a neural-approximated Schrodinger-F ¨ ollmer process ( ¨ T2.3) which minimises a loss function V by learning to generate samples from its Boltzmann distribution (green). The drift term is computed by a Fourier MLP (red), and the process as a whole is simulated with Euler-Maruyama discretisation (blue). With correct parameterisation of θ, σ, and T, this architecture is capable of global… view at source ↗
Figure 2
Figure 2. Univariate optimisation target functions. e −V1(x) 1 e −V2(x) 1 e −V3(x) 1 [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
Figure 3
Figure 3. Univariate optimisation target functions following the Boltzman transformation with σ = 1. e −V1(x) 0.1 e −V2(x) 0.1 e −V3(x) 0.1 [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Univariate optimisation target functions following the Boltzman transformation with σ = 0.1. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 6
Figure 6. Figure 6: On the left: PIO’s final attempt at the Moons labelling task after ×32 sweeping, with ground truth marked with a small dot in the center of each circle. On the right: a histogram of PIO’s labels compared to the ground truth, demonstrating room for improvement. Pred: 6,…
Figure 7
Figure 7. Figure 7: The MNIST labels predicted by PIO’s best run in its ×32 hyperparameter sweep; every label is the same, implying that the optimiser is stuck in a very early local minima. 0 5 10 15 20 25 30 35 40 −15 −10 −5 0 5 10 FeedForwardNN: values of 41 parameters (|max|=16.0, |min…
Figure 9
Figure 9. Figure 9: Training loss V (x) for various optimisers, tuned with a Narrow ×32 hyperparameter sweep of 32 runs. Confidence intervals are computed by re-running the best-found configuration across multiple seeds. E.0.2. WIDE ×64 SWEEP 0 20 40 60 80 100 Step 10−2 10−1 100 101 Loss …
Figure 10
Figure 10. Figure 10: Training loss V (x) for various optimisers, tuned with a Wide ×64 hyperparameter sweep of 64 runs, at learning three different task environments. Confidence intervals are com￾puted by re-running the best-found configuration across multiple seeds. Notably, PIO still fa…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 14 canonical work pages

  1. [1]

    Compound classification using the scikit-learn library.Tutorials in Chemoinformatics, pages 223–239, 2017

    Jenny Balfer, J ¨urgen Bajorath, and Martin V ogt. Compound classification using the scikit-learn library.Tutorials in Chemoinformatics, pages 223–239, 2017

    work page 2017

  2. [2]

    A consensus-based global optimization method for high dimensional machine learning problems.ESAIM: Control, Optimisation and Calculus of Variations, 27:S5, 2021

    Jos ´e A Carrillo, Shi Jin, Lei Li, and Yuhua Zhu. A consensus-based global optimization method for high dimensional machine learning problems.ESAIM: Control, Optimisation and Calculus of Variations, 27:S5, 2021

    work page 2021

  3. [3]

    On empirical comparisons of optimizers for deep learning.arXiv preprint arXiv:1910.05446, 2019

    Dami Choi, Christopher J Shallue, Zachary Nado, Jaehoon Lee, Chris J Maddison, and George E Dahl. On empirical comparisons of optimizers for deep learning.arXiv preprint arXiv:1910.05446, 2019

    arXiv 1910

  4. [4]

    Global Optimization via Schr{\"o}dinger-F{\"o}llmer Diffusion

    Yin Dai, Yuling Jiao, Lican Kang, Xiliang Lu, and Jerry Zhijian Yang. Global Optimization via Schr¨odinger-F¨ollmer Diffusion.arXiv preprint arXiv:2111.00402, 2021

    work page Pith review arXiv 2021

  5. [5]

    Convergence guarantees for rmsprop and adam in non-convex optimization and an empirical comparison to nesterov acceleration.arXiv preprint arXiv:1807.06766, 2018

    Soham De, Anirbit Mukherjee, and Enayat Ullah. Convergence guarantees for rmsprop and adam in non-convex optimization and an empirical comparison to nesterov acceleration.arXiv preprint arXiv:1807.06766, 2018

    arXiv 2018

  6. [6]

    UCI machine learning repository, 2017

    Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URLhttps:// archive.ics.uci.edu/ml

    work page 2017

  7. [7]

    Adaptive subgradient methods for online learning and stochastic optimization.Journal of machine learning research, 12(7), 2011

    John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization.Journal of machine learning research, 12(7), 2011

    2011

  8. [8]

    An entropy approach to the time reversal of diffusion processes

    Hans F ¨ollmer. An entropy approach to the time reversal of diffusion processes. InStochastic Differential Systems Filtering and Control: Proceedings of the IFIP-WG 7/1 Working Confer- ence Marseille-Luminy, France, March 12–17, 1984, pages 156–163. Springer, 2005

    work page 1984

Show all 31 references
  1. [9]

    Schr ¨odinger-f¨ollmer sampler: sampling without ergodicity.arXiv preprint arXiv:2106.10880, 2021

    Jian Huang, Yuling Jiao, Lican Kang, Xu Liao, Jin Liu, and Yanyan Liu. Schr ¨odinger-f¨ollmer sampler: sampling without ergodicity.arXiv preprint arXiv:2106.10880, 2021

    2021 arXiv

  2. [10]

    Improving generalization performance by switching from adam to sgd.arXiv preprint arXiv:1712.07628, 2017

    Nitish Shirish Keskar and Richard Socher. Improving generalization performance by switching from adam to sgd.arXiv preprint arXiv:1712.07628, 2017

    2017 arXiv

  3. [11]

    Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  4. [12]

    Kirkpatrick, C

    S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. Science, 220(4598):671–680, 1983. doi: 10.1126/science.220.4598.671. URLhttps: //www.science.org/doi/abs/10.1126/science.220.4598.671

    1983 doi

  5. [13]

    The mnist database of handwritten digits.http://yann

    Yann LeCun. The mnist database of handwritten digits.http://yann. lecun. com/exdb/mnist/, 1998

    1998

  6. [14]

    On the convergence of stochastic gradient descent with adaptive stepsizes

    Xiaoyu Li and Francesco Orabona. On the convergence of stochastic gradient descent with adaptive stepsizes. InThe 22nd international conference on artificial intelligence and statis- tics, pages 983–992. PMLR, 2019. 7 PATHINTEGRALOPTIMISER

    2019

  7. [15]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URLhttps://openreview.net/forum?id=Bkg6RiCqY7

    2019

  8. [16]

    Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, and James Martens

    Arvind Neelakantan, Luke Vilnis, Quoc V . Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, and James Martens. Adding gradient noise improves learning for very deep networks.CoRR, abs/1511.06807, 2015. URLhttp://arxiv.org/abs/1511.06807

    2015 arXiv

  9. [17]

    A method of solving a convex programming problem with con- vergence rate o\bigl(kˆ2\bigr)

    Yurii Evgen’evich Nesterov. A method of solving a convex programming problem with con- vergence rate o\bigl(kˆ2\bigr). InDoklady Akademii Nauk, volume 269, pages 543–547. Russian Academy of Sciences, 1983

    1983

  10. [18]

    On the expressive power of deep neural networks

    Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, and Jascha Sohl-Dickstein. On the expressive power of deep neural networks. Ininternational conference on machine learning, pages 2847–2854. PMLR, 2017

    2017

  11. [19]

    Hierarchical text-conditional image generation with clip latents.arXiv preprint arXiv:2204.06125, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents.arXiv preprint arXiv:2204.06125, 2022

    2022 arXiv

  12. [20]

    Rapin and O

    J. Rapin and O. Teytaud. Nevergrad - A gradient-free optimization platform.https:// GitHub.com/FacebookResearch/Nevergrad, 2018

    2018

  13. [21]

    To smooth a cloud or to pin it down: Expressiveness guarantees and insights on score matching in denoising diffusion models

    Teodora Reu, Francisco Vargas, Anna Kerekes, and Michael M Bronstein. To smooth a cloud or to pin it down: Expressiveness guarantees and insights on score matching in denoising diffusion models. InThe 40th Conference on Uncertainty in Artificial Intelligence

  14. [22]

    A stochastic approximation method.The annals of math- ematical statistics, pages 400–407, 1951

    Herbert Robbins and Sutton Monro. A stochastic approximation method.The annals of math- ematical statistics, pages 400–407, 1951

    1951

  15. [23]

    Rubinstein

    Reuven Y . Rubinstein. Optimization of computer simulation models with rare events.Euro- pean Journal of Operational Research, 99(1):89–112, 1997. ISSN 0377-2217. doi: https:// doi.org/10.1016/S0377-2217(96)00385-2. URLhttps://www.sciencedirect.com/ science/article/pii/S037722...

    1997 doi

  16. [24]

    A generalized path integral control approach to reinforcement learning.J

    Evangelos Theodorou, Jonas Buchli, and Stefan Schaal. A generalized path integral control approach to reinforcement learning.J. Mach. Learn. Res., 11:3137–3181, December 2010. ISSN 1532-4435

    2010

  17. [25]

    An incremental gradient (-projection) method with momentum term and adaptive stepsize rule.SIAM Journal on Optimization, 8(2):506–531, 1998

    Paul Tseng. An incremental gradient (-projection) method with momentum term and adaptive stepsize rule.SIAM Journal on Optimization, 8(2):506–531, 1998

    1998

  18. [26]

    Theoretical guarantees for sampling and inference in generative models with latent diffusions

    Belinda Tzen and Maxim Raginsky. Theoretical guarantees for sampling and inference in generative models with latent diffusions. InConference on Learning Theory, pages 3084–

  19. [27]

    Bayesian learning via neural schr ¨odinger-f¨ollmer flows.arXiv preprint arXiv:2111.10510, 2021

    Francisco Vargas, Andrius Ovsianas, David Fernandes, Mark Girolami, Neil D Lawrence, and Nikolas N ¨usken. Bayesian learning via neural schr ¨odinger-f¨ollmer flows.arXiv preprint arXiv:2111.10510, 2021. 8 PATHINTEGRALOPTIMISER

    2021 arXiv

  20. [28]

    Denoising diffusion samplers.arXiv preprint arXiv:2302.13834, 2023

    Francisco Vargas, Will Grathwohl, and Arnaud Doucet. Denoising diffusion samplers.arXiv preprint arXiv:2302.13834, 2023

    2023 arXiv

  21. [29]

    Bayesian learning via stochastic gradient langevin dy- namics

    Max Welling and Yee Whye Teh. Bayesian learning via stochastic gradient langevin dy- namics. In Lise Getoor and Tobias Scheffer, editors,Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28 - July 2, 2011, pages 68...

    2011

  22. [30]

    Path integral sampler: a stochastic control approach for sampling.arXiv preprint arXiv:2111.15141, 2021

    Qinsheng Zhang and Yongxin Chen. Path integral sampler: a stochastic control approach for sampling.arXiv preprint arXiv:2111.15141, 2021. 9 PATHINTEGRALOPTIMISER Appendix A. Proof of Main Theorem T4.6 A.1. General Approach We can build up our result by considering several Schr...

    2021 arXiv

  23. [31]

    10 PATHINTEGRALOPTIMISER A.3

    Note that this takes advantage of the fact thatV(X 1)andV( bY1)have the same event space, thusV(X 1)> τis the same event asV( bY1)> τ. 10 PATHINTEGRALOPTIMISER A.3. Proof of Main Theorem T4.6 We can phrase our optimisation probability in terms of its distance from the continuo...

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.