Nesterov Acceleration for Ensemble Kalman Inversion and Variants

Eviatar Bach; Oliver R. A. Dunbar; Sydney Vernon

REVIEW 2 major objections 5 minor 34 references

Nesterov Acceleration for Ensemble Kalman Inversion and Variants

T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Nesterov acceleration, inserted as a particle nudge, makes ensemble Kalman inversion converge faster at zero extra cost.

desk verdict Useful empirical acceleration for EKI, but the formal derivation in Appendix A is arithmetically wrong and should be fixed or demoted to heuristic. read the letter →

arxiv 2501.08779 v2 pith:ZFT2IWGT submitted 2025-01-15 math.OC cs.LGstat.CO

Sydney Vernon , Eviatar Bach , Oliver R. A. Dunbar This is my paper · ORCID

classification math.OCcs.LGstat.CO MSC 65J2265C05

keywords NesterovaccelerationensembleKalmaninversionunscentedtransforminverseproblemsmomentummethodsderivative-freeoptimizationmean-fieldgradientflow

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Ensemble Kalman inversion (EKI) is a derivative-free particle method for inverse problems, and the paper attempts to show that Nesterov acceleration—the standard momentum trick from gradient-based optimization—can be attached to it as a per-particle nudge. The proposed update is $v_j^{(n)} = u_j^{(n)} + \lambda_j(u_j^{(n)} - u_{j-1}^{(n)})$ with $\lambda_j = (j-1)/(j+2)$, applied before each ordinary EKI step; the same nudge is carried over to unscented Kalman inversion and ensemble transform Kalman inversion. The paper reports that on all tested inverse problems the accelerated variants reduce the cost function faster, with no extra forward-model evaluations and no new hyperparameters. If this holds, a practitioner could speed up an existing EKI code with a two-line change, and the continuous-time gradient-flow link would open the door to porting other gradient-descent accelerations into Kalman-based optimization. The acceleration result is numerical; the paper provides a formal continuous-time limit but notes that the key convergence improvement for the covariance-preconditioned flow remains unproved.

What carries the argument

The load-bearing object is the particle-level momentum nudge, $v_j^{(n)} = u_j^{(n)} + \lambda_j(u_j^{(n)} - u_{j-1}^{(n)})$, inserted before the covariance-based update of EKI, UKI, or ETKI. It is what lets the discrete algorithm look like a covariance-preconditioned Nesterov flow in the formal small-timestep, large-ensemble limit; the paper derives that limit by Taylor expansion and Neumann-series expansion of the covariance inverse, obtaining $\ddot{u} + (3/t)\dot{u} = C_{uG}\Gamma^{-1}(y - G(u))$ as motivation. The nudge preserves the invariant-subspace property of EKI, uses no extra forward-model evaluations, and is the only change separating the accelerated and unaccelerated algorithms.

What would settle it

Run accelerated and unaccelerated EKI on an inverse problem outside the tested set where the forward model is nonsmooth enough to break the covariance-to-Jacobian approximation, and count forward-model evaluations to reach a fixed cost; one problem on which the nudge requires no fewer evaluations would falsify the claimed acceleration.

Watch

Extended reading notes

Core claim

The central claim is that momentum, applied at the level of individual particles rather than to the ensemble statistics, reliably speeds up cost-function reduction in EKI and its variants. In place of the usual update on $u_j^{(n)}$, the algorithm first forms the nudged particle $v_j^{(n)} = u_j^{(n)} + \lambda_j(u_j^{(n)} - u_{j-1}^{(n)})$ and then runs the standard update on the nudged ensemble; with $\lambda_j = (j-1)/(j+2)$ there is no additional tuning. The paper motivates this by treating EKI as a discretization of the covariance-preconditioned gradient flow and the accelerated version as the formal limit of a covariance-preconditioned Nesterov ODE, $\ddot{u} + \lambda(t)\dot{u} = C_{uG}\Gamma^{-1}(y - G(u))$. The experiments show faster cost reduction across three problems of different dimensionality and nonlinearity, and the accelerated algorithm retains the property that all particles remain in the span of the initial ensemble. Temporary cost increases occur in some runs because momentum methods are not descent methods; overall the accelerated runs perform at least as well and often better.

Load-bearing premise

The argument assumes the discrete finite-ensemble algorithm inherits the acceleration of the continuous-time Nesterov flow; the paper states that this inheritance remains unproved, and that the small-timestep and infinite-ensemble limits may not commute.

Editorial extensions

If this is right

Existing EKI, UKI, and ETKI codes can acquire the acceleration by inserting two lines into the particle loop, leaving the update, the forward model, and the number of evaluations unchanged.
The coefficient $\lambda_j = (j-1)/(j+2)$ needs no tuning; a recursively defined variant behaves similarly, while a constant momentum coefficient can perform worse than no acceleration.
The speedup appears in the first few iterations as well as in the long run, so small iteration budgets still benefit.
Acceleration does not change the regularization implied by the initial ensemble, since particles stay in its span.
If the formal mean-field limit is made rigorous, the method inherits an interpretation as a covariance-preconditioned Nesterov flow, and the formal argument gives a target equation for that proof.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

One testable extension the paper does not run is to shut the nudge off after a fixed number of iterations, which could retain the early speedup while avoiding the temporary cost increases seen in momentum methods.
Because the nudge is decoupled from the specific covariance update, the same two-line momentum could plausibly be applied to ensemble Kalman samplers or other derivative-free particle methods whose updates approximate preconditioned gradients; this would need its own numerical validation.
The formal limit treats $\Delta t \to 0$ before $N \to \infty$, so for strongly nonlinear forward models or very small ensembles the speedup is not guaranteed by the theory given here and should be checked on the user's own problem before being relied on.
A natural benchmark would be comparing accelerated vs unaccelerated EKI on problems where the covariance-to-Jacobian approximation is known to be poor, to map where the empirical benefit ends.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a practical, honestly written paper that shows a simple Nesterov momentum nudge speeds up EKI, UKI, and ETKI on three inverse problems. The numerical work is the contribution; the theory in Appendix A is not sound as written.

What's actually new: Kovachki and Stuart suggested Nesterov acceleration for EKI but didn't test it; Nilsen et al. used constant momentum on the ensemble mean and covariance. Here the nudge is per-particle, the lambda_j schedule is the standard Nesterov one (or the recursive variant), and the extension to UKI and ETKI is new. The experiments are the real substance: three problems, standard errors, robustness checks over ensemble size, timestep, and three choices of lambda. The improvement is consistent and often large. The method is genuinely black-box, costs nothing extra, and is easy to implement in existing codes. That is worth having.

Soft spots, in proportion. The biggest is Appendix A. The derivation claims that D^{-1} is o(Delta t) where D = Gamma + Delta t CGG; for invertible Gamma it is actually O(1). The same error appears in the Neumann expansion a few lines earlier. That invalidates the asymptotic matching that yields (A.6), so the paper does not establish the advertised connection between Algorithm 1 and the Nesterov-accelerated mean-field EKI, even as a formal limit. The authors are honest that the theory is incomplete, but a flawed formal derivation is worse than none: it could mislead readers into thinking the lambda_j schedule is derived rather than borrowed from optimization. I'd want this fixed or explicitly marked as heuristic.

Minor issues: Algorithm 1's first update uses G(u_1) before u_1 exists (presumably G(u_0)); the UKI alpha value is not reported numerically; no scripts or commit hash are provided; and there is no comparison with Chada-Tong's convergence acceleration. None of these change the empirical conclusion, but they should be cleaned up.

Who is this for? Practitioners running EKI with expensive forward models, and researchers interested in transferring gradient-based optimization tricks into ensemble methods. The central empirical claim is credible. With the Appendix A issue addressed, this deserves a serious referee; I'd engage with it.

Referee Report

2 major / 5 minor

Summary. The paper proposes a Nesterov-acceleration scheme for Ensemble Kalman inversion (EKI) and its variants UKI and ETKI. The method is a particle-level momentum nudge v_j^{(n)} = u_j^{(n)} + λ_j (u_j^{(n)} - u_{j-1}^{(n)}), with λ_j taken from the continuous-time Nesterov literature. The authors report numerical experiments on three inverse problems (exponential sine, Lorenz 96, Darcy flow) showing consistent speedups in cost-function reduction with no extra tuning hyperparameters and negligible computational overhead. They also provide a formal continuous-time limit (Appendix A) intended to motivate the algorithm and additional experiments on ensemble size, timestep, and λ schedule in the appendices.

Significance. If the empirical finding is correct, the paper contributes a simple, black-box, hyperparameter-free acceleration that could benefit any existing EKI implementation, including variants. The numerical study is carefully conducted: multiple independent trials, standard-error ribbons, and robustness checks over ensemble size and timestep. The authors are also honest about the lack of proof for the covariance-preconditioned acceleration property and about the informal nature of the mean-field limit. The main weakness is that the formal continuous-time derivation in Appendix A contains an incorrect asymptotic-order statement, though the intended conclusion can be recovered with a corrected argument; this does not undermine the numerical evidence but does require revision of the theory-motivation section.

major comments (2)

[Appendix A] The asymptotic-order reasoning in Appendix A is incorrect as written. After defining D = Γ + Δt CGG(U(·)(t)), the text states that 'D^{-1} is of order Δt' and later 'o(Δt)'. For invertible Γ, D^{-1} = Γ^{-1} - Δt Γ^{-1} CGG Γ^{-1} + O((Δt)^2), which is O(1) as Δt → 0, not O(Δt). The preceding Neumann-series line (A−ΔtB)^{-1} = A^{-1} + Σ ... = o(Δt) is also wrong because the leading term is Γ^{-1}. However, the final asymptotic relations (A.4) and √Δt F^n({U}) = √Δt F̄^n({U}) + o(√Δt) do follow from the correct order: since (Γ+ΔtCGG)^{-1} - Γ^{-1} = O(Δt), multiplying by √Δt gives O((Δt)^{3/2}) = o(√Δt). Please rewrite the appendix with correct order estimates; as written, the derivation of the continuous-time limit is invalid, even though the intended conclusion is recoverable.
[Algorithm 1 / Section 3] There is a mismatch between the algorithm presented as the main method and the one used in the headline experiments. Algorithm 1 line 4 defines λ_j = (j-1)/(j+2), but Appendix B states that 'In the numerical experiments that follow in Section 3, we used the Recursive definition unless otherwise stated.' Since the Recursive formula is not given in the main text, the paper is not self-contained for reproducing the results of Figure 1. Please state explicitly which λ schedule is used for each experiment, and either use the Original schedule in the main experiments or move the Recursive definition into the main text so that Algorithm 1 matches the reported results.

minor comments (5)

[Section 3.1 / Section 3.3] The Darcy problem setup states N = 80 (Section 3.1), while Section 3.3 and Figure D.3 use N = 52 for the main Darcy experiment and N = 10 and 200 for the ensemble-size robustness check. Please align these numbers or explain the discrepancy.
[Section 3] The text says 'We present results of experiments over fifty trials', but Figure 1 uses only 10 trials for Darcy. Please specify the trial count per problem consistently.
[Appendix A] The line 'set j = t/√Δt − 2' appears to be a typo; given the ansatz u_j = U((j−2)√Δt), one should have j = t/√Δt + 2.
[Appendix A] The displayed Neumann-series formula contains an extra A^{-1} term and is dimensionally inconsistent. Please rewrite the expression as (A−ΔtB)^{-1} = Σ_{k=0}^∞ (Δt)^k (A^{-1}B)^k A^{-1}.
[Section 1.4 / Section 3.3] The paper cites existing accelerated-ensemble methods (e.g., [21], [22]) but does not compare against them. A short numerical comparison would help readers judge the practical advantage over prior momentum-based accelerations, though it is not required for the paper's main claim.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the momentum schedule is imported from the Nesterov literature, the theoretical limit is explicitly disclaimed as formal, and the empirical acceleration claim does not reduce to a fitted parameter.

full rationale

The paper's central claim is an empirical one: Nesterov's momentum nudge speeds up cost-function reduction for EKI, UKI, and ETKI. The momentum coefficient λ_j is taken directly from the optimization literature ([18,19]), not fitted to the test problems, and the paper explicitly reports that both the Original and Recursive schedules outperform the baseline while the constant-coefficient variant does not (Appendix B). The experimental comparisons in Section 3 use the Recursive schedule, but the improvement over non-accelerated EKI is not forced by a fitted constant and is robust across ensemble sizes, timesteps, and λ variants (Appendix D). The theoretical bridge in Appendix A is explicitly presented as a formal motivation rather than a proved derivation: Section 2 states that 'it remains to be proved that the covariance-preconditioned Nesterov will also have this improvement', Appendix A calls the argument 'formal' and notes that the limits 'may not commute', and Section 5 says the motivation 'still requires rigorous justification'. Even if the skeptic's arithmetic objection to the D^{-1} order in Appendix A is correct, that is a rigor/correctness issue, not circularity: the numerical results are not constructed from the continuous-time equation. The only self-citation is to the authors' EnsembleKalmanProcesses.jl package used for implementation ([26]), which is not load-bearing evidence for the acceleration claim. No equation or parameter was found that makes the reported predictions equivalent to the inputs by construction.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the standard EKI-to-gradient-flow analogy (Eq. 4), the unproven transfer of Nesterov acceleration to the covariance-preconditioned flow, and the formal discrete-to-continuum limit in Appendix A. No new entities are introduced. One UKI hyperparameter is left unspecified, preventing exact reproduction of the UKI experiments.

free parameters (1)

UKI parameter alpha
The paper states only alpha in (0,1] must be chosen for the UKI experiments and references [15], without reporting the value actually used. This prevents exact reproduction of the UKI results, though the acceleration comparison is internally consistent.

assumptions (3)

domain assumption EKI approximates the covariance-preconditioned gradient flow via C_uG approximately C_uu dG/du^T (Eq. 4), valid when the second derivative of G is small.
Invoked in Section 1.2 to derive the mean-field EKI; inherited from [11].
ad hoc to paper The covariance-preconditioned Nesterov ODE inherits the acceleration property of standard Nesterov flows.
Stated in Section 2 as unproved: 'it remains to be proved that the covariance-preconditioned Nesterov will also have this improvement'.
ad hoc to paper The discrete Algorithm 1 converges to the mean-field Nesterov-accelerated EKI (9) in the joint limit Delta t to 0 and N to infinity with N Delta t = 1, with the limits commuting.
Appendix A gives a formal argument and states 'these limits may not commute'; convergence to (9) is not rigorously established.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Nesterov Acceleration for Ensemble Kalman Inversion and Variants." pith.science (2026). https://pith.science/paper/ZFT2IWGT

@misc{pith2026250108779,
  author       = {Pith},
  title        = {Pith review of: Nesterov Acceleration for Ensemble Kalman Inversion and Variants},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZFT2IWGT}},
  note         = {Machine review of arXiv:2501.08779}
}

read the original abstract

Ensemble Kalman inversion (EKI) is a derivative-free, particle-based optimization method for solving inverse problems. It can be shown that EKI approximates a gradient flow, which allows the application of methods for accelerating gradient descent. Here, we show that Nesterov acceleration is effective in speeding up the reduction of the EKI cost function on a variety of inverse problems. We also implement Nesterov acceleration for two EKI variants, unscented Kalman inversion and ensemble transform Kalman inversion. Our specific implementation takes the form of a particle-level nudge that is demonstrably simple to couple in a black-box fashion with any existing EKI variant algorithms, comes with no additional computational expense, and with no additional tuning hyperparameters. This work shows a pathway for future research to translate advances in gradient-based optimization into advances in gradient-free Kalman optimization.

Figures

Figures reproduced from arXiv: 2501.08779 by the authors.

**Figure 1.** Comparison of EKI, ETKI, and UKI convergence with and without acceleration, over the three inverse problems of Section 3.1, as measured by the [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 23 canonical work pages

[1]

M. A. Iglesias, K. J. H. Law, A. M. Stuart, Ensemble Kalman methods for inverse problems, Inverse Problems 29 (4) (2013) 045001. doi:10. 1088/0266-5611/29/4/045001

work page 2013
[2]

P. J. van Leeuwen, G. Evensen, Data assimilation and inverse meth- ods in terms of a probabilistic formulation, Monthly Weather Re- view 124 (12) (1996) 2898–2913. doi:10.1175/1520-0493(1996) 124<2898:DAAIMI>2.0.CO;2

work page doi:10.1175/1520-0493(1996 1996
[3]

Evensen, P

G. Evensen, P. J. Van Leeuwen, An ensemble Kalman smoother for non- linear dynamics, Monthly Weather Review 128 (6) (2000) 1852–1867. doi:10.1175/1520-0493(2000)128<1852:AEKSFN>2.0.CO;2

doi:10.1175/1520-0493(2000)128 2000
[4]

Y . Chen, D. S. Oliver, D. Zhang, E fficient ensemble-based closed-loop production optimization, SPE Journal 14 (04) (2009) 634–645. doi:10. 2118/112873-PA

work page 2009
[5]

Y . Chen, D. S. Oliver, Ensemble randomized maximum likelihood method as an iterative ensemble smoother, Mathematical Geosciences 44 (2012) 1–26. doi:10.1007/s11004-011-9376-z

work page doi:10.1007/s11004-011-9376-z 2012
[6]

Bocquet, P

M. Bocquet, P. Sakov, An iterative ensemble Kalman smoother, Quar- terly Journal of the Royal Meteorological Society 140 (682) (2014) 1521–

work page 2014
[7]

G. Evensen, Sequential data assimilation with a nonlinear quasi- geostrophic model using Monte Carlo methods to forecast error statistics, Journal of Geophysical Research: Oceans 99 (C5) (1994) 10143–10162. doi:10.1029/94JC00572

doi:10.1029/94jc00572 1994
[8]

Garbuno-Inigo, F

A. Garbuno-Inigo, F. Hoffmann, W. Li, A. M. Stuart, Interacting Langevin diffusions: Gradient structure and ensemble Kalman sampler, SIAM Journal on Applied Dynamical Systems 19 (1) (2020) 412–441. doi: 10.1137/19M1251655

doi:10.1137/19m1251655 2020

Show all 34 references

[9]

Cleary, A

E. Cleary, A. Garbuno-Inigo, S. Lan, T. Schneider, A. M. Stuart, Cal- ibrate, emulate, sample, Journal of Computational Physics 424 (2021) 109716. doi:10.1016/j.jcp.2020.109716

2021
[10]

Iglesias, Y

M. Iglesias, Y . Yang, Adaptive regularisation for ensemble Kalman inversion, Inverse Problems 37 (2) (2021) 025008. doi:10.1088/ 1361-6420/abd29b

2021
[11]

Calvello, S

E. Calvello, S. Reich, A. M. Stuart, Ensemble Kalman methods: A mean field perspective (Aug. 2024). arXiv:2209.11371

2024 arXiv
[12]

Bl ¨omker, C

D. Bl ¨omker, C. Schillings, P. Wacker, A strongly convergent numerical scheme from ensemble Kalman inversion, SIAM Journal on Numerical Analysis 56 (4) (2018) 2537–2562. doi:10.1137/17M1132367

2018 doi
[13]

X. T. Tong, M. Morzfeld, Localized ensemble Kalman inversion, Inverse Problems 39 (6) (2023) 064002. doi:10.1088/1361-6420/accb08

2023 doi
[14]

Z. Ding, Q. Li, Ensemble Kalman inversion: Mean-field limit and con- vergence analysis, Statistics and Computing 31 (1) (2021) 9. doi: 10.1007/s11222-020-09976-0

2021 doi
[15]

D. Z. Huang, T. Schneider, A. M. Stuart, Iterated Kalman methodol- ogy for inverse problems, Journal of Computational Physics 463 (2022) 111262. doi:10.1016/j.jcp.2022.111262

2022
[16]

M. K. Tippett, J. L. Anderson, C. H. Bishop, T. M. Hamill, J. S. Whitaker, Ensemble square root filters, Monthly Weather Review 131 (7) (2003) 1485–1490. doi:10.1175/1520-0493(2003)131<1485:ESRF>2.0. CO;2

2003 doi
[17]

Y . E. Nesterov, A method of solving a convex programming problem with convergence rate O(1/k2), Soviet Mathematics—Doklady 27 (2) (1983) 372–376

1983
[18]

Nesterov, Introductory Lectures on Convex Optimization, V ol

Y . Nesterov, Introductory Lectures on Convex Optimization, V ol. 87 of Applied Optimization, Springer US, Boston, MA, 2004. doi:10.1007/ 978-1-4419-8853-9

2004
[19]

W. Su, S. Boyd, E. J. Cand `es, A differential equation for modeling Nes- terov’s accelerated gradient method: Theory and insights, Journal of Ma- chine Learning Research 17 (153) (2016) 1–43. URL https://jmlr.org/papers/v17/15-084.html

2016
[20]

N. B. Kovachki, A. M. Stuart, Ensemble Kalman inversion: A derivative- free technique for machine learning tasks, Inverse Problems 35 (9) (2019) 095005. doi:10.1088/1361-6420/ab1c3a

2019 doi
[21]

Chada, X

N. Chada, X. Tong, Convergence acceleration of ensemble Kalman inver- sion in nonlinear settings, Mathematics of Computation 91 (335) (2022) 1247–1280. doi:10.1090/mcom/3709

2022 doi
[22]

M. M. Nilsen, A. S. Stordal, R. J. Lorentzen, P. N. Raanes, K. S. Eikrem, Accelerated ensemble optimization using momentum methods, Scientific Reports 14 (1) (2024) 25336. doi:10.1038/s41598-024-76916-7

2024 doi
[23]

Z. Liu, A. M. Stuart, Y . Wang, Second order ensemble Langevin method for sampling and inverse problems (Oct. 2022). arXiv:2208.04506

2022 arXiv
[24]

E. N. Lorenz, Predictability: A problem partly solved, in: Proceedings of a Seminar Held at ECMWF on Predictability, 4-8 September 1995, V ol. 1, ECMWF, Shinfield Park, Reading, 1996, pp. 1–18

1995
[25]

J. Latz, M. Eisenberger, E. Ullmann, Fast sampling of parameterised Gaussian random fields, Computer Methods in Applied Mechanics and Engineering 348 (2019) 978–1012. doi:https://doi.org/10.1016/ j.cma.2019.02.003

2019
[26]

O. R. A. Dunbar, I. Lopez-Gomez, A. Garbuno-I ˜nigo, D. Z. Huang, E. Bach, J.-l. Wu, EnsembleKalmanProcesses.jl: Derivative-free ensemble-based model calibration, Journal of Open Source Software 7 (80) (2022) 4869. doi:10.21105/joss.04869

2022 doi
[27]

Y . Wang, W. Li, Accelerated information gradient flow, Jour- nal of Scientific Computing 90 (1) (2021) 11. doi:10.1007/ s10915-021-01709-3

2021
[28]

Taylor, Y

A. Taylor, Y . Drori, An optimal gradient method for smooth strongly con- vex minimization, Mathematical Programming 199 (1) (2023) 557–594. doi:10.1007/s10107-022-01839-y

2023 doi
[29]

Van Scoy, R

B. Van Scoy, R. A. Freeman, K. M. Lynch, The fastest known glob- ally convergent first-order method for minimizing strongly convex func- tions, IEEE Control Systems Letters 2 (1) (2018) 49–54. doi:10.1109/ LCSYS.2017.2722406

2018
[30]

d’Aspremont, D

A. d’Aspremont, D. Scieur, A. Taylor, Acceleration methods, Founda- tions and Trends® in Optimization 5 (1-2) (2021) 1–245.doi:10.1561/ 2400000036

2021
[31]

D. P. Kingma, J. Ba, Adam: A method for stochastic optimization (Jan. 2017). arXiv:1412.6980

2017 arXiv
[32]

Dozat, Incorporating Nesterov momentum into Adam, in: Proceedings of the 4th International Conference on Learning Representations, Work- shop Track, San Juan, Puerto Rico, 2016

T. Dozat, Incorporating Nesterov momentum into Adam, in: Proceedings of the 4th International Conference on Learning Representations, Work- shop Track, San Juan, Puerto Rico, 2016. URL https://openreview.net/forum?id= 8 OM0jvwB8jIp57ZJjtNEZ

2016
[33]

X. Xie, P. Zhou, H. Li, Z. Lin, S. Yan, Adan: Adaptive Nesterov mo- mentum algorithm for faster optimizing deep models, IEEE Transactions on Pattern Analysis and Machine Intelligence 46 (12) (2024) 9508–9520. doi:10.1109/TPAMI.2024.3423382

2024
[34]

N. B. Kovachki, A. M. Stuart, Continuous time analysis of momentum methods, Journal of Machine Learning Research 22 (17) (2021) 1–40. URL http://jmlr.org/papers/v22/19-466.html 9

2021

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

M. A. Iglesias, K. J. H. Law, A. M. Stuart, Ensemble Kalman methods for inverse problems, Inverse Problems 29 (4) (2013) 045001. doi:10. 1088/0266-5611/29/4/045001

work page 2013

[2] [2]

P. J. van Leeuwen, G. Evensen, Data assimilation and inverse meth- ods in terms of a probabilistic formulation, Monthly Weather Re- view 124 (12) (1996) 2898–2913. doi:10.1175/1520-0493(1996) 124<2898:DAAIMI>2.0.CO;2

work page doi:10.1175/1520-0493(1996 1996

[3] [3]

Evensen, P

G. Evensen, P. J. Van Leeuwen, An ensemble Kalman smoother for non- linear dynamics, Monthly Weather Review 128 (6) (2000) 1852–1867. doi:10.1175/1520-0493(2000)128<1852:AEKSFN>2.0.CO;2

doi:10.1175/1520-0493(2000)128 2000

[4] [4]

Y . Chen, D. S. Oliver, D. Zhang, E fficient ensemble-based closed-loop production optimization, SPE Journal 14 (04) (2009) 634–645. doi:10. 2118/112873-PA

work page 2009

[5] [5]

Y . Chen, D. S. Oliver, Ensemble randomized maximum likelihood method as an iterative ensemble smoother, Mathematical Geosciences 44 (2012) 1–26. doi:10.1007/s11004-011-9376-z

work page doi:10.1007/s11004-011-9376-z 2012

[6] [6]

Bocquet, P

M. Bocquet, P. Sakov, An iterative ensemble Kalman smoother, Quar- terly Journal of the Royal Meteorological Society 140 (682) (2014) 1521–

work page 2014

[7] [7]

G. Evensen, Sequential data assimilation with a nonlinear quasi- geostrophic model using Monte Carlo methods to forecast error statistics, Journal of Geophysical Research: Oceans 99 (C5) (1994) 10143–10162. doi:10.1029/94JC00572

doi:10.1029/94jc00572 1994

[8] [8]

Garbuno-Inigo, F

A. Garbuno-Inigo, F. Hoffmann, W. Li, A. M. Stuart, Interacting Langevin diffusions: Gradient structure and ensemble Kalman sampler, SIAM Journal on Applied Dynamical Systems 19 (1) (2020) 412–441. doi: 10.1137/19M1251655

doi:10.1137/19m1251655 2020