Pith. sign in

REVIEW 4 minor 58 references

Direct Acceleration of Stochastic Root-Finding Without Variance Reduction and Regularization

T0 review · 0 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A dual-anchor update carries acceleration into stochastic root-finding at O(ε⁻³) complexity, with constant batch size and no variance reduction.

desk verdict Solid, honest stochastic acceleration result; the O(epsilon^-3) claim holds under cocoercivity in expectation, and the authors say clearly where the assumption is doing the work. read the letter →

arxiv 2608.12043 v1 pith:7NAWZHGP submitted 2026-08-12 math.OC cs.LG

TaeHo Yoon , Nicolas Loizou This is my paper · ORCID
classification math.OCcs.LG MSC 65K1049J4090C25
keywords stochasticroot-findingdual-anchoraccelerationHalperniterationcocoercivityinexpectationoraclecomplexitymonotoneoperatorsfixed-pointmethodsmini-batching
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that acceleration for root-finding can survive stochastic noise when the right acceleration mechanism is used. The authors propose Stochastic Dual-OHM, a mini-batched version of the dual-anchor (dual-Halpern) update, and prove that for cocoercive operators it drives the expected squared residual to $4\|x_0-x^*\|^2/(\alpha^2 N^2) + 6\sigma^2/B$ after $N$ steps. Choosing $N=\Theta(1/\epsilon)$ and $B=\Theta(1/\epsilon^2)$ gives $\mathbb{E}[\|\mathbb{F}(x_{N-1})\|]\le\epsilon$ with $O(\epsilon^{-3})$ oracle evaluations, using constant batch size and no variance reduction or double-loop regularization. A sharper $\widetilde{O}(\epsilon^{-2})$ bound follows when the operator is strongly monotone. If true, stochastic acceleration is not inherently fragile; the choice of deterministic acceleration decides whether noise accumulates.

What carries the argument

The load-bearing object is the finite-horizon Dual-OHM update with weights $a_k=(N-k-1)/(N-k)$, which replaces the anchor-based convex combination by a telescoping correction between consecutive operator evaluations. The analysis rests on the exact algebraic identity of Lemma 4.2, expressing zero as a combination of the terminal residual, a cross term with the initial point, and weighted terms $Q_{N,j}$; while those terms are nonnegative in the deterministic case, in the stochastic case the paper controls their expected contribution by Lemmas 4.3 and 4.4. The crucial structural fact is that the total weight $\sum_{j=1}^{N-1}\lambda_{N,j}=\alpha(N-1)/2=O(N)$, whereas the analogous weight sum for stochastic OHM is $\Theta(N^2)$; this difference is what keeps the noise term bounded rather than $N$-dependent. Lemma A.2 provides the leave-one-out coupling that makes Lemma 4.4 hold under square-nonexpansivity in expectation.

What would settle it

Run S-Dual-OHM on the affine worst-case cocoercive operator with additive Gaussian noise, taking $N=1/\epsilon$ and $B=1/\epsilon^2$; if $\mathbb{E}[\|\mathbb{F}(x_{N-1})\|]$ does not decrease as $O(\epsilon)$ along a sequence $\epsilon\to0$, the main theorem is false. A sharper check is to compute $\mathbb{E}[\langle e_{j-1},\mathbb{T}x_{N-1}\rangle]$ for a moderately large $N$ and compare it with $\alpha\sigma^2/B$; if this inner product grows with $N$, the leave-one-out bound in Lemma 4.4 fails.

Watch

Extended reading notes

Core claim

The core discovery is that the dual-anchor update, defined for a fixed horizon $N$ by $x_{k+1}=x_k+\frac{N-k-1}{N-k}(\mathbb{T}_{B_k}(x_k)-\mathbb{T}_{B_{k-1}}(x_{k-1}))$, does not accumulate oracle error the way anchor-based Halpern methods do. Theorem 4.1 shows that under cocoercivity in expectation, S-Dual-OHM with step size $\alpha\in(0,2/L]$ and constant batch size $B$ satisfies $\mathbb{E}[\|\mathbb{F}(x_{N-1})\|]^2\le \mathbb{E}[\|\mathbb{F}(x_{N-1})\|^2]\le 4\|x_0-x^*\|^2/(\alpha^2 N^2) + 6\sigma^2/B$, so $O(\epsilon^{-3})$ oracle evaluations suffice for an $\epsilon$-accurate last iterate. The proof isolates the noise contribution through Lemma 4.4, whose key step is a leave-one-out stability bound showing that the error from one mini-batch propagates to the terminal iterate with size at most $\alpha\sigma^2/B$. For strongly monotone operators, early stopping yields a near-optimal $\widetilde{O}(\epsilon^{-2})$ oracle complexity.

Load-bearing premise

The proof needs the noisy oracle to be cocoercive on average: the expected squared gap between two noisy evaluations must be majorized by $L$ times the deterministic operator's monotonicity gap, a condition that holds for additive noise or per-sample cocoercive sums but fails for state-dependent noise and general monotone Lipschitz sample operators.

Editorial extensions

If this is right

  • Constant mini-batching with $B=\Theta(\epsilon^{-2})$ and $N=\Theta(\epsilon^{-1})$ reaches $\mathbb{E}[\|\mathbb{F}(x_{N-1})\|]\le\epsilon$ in $O(\epsilon^{-3})$ oracle evaluations for cocoercive-in-expectation operators, without variance reduction or double-loop regularization.
  • The same single-loop algorithm, early-stopped after $k=O((L/\mu)\log\epsilon^{-1})$ iterations, attains $\widetilde{O}(\epsilon^{-2})$ oracle complexity for strongly monotone operators, which is near-optimal in its $\epsilon$-dependence.
  • The guarantee is for the last iterate, not an averaged, best, or uniformly random iterate.
  • The finite-horizon dual-anchor update is optimized for its terminal iterate, whereas anytime anchor-based methods accumulate error weights of size $\Theta(N^2)$, which explains why the stochastic extension works for the former and not the latter.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The same "sum of proof weights" diagnostic could screen other members of the H-invariant family of optimal fixed-point algorithms for stochastic noise robustness before running any simulation.
  • Editorial inference: With a restart or batch-size schedule, the $O(\epsilon^{-3})$ result may be pushed toward the $\widetilde{O}(\epsilon^{-2})$ lower bound in the non-strongly-monotone case, since the present bottleneck is the variance term $6\sigma^2/B$.
  • Editorial inference: For finite-sum problems with per-component cocoercivity, the algorithm inherits a variance bound that may permit a finite-sum improvement analogous to what variance-reduced Halpern methods achieved for finite sums.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

0 major / 4 minor

Summary. The paper studies stochastic root-finding for cocoercive operators under an unbiased stochastic oracle. It proposes S-Dual-OHM, a mini-batch stochastic version of the dual-anchor Dual-OHM algorithm, and proves that under an additional cocoercivity-in-expectation assumption (Assumption 3.2), constant step size and constant batch size give E[||F(x_{N-1})||]^2 <= 4||x_0-x^*||^2/(alpha^2 N^2) + 6 sigma^2/B, which yields O(epsilon^{-3}) oracle complexity with a last-iterate guarantee and no variance reduction or double-loop regularization (Theorem 4.1). For strongly monotone cocoercive operators, the same algorithm is shown to achieve E[||F(x_k)||] <= epsilon in O((L/mu) log(1/epsilon)) iterations, giving eO(epsilon^{-2}) complexity under only Assumption 3.1 (Theorem 4.5). The appendix contains the algebraic identity, the error-propagation lemmas, the leave-one-out stability bound, and the contractive-case proofs, and the experiments compare S-Dual-OHM with anchor-based, variance-reduced, and regularized baselines.

Significance. If the result holds, it is a meaningful conceptual contribution: it shows that the choice of acceleration mechanism matters in the stochastic setting, and that the dual-anchor mechanism avoids the error accumulation that plagues stochastic Halpern-type methods. The proof is self-contained and the appendix is complete: Lemma 4.2 is a purely algebraic identity, Lemma A.2 is the key leave-one-out stability estimate, and Lemmas 4.3 and 4.4 correctly propagate the noise terms. The paper is also honest about the main restriction: Assumption 3.2 is genuinely stronger than unbiasedness with bounded variance, and the manuscript explicitly states that state-dependent noise and general monotone-Lipschitz sample operators are not covered. The strong-monotone result is a useful addition and uses only Assumption 3.1. The numerical study is careful, including a controlled comparison in Appendix B where S-OHM receives its own tuning. I find no load-bearing error in the central argument.

minor comments (4)
  1. [Appendix A.1, proof of Lemma 4.2] The 'ghost batch' B_{N-1} is informal; since the proof ultimately sets G_N = F(x_{N-1}), the identity is correct, but the exposition should state explicitly that this is only a bookkeeping device and not an additional stochastic oracle call.
  2. [Section 4.2, Theorem 4.5] The phrase 'for k=0,...,N-1' is slightly confusing because y_{N-1} is a fixed terminal point; I recommend writing 'for every fixed horizon N and every k in {0,...,N-1}' to avoid reading y_{N-1} as an iterate indexed by k.
  3. [Section 5, Figure 2 (right panel)] The vertical axis labels (1, 2 x 10^2, 3 x 10^2, 4 x 10^2, 6 x 10^2) use a nonstandard scale; a conventional log scale or linear ticks would improve readability.
  4. [Section 4.1, after Lemma 4.4] The derivation of the final bound would be easier to follow if the text explicitly wrote the step E[N alpha^2/4 ||F||^2 + alpha/2 <F,x^*-x_0>] - 3(N-1)alpha^2 sigma^2/(4B), since the Young and cocoercivity steps are otherwise compressed into the two displayed lines.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the S-Dual-OHM O(epsilon^-3) bound is proven from Assumptions 3.1/3.2 plus an external deterministic baseline, not from a fitted or self-defined quantity.

full rationale

The derivation of Theorem 4.1 is self-contained: Lemmas 4.2-4.4 and A.1-A.2 are proved in the paper from Assumptions 3.1 and 3.2, and the final bound E||F(x_{N-1})||^2 <= 4||x_0-x*||^2/(alpha^2 N^2) + 6 sigma^2/B follows by algebra from those lemmas. Assumption 3.2, cocoercivity in expectation, is a structural condition on the noisy oracle that bounds expected squared differences of the stochastic operator; it is not the target residual bound, and its use in Lemma A.2 to obtain square-nonexpansivity in expectation does not pre-suppose the conclusion. Deterministic Proposition 3.3 is imported from prior work [29,22,55]; [55] has an overlapping author, but the result is an established external worst-case rate for OHM/Dual-OHM and is used only as the noiseless building block. All stochastic parts--error propagation, leave-one-out stability, variance terms, and the complexity arithmetic N*B = O(epsilon^-1) * O(epsilon^-2) = O(epsilon^-3)--are derived in the appendix rather than assumed. The paper explicitly flags the monotone-Lipschitz extension and the optimal O(epsilon^-2 log(1/epsilon)) rate as open questions, which are limitations, not circular inputs. Therefore no load-bearing step reduces by construction to its own input.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The theorem introduces no fitted free parameters: the step size alpha, horizon N, and batch size B are selected from the problem constants L, D, sigma, and epsilon. The analysis depends on the stochastic oracle assumptions (3.1, 3.2) and on the deterministic Dual-OHM rate from prior work by the same group, which is an independent published result. No new entities are postulated.

assumptions (5)
  • domain assumption Stochastic oracle is unbiased and has bounded variance (Assumption 3.1).
    The central complexity bound depends on the variance bound sigma^2; without it, no finite-sample guarantee is possible.
  • domain assumption Cocoercivity in expectation (Assumption 3.2).
    This inequality is used in Lemma 4.3 to control the quadratic error terms and in Lemma A.2 to establish square-nonexpansivity in expectation; it is the main structural restriction.
  • domain assumption Deterministic OHM and Dual-OHM attain the optimal O(1/N^2) residual rate (Proposition 3.3).
    Taken from prior literature [29, 22, 55]; it is the starting point for the deterministic error decomposition, and the stochastic analysis reduces to it when sigma = 0.
  • standard math T = I - alpha F is nonexpansive for 0 < alpha <= 2/L when F is 1/L-cocoercive.
    Standard result from monotone operator theory (Bauschke and Combettes), used to formulate the fixed-point reformulation.
  • standard math For mu-strongly monotone and 1/L-cocoercive F, T is gamma-contractive with gamma = sqrt(1 - alpha mu (2 - alpha L)).
    Standard contraction estimate, used in the early-stopping theorem 4.5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Direct Acceleration of Stochastic Root-Finding Without Variance Reduction and Regularization." pith.science (2026). https://pith.science/paper/7NAWZHGP

@misc{pith2026260812043,
  author       = {Pith},
  title        = {Pith review of: Direct Acceleration of Stochastic Root-Finding Without Variance Reduction and Regularization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7NAWZHGP}},
  note         = {Machine review of arXiv:2608.12043}
}
abstract

Acceleration for deterministic root-finding problems has been extensively studied in recent years; specifically, the anchor-based, or Halpern-type methods achieve optimal convergence rates with respect to the operator norm. However, acceleration via these methods does not directly carry over to stochastic setting due to accumulation of errors, unless one enforces diminishing variance via increasing batch sizes or variance reduction techniques. In this work, we show that another class of acceleration, namely the dual-anchor mechanism, extends to the stochastic setting without such error accumulation, in contrast to anchor-based algorithms. Consequently, we cleanly achieve $O(\epsilon^{-3})$ complexity with iteration-independent batch size, without any variance reduction or double-loop recursive regularization, for stochastic root-finding (resp. fixed-point) problems with cocoercivity (resp. square-nonexpansivity) in expectation. For strongly monotone operators, the same algorithm attains a sharper $\widetilde{O} (\epsilon^{-2})$ complexity, nearly matching the lower bound in terms of $\epsilon$-dependence.

Figures

Figures reproduced from arXiv: 2608.12043 by the authors.

Figure 1
Figure 1. Plot of residual norm versus stochastic samples. [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Plot of residual norm versus stochastic samples for SCSC Huber-type minimax problem [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Comparison after independently tuning the constant batch size [PITH_FULL_IMAGE:figures/full_fig_p022_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 39 canonical work pages

  1. [1]

    Moving anchor extragradient methods for smooth structured minimax problems.arXiv:2308.12359, 2023

    James K Alcala, Yat Tin Chow, and Mahesh Sunkula. Moving anchor extragradient methods for smooth structured minimax problems.arXiv:2308.12359, 2023

    arXiv 2023

  2. [2]

    Katyusha: The first direct acceleration of stochastic gradient methods.Journal of Machine Learning Research, 18(221):1–51, 2018

    Zeyuan Allen-Zhu. Katyusha: The first direct acceleration of stochastic gradient methods.Journal of Machine Learning Research, 18(221):1–51, 2018

    2018

  3. [3]

    Bauschke and Patrick L

    Heinz H. Bauschke and Patrick L. Combettes.Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer International Publishing, 2nd edition, 2017

    work page 2017

  4. [4]

    Stochastic gradient descent-ascent: Unified theory and new efficient methods.International Conference on Artificial Intelligence and Statistics, 2023

    Aleksandr Beznosikov, Eduard Gorbunov, Hugo Berard, and Nicolas Loizou. Stochastic gradient descent-ascent: Unified theory and new efficient methods.International Conference on Artificial Intelligence and Statistics, 2023

    2023

  5. [5]

    Bot ¸ and Enis Chenchene

    Radu I. Bot ¸ and Enis Chenchene. Extragradient method with flexible anchoring: Strong conver- gence and fast residual decay.SIAM Journal on Optimization, 36(3):1420–1445, 2026

    work page 2026

  6. [6]

    Fast Optimistic Gradient De- scent Ascent (OGDA) method in continuous and discrete time.Foundations of Computational Mathematics, 2023

    Radu Ioan Bot ¸, Ern¨ o Robert Csetnek, and Dang-Khoa Nguyen. Fast Optimistic Gradient De- scent Ascent (OGDA) method in continuous and discrete time.Foundations of Computational Mathematics, 2023

    work page 2023

  7. [7]

    Stochastic fixed-point iterations for nonexpansive maps: Convergence and error bounds.SIAM Journal on Control and Optimization, 62(1):191–219, 2024

    Mario Bravo and Roberto Cominetti. Stochastic fixed-point iterations for nonexpansive maps: Convergence and error bounds.SIAM Journal on Control and Optimization, 62(1):191–219, 2024

    2024

  8. [8]

    Stochastic Halpern iteration in normed spaces and appli- cations to reinforcement learning.Mathematical Programming, 2026

    Mario Bravo and Juan Pablo Contreras. Stochastic Halpern iteration in normed spaces and appli- cations to reinforcement learning.Mathematical Programming, 2026

    work page 2026

Show all 58 references
  1. [9]

    Stochastic halpern iteration with variance reduction for stochastic monotone inclusions.Neural Information Processing Systems, 2022

    Xufeng Cai, Chaobing Song, Crist´ obal A Guzm´ an, and Jelena Diakonikolas. Stochastic halpern iteration with variance reduction for stochastic monotone inclusions.Neural Information Processing Systems, 2022

    2022

  2. [10]

    Variance reduced halpern iteration for finite-sum monotone inclusions.International Conference on Learning Representations, 2024

    Xufeng Cai, Ahmet Alacaoglu, and Jelena Diakonikolas. Variance reduced halpern iteration for finite-sum monotone inclusions.International Conference on Learning Representations, 2024

    2024

  3. [11]

    Accelerated single-call methods for constrained min-max optimiza- tion.International Conference on Learning Representations, 2023

    Yang Cai and Weiqiang Zheng. Accelerated single-call methods for constrained min-max optimiza- tion.International Conference on Learning Representations, 2023

    2023

  4. [12]

    Near-optimal algorithms for making the gradient small in stochastic minimax optimization.Journal of Machine Learning Research, 25(387):1–44, 2024

    Lesi Chen and Luo Luo. Near-optimal algorithms for making the gradient small in stochastic minimax optimization.Journal of Machine Learning Research, 25(387):1–44, 2024

    2024

  5. [13]

    Training GANs with optimism.International Conference on Learning Representations, 2018

    Constantinos Daskalakis, Andrew Ilyas, Vasilis Syrgkanis, and Haoyang Zeng. Training GANs with optimism.International Conference on Learning Representations, 2018

    2018

  6. [14]

    Variance reduction for root-finding problems.Mathematical Programming, 197(1): 375–410, 2023

    Damek Davis. Variance reduction for root-finding problems.Mathematical Programming, 197(1): 375–410, 2023

    2023

  7. [15]

    First-order methods of smooth convex optimization with inexact oracle.Mathematical Programming, 146(1):37–75, 2014

    Olivier Devolder, Fran¸ cois Glineur, and Yurii Nesterov. First-order methods of smooth convex optimization with inexact oracle.Mathematical Programming, 146(1):37–75, 2014

    2014

  8. [16]

    Halpern iteration for near-optimal and parameter-free monotone inclusion and strong solutions to variational inequalities.Conference on Learning Theory, 2020

    Jelena Diakonikolas. Halpern iteration for near-optimal and parameter-free monotone inclusion and strong solutions to variational inequalities.Conference on Learning Theory, 2020

    2020

  9. [17]

    Springer-Verlag, 2003

    Francisco Facchinei and Jong-Shi Pang.Finite-Dimensional Variational Inequalities and Comple- mentarity Problems. Springer-Verlag, 2003. 11

    2003

  10. [18]

    Accelerated gradient methods for nonconvex nonlinear and stochastic programming.Mathematical Programming, 156(1):59–99, 2016

    Saeed Ghadimi and Guanghui Lan. Accelerated gradient methods for nonconvex nonlinear and stochastic programming.Mathematical Programming, 156(1):59–99, 2016

    2016

  11. [19]

    Generative adversarial nets.Neural Information Processing Systems, 2014

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets.Neural Information Processing Systems, 2014

    2014

  12. [20]

    Stochastic extragradient: General analysis and improved rates.International Conference on Artificial Intelligence and Statis- tics, 2022

    Eduard Gorbunov, Hugo Berard, Gauthier Gidel, and Nicolas Loizou. Stochastic extragradient: General analysis and improved rates.International Conference on Artificial Intelligence and Statis- tics, 2022

    2022

  13. [21]

    Fixed points of nonexpanding maps.Bulletin of the American Mathematical Society, 73(6):957–961, 1967

    Benjamin Halpern. Fixed points of nonexpanding maps.Bulletin of the American Mathematical Society, 73(6):957–961, 1967

    1967

  14. [22]

    Accelerated proximal point method for maximally monotone operators.Mathe- matical Programming, 190(1–2):57–87, 2021

    Donghwan Kim. Accelerated proximal point method for maximally monotone operators.Mathe- matical Programming, 190(1–2):57–87, 2021

    2021

  15. [23]

    G. M. Korpelevich. The extragradient method for finding saddle points and other problems. Ekonomika i Matematicheskie Metody, 12(4):747–756, 1976

    1976

  16. [24]

    M. A. Krasnosel’skii. Two remarks on the method of successive approximations.Uspekhi Matem- aticheskikh Nauk, 10(1):123–127, 1955

    1955

  17. [25]

    Jongmin Lee and Ernest K. Ryu. Accelerating value iteration with anchoring.Neural Information Processing Systems, 2023

    2023

  18. [26]

    Near-optimal sample complexity for MDPs via anchoring.Proceedings of the 42nd international conference on machine learning, 2025

    Jongmin Lee, Mario Bravo, and Roberto Cominetti. Near-optimal sample complexity for MDPs via anchoring.Proceedings of the 42nd international conference on machine learning, 2025

    2025

  19. [27]

    Fast extra gradient methods for smooth structured nonconvex– nonconcave minimax problems.Neural Information Processing Systems, 2021

    Sucheol Lee and Donghwan Kim. Fast extra gradient methods for smooth structured nonconvex– nonconcave minimax problems.Neural Information Processing Systems, 2021

    2021

  20. [28]

    PAGE: A simple and optimal probabilistic gradient estimator for nonconvex optimization.International Conference on Machine Learning, 2021

    Zhize Li, Hongyan Bao, Xiangliang Zhang, and Peter Richtarik. PAGE: A simple and optimal probabilistic gradient estimator for nonconvex optimization.International Conference on Machine Learning, 2021

    2021

  21. [29]

    On the convergence rate of the Halpern-iteration.Optimization Letters, 15(2):405– 418, 2021

    Felix Lieder. On the convergence rate of the Halpern-iteration.Optimization Letters, 15(2):405– 418, 2021

    2021

  22. [30]

    A universal catalyst for first-order optimization

    Hongzhou Lin, Julien Mairal, and Zaid Harchaoui. A universal catalyst for first-order optimization. Neural Information Processing Systems, 2015

    2015

  23. [31]

    Tianyi Lin, Chi Jin, and Michael I. Jordan. Near-optimal algorithms for minimax optimization. Conference on Learning Theory, 125, 2020

    2020

  24. [32]

    Stochastic gradient descent-ascent and consensus optimization for smooth games: Convergence analysis under expected co-coercivity.Neural Information Processing Systems, 2021

    Nicolas Loizou, Hugo Berard, Gauthier Gidel, Ioannis Mitliagkas, and Simon Lacoste-Julien. Stochastic gradient descent-ascent and consensus optimization for smooth games: Convergence analysis under expected co-coercivity.Neural Information Processing Systems, 2021

    2021

  25. [33]

    To- wards deep learning models resistant to adversarial attacks.International Conference on Learning Representations, 2018

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant to adversarial attacks.International Conference on Learning Representations, 2018

    2018

  26. [34]

    Mean value methods in iteration.Proceedings of the American Mathematical Society, 4(3):506–510, 1953

    William Robert Mann. Mean value methods in iteration.Proceedings of the American Mathematical Society, 4(3):506–510, 1953

    1953

  27. [35]

    Foo, Vijay Chandrasekhar, and Georgios Piliouras

    Panayotis Mertikopoulos, Bruno Lecouat, Houssam Zenati, C.-S. Foo, Vijay Chandrasekhar, and Georgios Piliouras. Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile.International Conference on Learning Representations, 2019

    2019

  28. [36]

    George J. Minty. Monotone (nonlinear) operators in Hilbert space.Duke Mathematical Journal, 29(3):341–346, 1962. 12

    1962

  29. [37]

    Cocoercivity, smoothness and bias in variance-reduced stochastic gradient methods.Numerical Algorithms, 91(2):749–772, 2022

    Martin Morin and Pontus Giselsson. Cocoercivity, smoothness and bias in variance-reduced stochastic gradient methods.Numerical Algorithms, 91(2):749–772, 2022

    2022

  30. [38]

    Arkadi Nemirovski. Prox-method with rate of convergenceO(1/t) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems.SIAM Journal on Optimization, 15(1):229–251, 2004

    2004

  31. [39]

    Nguyen, Jie Liu, Katya Scheinberg, and Martin Tak´ aˇ c

    Lam M. Nguyen, Jie Liu, Katya Scheinberg, and Martin Tak´ aˇ c. SARAH: A novel method for ma- chine learning problems using stochastic recursive gradient.International Conference on Machine Learning, 2017

    2017

  32. [40]

    Exact optimal accelerated complexity for fixed-point iterations

    Jisun Park and Ernest K Ryu. Exact optimal accelerated complexity for fixed-point iterations. International Conference on Machine Learning, 2022

    2022

  33. [41]

    Asymptotic Regularity of a Generalised Stochastic Halpern Scheme.Journal of Optimization Theory and Applications, 210(1):3, 2026

    Nicholas Pischke and Thomas Powell. Asymptotic Regularity of a Generalised Stochastic Halpern Scheme.Journal of Optimization Theory and Applications, 210(1):3, 2026

    2026

  34. [42]

    Tyrrell Rockafellar

    R. Tyrrell Rockafellar. Monotone operators and the proximal point algorithm.SIAM Journal on Control and Optimization, 14(5):877–898, 1976

    1976

  35. [43]

    Cam- bridge University Press, 2022

    Ernest K Ryu and Wotao Yin.Large-Scale Convex Optimization via Monotone Operators. Cam- bridge University Press, 2022

    2022

  36. [44]

    Ryu, Kun Yuan, and Wotao Yin

    Ernest K. Ryu, Kun Yuan, and Wotao Yin. ODE analysis of stochastic gradient methods with optimism and anchoring for minimax problems and GANs.arXiv:1905.10899, 2019

    1905 arXiv

  37. [45]

    A first order method for solving convex bilevel optimization problems.SIAM Journal on Optimization, 27(2):640–660, 2017

    Shoham Sabach and Shimrit Shtern. A first order method for solving convex bilevel optimization problems.SIAM Journal on Optimization, 27(2):640–660, 2017

    2017

  38. [46]

    Gesualdo Scutari, Francisco Facchinei, Jong-Shi Pang, and Daniel P. Palomar. Real and complex monotone communication games.IEEE Transactions on Information Theory, 60(7):4197–4231, 2014

    2014

  39. [47]

    Formes bilineaires coercitives sur les ensembles convexes.Comptes Rendus Hebdomadaires Des Seances De L Academie Des Sciences, 258(18):4413, 1964

    Guido Stampacchia. Formes bilineaires coercitives sur les ensembles convexes.Comptes Rendus Hebdomadaires Des Seances De L Academie Des Sciences, 258(18):4413, 1964

    1964

  40. [48]

    Suh, Jisun Park, and Ernest K

    Jaewook J. Suh, Jisun Park, and Ernest K. Ryu. Continuous-time analysis of anchor acceleration. Neural Information Processing Systems, 2023

    2023

  41. [49]

    From Halpern’s fixed-point iterations to Nesterov’s accelerated interpretations for root-finding problems.Computational Optimization and Applications, 87(1):181–218, 2024

    Quoc Tran-Dinh. From Halpern’s fixed-point iterations to Nesterov’s accelerated interpretations for root-finding problems.Computational Optimization and Applications, 87(1):181–218, 2024

    2024

  42. [50]

    Halpern-type accelerated and splitting algorithms for monotone inclusions.arXiv:2110.08150, 2021

    Quoc Tran-Dinh and Yang Luo. Halpern-type accelerated and splitting algorithms for monotone inclusions.arXiv:2110.08150, 2021

    2021 arXiv

  43. [51]

    Approximation of fixed points of nonexpansive mappings.Archiv der Mathe- matik, 58(5):486–491, 1992

    Rainer Wittmann. Approximation of fixed points of nonexpansive mappings.Archiv der Mathe- matik, 58(5):486–491, 1992

    1992

  44. [52]

    A Theory of Composition and Duality of Extremal Optimal Fixed-Point Algorithms.arXiv:2605.02231, 2026

    TaeHo Yoon and Benjamin Grimmer. A Theory of Composition and Duality of Extremal Optimal Fixed-Point Algorithms.arXiv:2605.02231, 2026

    2026 arXiv

  45. [53]

    TaeHo Yoon and Ernest K. Ryu. Accelerated algorithms for smooth convex-concave minimax problems withO(1/k 2) rate on squared gradient norm.International Conference on Machine Learning, 2021

    2021

  46. [54]

    TaeHo Yoon and Ernest K. Ryu. Accelerated minimax algorithms flock together.SIAM Journal on Optimization, 35(1):180–209, 2025

    2025

  47. [55]

    Suh, and Ernest K

    TaeHo Yoon, Jaeyeon Kim, Jaewook J. Suh, and Ernest K. Ryu. Optimal acceleration for minimax and fixed-point problems is not unique.International Conference on Machine Learning, 2024. 13

    2024

  48. [56]

    H-invariance theory: A complete charac- terization of minimax optimal fixed-point algorithms.Accepted for publication in Mathematical Programming, 2025

    TaeHo Yoon, Ernest K Ryu, and Benjamin Grimmer. H-invariance theory: A complete charac- terization of minimax optimal fixed-point algorithms.Accepted for publication in Mathematical Programming, 2025

    2025

  49. [57]

    Multiplayer federated learning: Reaching equilibrium with less communication.Advances in Neural Information Processing Systems, 2026

    TaeHo Yoon, Sayantan Choudhury, and Nicolas Loizou. Multiplayer federated learning: Reaching equilibrium with less communication.Advances in Neural Information Processing Systems, 2026

    2026

  50. [58]

    Siqi Zhang, Sayantan Choudhury, Sebastian Stich, and Nicolas Loizou. Communication-efficient gradient descent-accent methods for distributed variational inequalities: Unified analysis and local updates.International Conference on Learning Representations, 2024. 14 Appendix Con...

    2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.