Variance-Reduced Fast Operator Splitting Methods for Generalized Equations

Quoc Tran-Dinh

REVIEW 3 major objections 4 minor 120 references

Variance-Reduced Fast Operator Splitting Methods for Generalized Equations

T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that variance-reduced Nesterov acceleration, applied through forward-backward and backward-forward splitting, drives the squared FBS residual of a generalized equation to zero at $o(1/k^2)$ in expectation and almost…

desk verdict Genuine framework paper with solid finite-sum theory; the expectation-setting o(1/k^2) and a.s. claims overreach because the stated mega-batch schedules make condition (39) fail. read the letter →

arxiv 2504.13046 v3 pith:3VEID65L submitted 2025-04-17 math.OC stat.ML

Quoc Tran-Dinh This is my paper · ORCID

classification math.OCstat.ML MSC 90C2547H0565K1090C33

keywords generalizedequationsoperatorsplittingvariancereductionNesterovaccelerationco-hypomonotonicityforward-backwardbackward-forwardfinite-sumoptimization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper develops two single-loop, accelerated operator-splitting algorithms for generalized equations—inclusions of the form $0 \in F x + T x$ that cover composite minimization, minimax problems, variational inequalities, and fixed-point problems. It claims that, when $F$ is co-coercive and $T$ is maximally $\rho$-co-hypomonotone with $L\rho<1$, the expected squared norm of the forward-backward splitting residual $G_\lambda x^k$ decreases as $O(1/k^2)$ and actually as $o(1/k^2)$, with almost sure convergence of the iterates to a solution. The framework also accommodates both unbiased and biased variance-reduced estimators—SVRG, SAGA, SARAH, and Hybrid-SGD—and derives oracle complexity bounds that match the best known for such estimators. If correct, this gives accelerated, variance-reduced splitting methods for a broader class of problems, including some nonmonotone saddle-point models, in a single-loop algorithm without restarts or catalyst steps.

What carries the argument

The load-bearing object is the forward-backward splitting residual $G_\lambda x := \lambda^{-1}(x - J_{\lambda T}(x-\lambda F x))$, which turns the inclusion $0\in F x + T x$ into the equation $G_\lambda x=0$. Lemma 3 shows that when $F$ is $1/L$-co-coercive and $T$ is maximally $\rho$-co-hypomonotone with $L\rho<1$, $G_\lambda$ is strongly co-coercive in the sense of (10), with explicit constants $\bar\beta$ and $\Lambda$. That inequality feeds a Lyapunov function $P_k$ that combines the residual norm, a cross term between the residual and the anchoring variable, and the variance-reduced estimator error $\Delta_k$, whose recursion is captured by the estimator class of Definition 4. The same machinery is repeated for the backward-forward residual $S_\lambda$.

What would settle it

Run VFOSA+ on a monotone but not co-coercive instance, for example $Fx = [L^{\top}v; -Lu]$ with $T$ the normal cone of simplexes and no regularizer, and monitor $\mathbb{E}[\|G_\lambda x^k\|^2]$. If the residual still decays like $o(1/k^2)$, then Assumption 1.2 is not necessary; if it stalls at a positive floor, co-coercivity is doing the work. More directly, numerically evaluate inequality (10) for such an $F$ and check whether the claimed constants $\bar\beta$ and $\Lambda$ make the right-hand side meaningful; if the inequality fails, the Lyapunov descent behind Theorems 12–14 collapses.

Watch

Extended reading notes

Core claim

On its own terms, the central discovery is a rate: under Assumptions 1.1 and 1.2 with $L\rho<1$, the variance-reduced fast forward-backward splitting method (VFOSA+) produces iterates whose FBS residual satisfies $\mathbb{E}[\|G_\lambda x^k\|^2] \le 2(\Psi_0^2+E_0^2+B_\infty)/(\mu^2(k+r-1)^2)$, together with $o(1/k^2)$ rates in expectation and almost surely, and almost sure convergence of $(x^k,z^k)$ to a zero of $\Phi=F+T$. The paper proves the same statements for a backward-forward variant (VFOSA$^-$) with respect to the BFS residual. It also specifies the algorithm for four concrete estimators and shows the finite-sum oracle complexities $\tilde O(n+n^{2/3}\epsilon^{-1})$ for SVRG/SAGA and $\tilde O(n+n^{1/2}\epsilon^{-1})$ for SARAH, and $O(\epsilon^{-3})$ in the expectation setting, matching the best-known bounds without added acceleration tricks.

Load-bearing premise

The smooth part $F$ must be co-coercive on average or in expectation (Assumption 1.2), not merely monotone and Lipschitz, so skew-symmetric linear maps such as bilinear games violate the assumption unless one adds a small regularizer.

Editorial extensions

If this is right

The SVRG and SAGA variants need $\tilde O(n + n^{2/3}\epsilon^{-1})$ evaluations of $F_i$ and resolvents to reach $\mathbb{E}[\|G_\lambda x^K\|^2]\le \epsilon^2$, while the SARAH variant needs $\tilde O(n+n^{1/2}\epsilon^{-1})$; the expectation-setting variants need $O(\epsilon^{-3})$.
The same single-loop algorithm converges almost surely, with iterates landing on a solution of the generalized equation rather than only on a stationary point of some merit function.
Because $T$ is only required to be co-hypomonotone, the result covers nonmonotone operators; the experiments include a robust logistic regression model with a nonconvex SCAD regularizer, where $T$ is locally co-hypomonotone.
The backward-forward variant gives the same rates and complexity, which is useful when the solver wants to preserve the finite-sum structure of $F$ after composition with the resolvent.
For convex composite minimization, VFOSA+ reduces to a new accelerated proximal-gradient-type scheme whose residual is evaluated at $x^k$ rather than at the extrapolated point $y^k$.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural testable extension is to replace co-coercivity by plain Lipschitz monotonicity; the current proof would break at Lemma 3, and the paper's own Policeman-vs-Burglar experiment adds a small regularizer precisely because the skew-symmetric operator is not co-coercive.
The remark that local co-hypomonotonicity may suffice suggests the rates could hold when co-hypomonotonicity holds only near solutions; verifying this would widen the reach to nonconvex minimax models beyond the examples tested.
Because the estimator class of Definition 4 is defined by a two-line variance recursion, other estimators such as SAG, SEGA, or JacSketch could plausibly be plugged in and inherit the same rates, though the paper only proves the bounds for four estimators.
The residual metric $\mathbb{E}\|G_\lambda x^k\|^2$ doubles as a stopping criterion; if future work extends the methods to extragradient updates, the same Lyapunov template might weaken the co-coercivity assumption.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The short version: this is a substantial paper and the core of it holds up. The one thing to know before citing the expectation-setting results is that the o(1/k^2) and almost-sure claims require B∞ < ∞ (condition (39)), and the fixed mega-batch schedules in Corollaries 22–24 do not deliver it. The finite-horizon O(1/k^2) rates and the ε-complexity claims are not affected.

What is actually new: Definition 4 is a clean general estimator class that covers SVRG, SAGA, SARAH, and Hybrid-SGD under a single variance-reduction property, and VFOSA+/VFOSA− are the first frameworks I know of that combine biased variance-reduced estimators with Nesterov-type acceleration and co-hypomonotone T (which need not be monotone). The Lyapunov analysis in Section 4 is the real contribution; Lemma 3's strong co-coercivity of the FBS residual is the load-bearing piece, and the appendix proofs are internally coherent and self-contained. The finite-sum results are genuinely clean: with full-batch snapshots σ_k = 0, so B∞ = 0 and the o(1/k^2), summability, a.s., and iterate-convergence results all go through via Corollary 21. The numerics are honest—real datasets, several baselines, a SCAD-regularized nonmonotone instance, and the Policeman-Burglar example with the regularizer built into the model.

Soft spots, in proportion. The main one is the expectation setting. The stress-test note is correct: with fixed n_k = Θ(ε−3) and p_k bounded below by a positive ε-dependent constant (p_k ≥ 2ε in Corollary 22), σ_k^2/Θ_k ~ p_k^2/n stays bounded below by a positive constant, so the summand in (39) grows like k^2 and B∞ = +∞. Theorems 13–14 therefore do not apply to the algorithms as instantiated in Section 4.5, and the abstract's blanket o(1/k^2)/a.s. claim overstates what is proven. Remark 15 acknowledges that growing mega-batches n_k = O(k^{3+ω}) are needed for B∞ < ∞, but with constant p_k the expected oracle cost becomes Ω(ε−(4+ω)), well off the claimed Θ(ε−3). This is fixable—restrict the infinite-horizon claims to the finite-sum setting, or accept the larger complexity—but it should be stated precisely. Minor items: the "best-known" complexity wording holds only up to the log factors the paper itself discloses in a footnote, and Assumption 1.2 excludes skew-symmetric operators (bilinear games), which Section 6.2 papers over with an ε = 10−8 regularizer. Both are acknowledged to the author's credit.

Bottom line: for anyone working on variance-reduced operator splitting or nonmonotone inclusions, this paper deserves a serious referee. I'd send it out; the revision should reconcile (39) with the Section 4.5 parameter choices and align the abstract with what is proven.

Referee Report

3 major / 4 minor

Summary. The paper develops two variance-reduced accelerated operator-splitting frameworks, VFOSA+ (forward-backward) and VFOSA- (backward-forward), for solving generalized equations 0 in Fx + Tx under average/expectation co-coercivity of F and maximal rho-co-hypomonotonicity of T. It introduces a unified class of variance-reduced estimators (Definition 4) that covers SVRG, SAGA, SARAH, and Hybrid-SGD, and proves O(1/k^2) and o(1/k^2) rates on the expected squared FBS residual, almost sure o(1/k^2) rates, and almost sure convergence of iterates. The paper also derives oracle complexity bounds in the finite-sum and expectation settings and reports numerical experiments on robust logistic regression, a robust minimax problem, and a Policeman-vs-Burglar game.

Significance. If the stated results hold, this is a substantial contribution: it provides a unified single-loop accelerated splitting framework with variance reduction for a class of generalized equations that includes nonmonotone co-hypomonotone operators, and it matches the best-known oracle complexity bounds in several settings. The Lyapunov analysis in the appendix is detailed and internally structured, and the estimator class in Definition 4 is broad enough to be of independent interest. The numerical experiments are extensive and compare favorably with recent methods. The main weakness is that the infinite-horizon o(1/k^2)/almost-sure claims in the expectation setting are conditioned on B_infinity < infinity, while the concrete parameter schedules given for the expectation-setting variants do not satisfy that condition and no alternative schedule achieving both the condition and the claimed O(epsilon^{-3}) complexity is supplied.

major comments (3)

[Section 4.5, Eq. (39), Corollaries 22-24] The expectation-setting variants as implemented in Corollaries 22-24 do not satisfy the condition (39) required by Theorems 13 and 14. With t_k = mu(k+r), Eq. (39) requires B_infinity = (Lambda/beta) sum_k t_{k-1}(t_{k-1}-1) sigma_k^2/Theta_k < infinity. In Corollary 22 (L-SVRG) one has p_k >= 2epsilon, b_k = b = Theta(epsilon^{-2}), n_k = n = Theta(epsilon^{-3}), and by Lemma 5 sigma_k^2 = p_k sigma^2/n and Theta_k = 4/(b_k p_k). Hence sigma_k^2/Theta_k = b_k p_k^2 sigma^2/(4n), which is bounded below by a positive constant times epsilon^3 for large k, so the summand is at least c k^2 epsilon^3 and B_infinity diverges. Corollary 23 (L-SARAH) gives the same divergence with sigma_k^2/Theta_k of order epsilon^4, and Corollary 24 (HSGD) uses fixed batches with tau_k -> 1 - sqrt(1-epsilon), which again makes B_infinity infinite. Therefore Theorems 13 and 14, both of which are explicitly conditioned on (39), do not apply to the expectation-setting algorithms under the stated parameter choices. The finite-horizon bound in Theorem 12 and the O(epsilon^{-3}) complexity statements derived from it are not affected, but the o(1/k^2) expected rates, the almost sure o(1/k^2) rates, and the almost sure iterate convergence in the expectation setting are not established by the provided schedules.
[Section 4.5, Remark 15, and Table 1] The manuscript does not provide any expectation-setting parameter schedule for which both (39) holds and the claimed O(epsilon^{-3}) oracle complexity is achieved. Remark 15 suggests increasing mega-batch sizes n_k = O(k^{3+omega}) for SVRG and SARAH, but under the constant reset probabilities p_k >= c epsilon used in Corollaries 22-23 the expected mega-batch oracle cost becomes Omega(sum_k k^{3+omega}) = Omega(epsilon^{-(4+omega)}), which is far worse than the claimed O(epsilon^{-3}). The HSGD schedule in Corollary 24 similarly keeps fixed batch sizes and a reset weight tau_k of order epsilon, so B_infinity diverges. Since Corollary 21 establishes B_infinity = 0 only for the finite-sum setting, the expectation-setting entries in Table 1 and the corresponding abstract claims overstate what is proven. The authors should either provide a schedule satisfying (39) with the claimed complexity, or explicitly restrict the o(1/k^2) and almost-sure claims in the expectation setting.
[Section 5, Theorems 25-27] The backward-forward splitting results inherit the same gap. Theorem 25's bound (47) and Theorems 26-27 all depend on the same B_K / B_infinity quantity from Theorem 12, and the proof of Theorem 25 explicitly reduces to the VFOSA+ analysis. Since no expectation-setting schedule satisfying (39) is supplied for the concrete estimators, the advertised o(1/k^2) and almost-sure convergence properties of VFOSA- in the expectation setting rest on the same unsupported condition. The finite-horizon complexity statements for VFOSA- are not in question, but the infinite-horizon claims should be corrected or supplemented with a valid schedule.

minor comments (4)

[Section 2.3(c), near Eq. (8)] The sentence discussing preservation of the finite-sum/expectation structure appears to refer to G_lambda in Eq. (6) and S_lambda in Eq. (8), but the text says "G_lambda in (8)". This should be corrected for consistency.
[Appendix B.1, proof of Lemma 5] The phrase "setting tau = 0 in (14)" is formally problematic because Eq. (14) contains the coefficient (1+tau)/tau. The full-batch case should be obtained as the limit tau -> 0 or by a separate argument, not by direct substitution.
[Throughout the appendix and Section 4.5] There are several typographical errors, e.g., "Subections" in the heading of Section D, "Appendicies" in the proof of Corollary 22, and "these methods were though derived" in Section 4.1(c). These should be corrected in a revision.
[Section 6.2] The Policeman-vs-Burglar experiment adds a 10^{-8} regularizer to make the skew-symmetric operator average co-coercive. This changes the problem, and the limitation should be acknowledged more explicitly, especially because Section 2.2(a) suggests reformulations as an alternative route.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the convergence analysis is self-contained and the rates are derived, not imported from a fit or from a load-bearing self-citation.

full rationale

The paper's central claims are derived from explicit assumptions through a self-contained Lyapunov analysis. Lemma 3 proves the key co-coercivity-like inequalities (10)-(11) from Assumptions 1.1-1.2, and Theorems 12-14 build on Lemmas 9-11 and the estimator bounds of Lemmas 5-8, all proved in the appendices. No parameter is fitted to the target residual, and no theorem is obtained by renaming an existing result: the O(1/k^2), o(1/k^2), summability, and almost-sure statements are obtained by telescoping the Lyapunov function P_k with conditions (35) and (39). Self-citations to Tran-Dinh (2024a,b, 2025) are used for context, comparison, or motivating reformulations, not as the justification of the main convergence theorem; the one external convergence lemma, Davis (2022, Proposition 4.1), is a standard demiclosedness/iterate-convergence principle and is not the source of the rates. The parameter schedules in Corollaries 22-24 support finite-horizon complexity via Theorem 12, while the infinite-horizon B_infinity < infinity condition is explicitly flagged in Remark 15 as requiring increasing mega-batches; whether the fixed mega-batch schedules satisfy the hypotheses of Theorems 13-14 is a correctness/parameterization concern, not circularity.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claims rest on two structural assumptions: solvability and bounded variance of the stochastic oracle (Assumption 1.1), and the 1/L co-coercivity (in average or expectation) of F (Assumption 1.2). The paper proves its key residual co-coercivity result (Lemma 3) from these, so no hidden fitted constants are smuggled in. The algorithmic parameters μ, r, ν, β, λ and Γ_k are chosen by hand within theorem ranges; they are not tuned to data. No new physical entities are introduced.

free parameters (6)

μ (acceleration parameter) = 0.95·2/3 (experiments); theory requires 0<μ<2/3
Appears in the update t_k=μ(k+r) and the rates; chosen by hand to satisfy 0<μ<2/3.
r (shift parameter) = 2+1/μ
Theorem requires r ≥ 2+1/μ; set to the lower bound in experiments.
ν (mixing parameter) = μ/2
Set by Theorem 12; controls the z-update.
β (step-size parameter) = (2-μ)̄β/(2+μ)
Upper bound from Theorem 12; chosen at the boundary in experiments.
λ (FBS step-size) = 1/(2L) in experiments; must satisfy 2ρ ≤ λ < 2(1+√(1-̂Lρ))/̂L
Step size in the forward-backward residual; depends on estimates of L and ρ.
Γ_k (Lyapunov scaling) = e.g., 5 c_p β n^ω/μ (SVRG); varies per estimator
Chosen to satisfy condition (35) and (43); affects the Lyapunov function.

assumptions (6)

domain assumption zer(Φ) ≠ ∅ (Assumption 1.1(i))
Guarantees the generalized equation has a solution; standard, but without it the rates target the empty set.
domain assumption Bounded variance E||F(x,ξ)-Fx||^2 ≤ σ^2 (Assumption 1.1(ii))
Limits stochastic noise; used in the complexity bounds for the expectation setting.
domain assumption T maximally ρ-co-hypomonotone (Assumption 1.1(iii))
Weakens monotonicity; enables nonmonotone examples but still requires the resolvent to be well-behaved.
domain assumption F is 1/L-average co-coercive (finite-sum) or 1/L-co-coercive in expectation (Assumption 1.2)
The load-bearing smoothness condition; used to prove Lemma 3 and all main rates. Fails for general monotone Lipschitz operators.
standard math Robbins-Siegmund supermartingale theorem
Used in the almost sure convergence proofs (Appendix C.6).
standard math Davis (2022) Proposition 4.1 (demiclosedness criterion)
Imported to convert a.s. residual convergence into a.s. iterate convergence.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Variance-Reduced Fast Operator Splitting Methods for Generalized Equations." pith.science (2026). https://pith.science/paper/3VEID65L

@misc{pith2026250413046,
  author       = {Pith},
  title        = {Pith review of: Variance-Reduced Fast Operator Splitting Methods for Generalized Equations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3VEID65L}},
  note         = {Machine review of arXiv:2504.13046}
}

abstract

We develop two variance-reduced fast operator splitting methods to approximate solutions of a class of generalized equations, covering fundamental problems such as \rvs{minimization}, minimax problems, and variational inequalities as special cases. Our approach integrates recent advances in accelerated operator splitting and fixed-point methods, co-hypomonotonicity, and variance reduction. First, we introduce a class of variance-reduced estimators and establish their variance-reduction bounds. This class includes both unbiased and biased instances and comprises common estimators as special cases, including SVRG, SAGA, SARAH, and Hybrid-SGD. Second, we design a novel accelerated variance-reduced forward-backward splitting (FBS) method using these estimators to solve generalized equations in both finite-sum and expectation settings. Our algorithm achieves both $\mathcal{O}(1/k^2)$ and $o(1/k^2)$ convergence rates on the expected squared norm $\mathbb{E}[ \| G_{\lambda}x^k\|^2]$ of the FBS residual $G_{\lambda}$, where $k$ is the iteration counter. Additionally, we establish almost sure convergence rates and the almost sure convergence of iterates to a solution of the underlying generalized equation. Unlike existing stochastic operator splitting algorithms, our methods accommodate co-hypomonotone operators, which can include nonmonotone problems arising in recent applications. Third, we specify our method for each concrete estimator mentioned above and derive the corresponding oracle complexity, demonstrating that these variants achieve the best-known oracle complexity bounds without requiring additional enhancement techniques. Fourth, we develop a variance-reduced fast backward-forward splitting (BFS) method, which attains similar convergence results and oracle complexity bounds as our FBS-based algorithm.

Figures

Figures reproduced from arXiv: 2504.13046 by the authors.

**Figure 1.** The performance of 4 variants of VFOSA+: L-SVRG, SAGA, L-SARAH, and HSGD for solving (49) with the ℓ1-regularizer on two datasets: w8a and gisette. 0 25 50 75 100 125 150 175 200 Number of epochs 10−4 10−3 10−2 10−1 100 Relative Norm kG λ x kk/kG λ x 0k mnist Dataset: (n, p1, p2) = (60000, 791, 10) VFOSA+-Svrg VFOSA+-Saga VFOSA+-Sarah VFOSA+-Hsgd 0 25 50 75 100 125 150 175 200 Number of epochs 10−4 10−3 10−2 10−1 10… view at source ↗

**Figure 2.** The performance of 4 variants of VFOSA+: L-SVRG, SAGA, L-SARAH, and HSGD for solving (49) using the ℓ1-regularizer on two datasets: mnist and a9a. Again, we observe a similar performance as in the first experiment. Both VFOSA+-Sarah and VFOSA+-Hsgd still outperform VFOSA+-Svrg and VFOSA+-Saga. Overall, VFOSA+-Hsg is still the best in this experiment. (b) Comparing 4 variants of VFOSA+ on nonmonotone problems. Next, … view at source ↗

**Figure 3.** The performance of 4 variants of VFOSA+: SVRG, SAGA, SARAH, and HSGD for solving (49) using the SCAD regularizer on two datasets: w8a and gisette. 0 25 50 75 100 125 150 175 200 Number of epochs 10−4 10−3 10−2 10−1 100 Relative Norm kG λ x kk/kG λ x 0k mnist Dataset: (n, p1, p2) = (60000, 791, 10) VFOSA+-Svrg VFOSA+-Saga VFOSA+-Sarah VFOSA+-Hsgd 0 25 50 75 100 125 150 175 200 Number of epochs 10−4 10−3 10−2 10−1 100… view at source ↗

Figures from the paper (7 more)

**Figure 4.** Figure 4: The performance of 4 variants of VFOSA+: L-SVRG, SAGA, L-SARAH, and HSGD for solving (49) using the SCAD regularizer on the mnist and a9a datasets. the SCAD regularizer (nonmonotone case). The results of this experiment are presented in Figures 5 and 6, respectively. W…

**Figure 5.** Figure 5: The performance of 4 variants of VFOSA−: L-SVRG, SAGA, L-SARAH, and HSGD for solving (49) with the ℓ1-norm regularizer on mnist and gisette. 0 25 50 75 100 125 150 175 200 Number of epochs 10−4 10−3 10−2 10−1 100 Relative Norm kG λ x kk/kG λ x 0k mnist Dataset: (n, p1,…

**Figure 6.** Figure 6: The performance of 4 variants of VFOSA−: L-SVRG, SAGA, L-SARAH, and HSGD for solving (49) with the SCAD regularizer on mnist and gisette. 0 25 50 75 100 125 150 175 200 Number of epochs 10−4 10−3 10−2 10−1 100 Relative operator norm kG λ x kk/kG λ x 0k The mnist Datase…

**Figure 7.** Figure 7: The performance of 7 algorithms: 3 variants of each VFOSA+ and VFOSA−, and VrHalpern using the ℓ1-regularizer on two datasets: mnist and gisette. 30 [PITH_FULL_IMAGE:figures/full_fig_p030_7.png]

**Figure 8.** Figure 8: The performance of 7 algorithms: 3 variants of each VFOSA+ and VFOSA−, and VrHalpern using the SCAD regularizer on 2 datasets: mnist and gisette. As observed in both Figures 7 and 8, our L-SARAH and HSGD variants perform better than L-SVRG and VrHalpern on the mnist an…

**Figure 9.** Figure 9: The performance of 2 variants of VFOSA+ and 5 competitors for solving (50) using theoretical parameters. The average of 10 problems in each experiment. VFOSA+-Svrg and VrHalpern. However, our VFOSA+-Sarah achieves the best performance, attaining the lowest relative res…

**Figure 10.** Figure 10: The performance of 2 variants of VFOSA+ and 5 competitors for solving (50) using smaller pk and bk. The average of 10 problems in each experiment. As shown in [PITH_FULL_IMAGE:figures/full_fig_p033_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

120 extracted references · 63 canonical work pages

[1]

Adly and H

S. Adly and H. Attouch. First-order inertial algorithms involving dry friction damping. Math. Program., pages 1--41, 2021

2021
[2]

R. P. Agarwal, M. Meehan, and D. O'regan. Fixed point theory and applications, volume 141. Cambridge university press, 2001

2001
[3]

Alacaoglu and Y

A. Alacaoglu and Y. Malitsky. Stochastic variance reduction for variational inequality methods. In Conference on Learning Theory, pages 778--816. PMLR, 2022

2022
[4]

Alacaoglu, Y

A. Alacaoglu, Y. Malitsky, and V. Cevher. Forward-reflected-backward method with variance reduction. Comput. Optim. Appl., 80 0 (2): 0 321--346, 2021

2021
[5]

J. K. Alcala, Y. T. Chow, and M. Sunkula. Moving anchor extragradient methods for smooth structured minimax problems. arXiv preprint arXiv:2308.12359, 2023

arXiv 2023
[6]

Arjovsky, S

M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein generative adversarial networks. In International Conference on Machine Learning, pages 214--223, 2017

2017
[7]

Attouch and A

H. Attouch and A. Cabot. Convergence of a relaxed inertial proximal algorithm for maximally monotone operators. Math. Program., 184 0 (1): 0 243--287, 2020

2020
[8]

Attouch and J

H. Attouch and J. Fadili. From the R avine method to the N esterov method and vice versa: A dynamical system perspective. SIAM J. Optim., 32 0 (3): 0 2074--2101, 2022

2022

Show all 120 references

[9]

Attouch and J

H. Attouch and J. Peypouquet. Convergence of inertial dynamics and proximal algorithms governed by maximally monotone operators. Math. Program., 174 0 (1-2): 0 391--432, 2019

2019
[10]

Attouch, J

H. Attouch, J. Peypouquet, and P. Redont. Backward--forward algorithms for structured monotone inclusions in H ilbert spaces. J. Math. Anal. Appl., 457 0 (2): 0 1095--1117, 2018

2018
[11]

H. H. Bauschke and P. Combettes. Convex analysis and monotone operators theory in H ilbert spaces . Springer-Verlag, 2nd edition, 2017

2017
[12]

H. H. Bauschke, W. M. Moursi, and X. Wang. Generalized monotone operators and their averaged resolvents. Math. Program., pages 1--20, 2020

2020
[13]

Beck and M

A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci., 2 0 (1): 0 183--202, 2009

2009
[14]

Ben-Tal, T

A. Ben-Tal, T. Margalit, and A. Nemirovski. The ordered subsets mirror descent optimization method with applications to tomography. SIAM J. Optim., 12: 0 79--108, 2001

2001
[15]

Beznosikov, E

A. Beznosikov, E. Gorbunov, H. Berard, and N. Loizou. Stochastic gradient descent-ascent: Unified theory and new efficient methods. In International Conference on Artificial Intelligence and Statistics, pages 172--235. PMLR, 2023

2023
[16]

A. Bohm, M. Sedlmayer, R. E. Csetnek, and R. I. Bot. Two steps at a time--- T aking GAN training in stride with T seng's method. SIAM Journal on Mathematics of Data Science, 4 0 (2): 0 750--771, 2022

2022
[17]

R. I. Bot and D. K. Nguyen. Fast K rasnosel\'skii- M ann algorithm with a convergence rate of the fixed point iteration of o(1/k) . arXiv preprint arXiv:2206.09462, 2022

2022 arXiv
[18]

R. I. Bot, P. Mertikopoulos, M. Staudigl, and P. T. Vuong. Forward-backward-forward methods with variance reduction for stochastic variational inequalities. arXiv preprint arXiv:1902.03355, 2019

1902 arXiv
[19]

R. I. Bo t , E. Chenchene, and J. M. Fadili. Generalized fast krasnoselskii-mann method with preconditioners. arXiv preprint arXiv:2411.18574, 2024

2024
[20]

Bottou, F

L. Bottou, F. E. Curtis, and J. Nocedal. O ptimization M ethods for L arge- S cale M achine L earning. SIAM Rev., 60 0 (2): 0 223--311, 2018

2018
[21]

R. S. Burachik and A. Iusem. Set-Valued Mappings and Enlargements of Monotone Operators. New York: Springer, 2008

2008
[22]

X. Cai, C. Song, C. Guzm\' a n, and J. Diakonikolas. A stochastic H alpern iteration with variance reduction for stochastic monotone inclusion problems. In Proceedings of the 12th International Conference on Learning Representations (ICLR 2022), 2022 a . URL https://openreview...

2022
[23]

X. Cai, A. Alacaoglu, and J. Diakonikolas. Variance reduced halpern iteration for finite-sum monotone inclusions. In The 12th International Conference on Learning Representations (ICLR), pages 1--33, 2024

2024
[24]

Cai and W

Y. Cai and W. Zheng. A ccelerated S ingle- C all M ethods for C onstrained M in- M ax O ptimization. In The 11th International Conference on Learning Representations, ICLR 2023. The Eleventh International Conference on Learning Representations, ICLR 2023, 2023

2023
[25]

Y. Cai, A. Oikonomou, and W. Zheng. Accelerated algorithms for monotone inclusions and constrained nonconvex-nonconcave min-max optimization. OPT 2022: Optimization for Machine Learning (NeurIPS 2022 Workshop), 2022 b

2022
[26]

Carmon, Y

Y. Carmon, Y. Jin, A. Sidford, and K. Tian. Variance reduction for matrix games. Advances in Neural Information Processing Systems, 32, 2019

2019
[27]

Chang and C.-J

C.-C. Chang and C.-J. Lin. LIBSVM : A library for S upport V ector M achines. ACM Transactions on Intelligent Systems and Technology, 2: 0 27:1--27:27, 2011

2011
[28]

Chavdarova, G

T. Chavdarova, G. Gidel, F. Fleuret, and S. Lacoste-Julien. Reducing noise in gan training with variance reduced extragradient. Advances in Neural Information Processing Systems, 32: 0 393--403, 2019

2019
[29]

Y. Chen, G. Lan, and Y. Ouyang. Accelerated schemes for a class of variational inequalities. Math. Program., 165 0 (1): 0 113--149, 2017

2017
[30]

Combettes and J.-C

P. Combettes and J.-C. Pesquet. F ixed- P oint A lgorithms for I nverse P roblems in S cience and E ngineering , chapter P roximal S plitting M ethods in S ignal P rocessing, pages 185--212. Springer-Velarg, 2011

2011
[31]

Condat and P

L. Condat and P. Richt \'a rik. Murana : A generic framework for stochastic variance-reduced optimization. In Mathematical and Scientific Machine Learning, pages 81--96. PMLR, 2022

2022
[32]

Cui and U

S. Cui and U. Shanbhag. On the analysis of variance-reduced and randomized projection variants of single projection schemes for monotone stochastic variational inequality problems. Set-Valued and Variational Analysis, 29 0 (2): 0 453--499, 2021

2021
[33]

Cutkosky and F

A. Cutkosky and F. Orabona. Momentum-based variance reduction in non-convex SGD . In Advances in Neural Information Processing Systems, pages 15210--15219, 2019

2019
[34]

Daskalakis, A

C. Daskalakis, A. Ilyas, V. Syrgkanis, and H. Zeng. Training GANs with O ptimism. In International Conference on Learning Representations (ICLR 2018), 2018

2018
[35]

D. Davis. SMART : T he stochastic monotone aggregated root-finding algorithm. arXiv preprint arXiv:1601.00698, 2016

2016 arXiv
[36]

D. Davis. Variance reduction for root-finding problems. Math. Program., pages 1--36, 2022

2022
[37]

Defazio, F

A. Defazio, F. Bach, and S. Lacoste-Julien. SAGA : A fast incremental gradient method with support for non-strongly convex composite objectives. In Advances in Neural Information Processing Systems (NIPS), pages 1646--1654, 2014

2014
[38]

Demidovich, G

Y. Demidovich, G. Malinovsky, I. Sokolov, and P. Richt \'a rik. A guide through the zoo of biased SGD . Advances in Neural Information Processing Systems, 36: 0 23158--23171, 2023

2023
[39]

Diakonikolas

J. Diakonikolas. H alpern iteration for near-optimal and parameter-free monotone inclusion and strong solutions to variational inequalities. In Conference on Learning Theory, pages 1428--1451. PMLR, 2020

2020
[40]

Driggs, M

D. Driggs, M. J. Ehrhardt, and C.-B. Sch \"o nlieb. Accelerating variance-reduced stochastic gradient methods. Math. Program., (online first): 0 1--45, 2020

2020
[41]

Driggs, J

D. Driggs, J. Liang, and C.-B. Sch \"o nlieb. On biased stochastic gradient estimation. Journal of Machine Learning Research, 23 0 (24): 0 1--43, 2022

2022
[42]

W. Du, D. Xu, X. Wu, and H. Tong. Fairness-aware agnostic federated learning. In Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), pages 181--189. SIAM, 2021

2021
[43]

R. Durrett. Probability: theory and examples, volume 49. Cambridge university press, 2019

2019
[44]

Emmanouilidis, R

K. Emmanouilidis, R. Vidal, and N. Loizou. Stochastic extragradient with random reshuffling: Improved convergence for variational inequalities. In International Conference on Artificial Intelligence and Statistics, pages 3682--3690. PMLR, 2024

2024
[45]

Evens, P

B. Evens, P. Pas, P. Latafat, and P. Patrinos. Convergence of the preconditioned proximal point method and D ouglas- R achford splitting in the absence of monotonicity. arXiv preprint arXiv:2305.03605, 2023

2023 arXiv
[46]

Facchinei and J.-S

F. Facchinei and J.-S. Pang. Finite-dimensional variational inequalities and complementarity problems, volume 1-2. Springer-Verlag, 2003

2003
[47]

Faghri, C

F. Faghri, C. N. Vasconcelos, D. J. Fleet, F. Pedregosa, and N. L. Roux. Bridging the gap between adversarial robustness and optimization bias. ICLR, 2025

2025
[48]

Fan and R

J. Fan and R. Li. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American statistical Association, 96 0 (456): 0 1348--1360, 2001

2001
[49]

Friedman, T

J. Friedman, T. Hastie, and R. Tibshirani. The elements of statistical learning, volume 1. Springer-Verlag, New York, 2001

2001
[50]

Goodfellow, J

I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672--2680, 2014

2014
[51]

Gorbunov, F

E. Gorbunov, F. Hanzely, and P. Richt. A unified theory of SGD : V ariance reduction, sampling, quantization and coordinate descent. In International Conference on Artificial Intelligence and Statistics, pages 680--690. PMLR, 2020

2020
[52]

Gorbunov, H

E. Gorbunov, H. Berard, G. Gidel, and N. Loizou. Stochastic extragradient: General analysis and improved rates. In International Conference on Artificial Intelligence and Statistics, pages 7865--7901. PMLR, 2022 a

2022
[53]

Gorbunov, A

E. Gorbunov, A. Taylor, S. Horv \'a th, and G. Gidel. Convergence of proximal point and extragradient-based methods beyond monotonicity: T he case of negative comonotonicity. arXiv preprint arXiv:2210.13831, 2022 b

2022 arXiv
[54]

R. M. Gower, P. Richt \'a rik, and F. Bach. Stochastic quasi-gradient methods: V ariance reduction via J acobian sketching. Math. Program., 188 0 (1): 0 135--192, 2021

2021
[55]

Grimmer, H

B. Grimmer, H. Lu, P. Worah, and V. Mirrokni. The landscape of the proximal point method for nonconvex--nonconcave minimax optimization. Math. Program., 201 0 (1-2): 0 373--407, 2023

2023
[56]

B. Halpern. Fixed points of nonexpanding maps. Bull. Am. Math. Soc., 73 0 (6): 0 957--961, 1967

1967
[57]

Hanzely, K

F. Hanzely, K. Mishchenko, and P. Richt \'a rik. SEGA : V ariance reduction via gradient sketching. In Advances in Neural Information Processing Systems, pages 2082--2093, 2018

2018
[58]

He and R.-D

Y. He and R.-D. Monteiro. An accelerated HPE -type algorithm for a class of composite convex-concave saddle-point problems. SIAM J. Optim., 26 0 (1): 0 29--56, 2016

2016
[59]

E. Ho, A. Rajagopalan, A. Skvortsov, S. Arulampalam, and M. Piraveenan. Game theory in defence applications: A review. Sensors, 22 0 (3): 0 1032, 2022

2022
[60]

Huang, N

K. Huang, N. Wang, and S. Zhang. An accelerated variance reduced extra-point approach to finite-sum vi and optimization. arXiv preprint arXiv:2211.03269, 2022

2022 arXiv
[61]

A. N. Iusem, A. Jofr \'e , R. I. Oliveira, and P. Thompson. Extragradient method with variance reduction for stochastic variational inequalities. SIAM J. Optim., 27 0 (2): 0 686--724, 2017

2017
[62]

Johnson and T

R. Johnson and T. Zhang. Accelerating stochastic gradient descent using predictive variance reduction. In NIPS, pages 315--323, 2013

2013
[63]

Juditsky, A

A. Juditsky, A. Nemirovski, and C. Tauvel. Solving variational inequalities with stochastic mirror-prox algorithm. Stochastic Systems, 1 0 (1): 0 17--58, 2011

2011
[64]

Kannan and U

A. Kannan and U. V. Shanbhag. Optimal stochastic extragradient schemes for pseudomonotone stochastic variational inequality problems and their variants. Comput. Optim. Appl., 74 0 (3): 0 779--820, 2019

2019
[65]

Khalafi and D

M. Khalafi and D. Boob. Accelerated primal-dual methods for convex-strongly-concave saddle point problems. In International Conference on Machine Learning, pages 16250--16270. PMLR, 2023

2023
[66]

D. Kim. Accelerated proximal point method for maximally monotone operators. Math. Program., 190: 0 57--87, 2021

2021
[67]

Kolossoski and R

O. Kolossoski and R. D. Monteiro. An accelerated non- E uclidean hybrid proximal extragradient-type algorithm for convex--concave saddle-point problems. Optim. Meth. Soft., 32 0 (6): 0 1244--1272, 2017

2017
[68]

I. Konnov. Combined relaxation methods for variational inequalities. Springer-Verlag, 2001

2001
[69]

Kotsalis, G

G. Kotsalis, G. Lan, and T. Li. Simple and optimal methods for stochastic variational inequalities, i: operator extrapolation. SIAM J. Optim., 32 0 (3): 0 2041--2073, 2022

2022
[70]

Kovalev, S

D. Kovalev, S. Horvath, and P. Richtarik. D on't jump through hoops and remove those loops: SVRG and K atyusha are better without the outer loop. In Algorithmic Learning Theory, pages 451--467. PMLR, 2020

2020
[71]

D. Kuhn, S. Shafiee, and W. Wiesemann. Distributionally robust optimization. Acta Numerica, 34: 0 579--804, 2025

2025
[72]

H. W. Kuhn, J. Harsanyi, R. Selten, J. Weibul, and E. van Damme. The work of J ohn nash in game theory. journal of economic theory, 69 0 (1): 0 153--185, 1996

1996
[73]

Le Roux, M

N. Le Roux, M. Schmidt, and F. Bach. A stochastic gradient method with an exponential convergence rate for finite training sets. In NIPS, pages 2663--2671, 2012

2012
[74]

Lee and D

S. Lee and D. Kim. Fast extra gradient methods for smooth structured nonconvex-nonconcave minimax problems. Thirty-fifth Conference on Neural Information Processing Systems (NeurIPs2021), 2021 a

2021
[75]

Lee and D

S. Lee and D. Kim. Semi-anchored multi-step gradient descent ascent method for structured nonconvex-nonconcave composite minimax problems. arXiv preprint arXiv:2105.15042, 2021 b

2021 arXiv
[76]

B. Li, M. Ma, and G. B. Giannakis. On the convergence of SARAH and beyond. ArXiv preprint (arxiv.org/abs/1906.02351), Tech. Report., 2019

1906 arXiv
[77]

Z. Li, H. Bao, X. Zhang, and P. Richt \'a rik. PAGE : A simple and optimal probabilistic gradient estimator for nonconvex optimization. arXiv preprint arXiv:2008.10898, 2020

2008 arXiv
[78]

F. Lieder. On the convergence rate of the halpern-iteration. Optim. Letters, 15 0 (2): 0 405--418, 2021

2021
[79]

Loizou, H

N. Loizou, H. Berard, G. Gidel, I. Mitliagkas, and S. Lacoste-Julien. Stochastic gradient descent-ascent and consensus optimization for smooth games: C onvergence analysis under expected co-coercivity. Advances in Neural Information Processing Systems, 34: 0 19095--19108, 2021

2021
[80]

Madry, A

A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018

2018
[81]

Maing \'e

P.-E. Maing \'e . Accelerated proximal algorithms with a correction term for monotone inclusions. Applied Mathematics & Optimization, 84 0 (2): 0 2027--2061, 2021

2027
[82]

P. E. Maing \'e . Fast convergence of generalized forward-backward algorithms for structured monotone inclusions. J. Convex Anal., 29: 0 893--920, 2022

2022
[83]

Martinez, M

N. Martinez, M. Bertran, and G. Sapiro. Minimax pareto fairness: A multi objective perspective. In International Conference on Machine Learning, pages 6755--6764. PMLR, 2020

2020
[84]

Mishchenko, D

K. Mishchenko, D. Kovalev, E. Shulgin, P. Richt \'a rik, and Y. Malitsky. Revisiting stochastic extragradient. In International Conference on Artificial Intelligence and Statistics, pages 4573--4582. PMLR, 2020

2020
[85]

Namkoong and J

H. Namkoong and J. Duchi. Stochastic gradient methods for distributionally robust optimization with f-divergences. Advances in neural information processing systems, 29, 2016

2016
[86]

Nemirovski

A. Nemirovski. Mini-course on convex programming algorithms. Lecture notes, 2013

2013
[87]

Nesterov

Y. Nesterov. A method for unconstrained convex minimization problem with the rate of convergence O (1/k^2) . Doklady AN SSSR, 269: 0 543--547, 1983. Translated as Soviet Math. Dokl

1983
[88]

Nesterov

Y. Nesterov. I ntroductory lectures on convex optimization: A basic course , volume 87 of Applied Optimization. Kluwer Academic Publishers, 2004

2004
[89]

L. M. Nguyen, J. Liu, K. Scheinberg, and M. Tak \'a c . SARAH : A novel method for machine learning problems using stochastic recursive gradient. In Proceedings of the 34th International Conference on Machine Learning, pages 2613--2621, 2017

2017
[90]

Palaniappan and F

B. Palaniappan and F. Bach. Stochastic variance reduction methods for saddle-point problems. In Advances in Neural Information Processing Systems, pages 1416--1424, 2016

2016
[91]

Park and E

J. Park and E. K. Ryu. Exact optimal accelerated complexity for fixed-point iterations. In International Conference on Machine Learning, pages 17420--17457. PMLR, 2022

2022
[92]

Z. Peng, Y. Xu, M. Yan, and W. Yin. AR ock: an algorithmic framework for asynchronous parallel coordinate updates. SIAM J. Scientific Comput., 38 0 (5): 0 2851--2879, 2016

2016
[93]

Pethick, O

T. Pethick, O. Fercoq, P. Latafat, P. Patrinos, and V. Cevher. Solving stochastic weak M inty variational inequalities without increasing batch size. In Proceedings of International Conference on Learning Representations (ICLR), pages 1--34, 2023

2023
[94]

H. N. Pham, M. L. Nguyen, T. D. Phan, and Q. Tran-Dinh. ProxSARAH : A n efficient algorithmic framework for stochastic composite nonconvex optimization. J. Mach. Learn. Res., 21: 0 1--48, 2020

2020
[95]

R. R. Phelps. Convex functions, monotone operators and differentiability, volume 1364. Springer, 2009

2009
[96]

Rahimian and S

H. Rahimian and S. Mehrotra. Distributionally robust optimization: A review. arXiv preprint arXiv:1908.05659, 2019

1908 arXiv
[97]

Robbins and D

H. Robbins and D. Siegmund. A convergence theorem for non negative almost supermartingales and some applications. In Optimizing methods in statistics, pages 233--257. Elsevier, 1971

1971
[98]

Rockafellar and R

R. Rockafellar and R. Wets. V ariational A nalysis , volume 317. Springer, 2004

2004
[99]

Ryu and W

E. Ryu and W. Yin. Large-scale convex optimization: A lgorithms & analyses via monotone operators . Cambridge University Press, 2022

2022
[100]

E. K. Ryu and S. Boyd. Primer on monotone operator methods. Appl. Comput. Math, 15 0 (1): 0 3--43, 2016

2016
[101]

Sabach and S

S. Sabach and S. Shtern. A first order method for solving convex bilevel optimization problems. SIAM J. Optim., 27 0 (2): 0 640--660, 2017

2017
[102]

Sadiev, L

A. Sadiev, L. Condat, and P. Richt \'a rik. Stochastic proximal point methods for monotone inclusions under expected similarity. arXiv preprint arXiv:2405.14255, 2024

2024 arXiv
[103]

Schmidt, N

M. Schmidt, N. L. Roux, and F. Bach. Minimizing finite sums with the stochastic average gradient. Math. Program., 162 0 (1-2): 0 83--112, 2017

2017
[104]

C. Shi, M. Uehara, J. Huang, and N. Jiang. A minimax learning approach to off-policy evaluation in confounded partially observable M arkov decision processes. In International Conference on Machine Learning, pages 20057--20094. PMLR, 2022

2022
[105]

S. Sra, S. Nowozin, and S. J. Wright. O ptimization for M achine L earning . MIT Press, 2012

2012
[106]

Swamy, S

G. Swamy, S. Choudhury, J. A. Bagnell, and S. Wu. Of moments and matching: A game-theoretic framework for closing the imitation gap. In International Conference on Machine Learning, pages 10022--10032. PMLR, 2021

2021
[107]

Tran-Dinh

Q. Tran-Dinh. E xtragradient- T ype M ethods with O (1/k) - C onvergence R ates for C o- H ypomonotone I nclusions. J. Global Optim., pages 1--25, 2023

2023
[108]

Tran-Dinh

Q. Tran-Dinh. From H alpern's fixed-point iterations to N esterov's accelerated interpretations for root-finding problems. Comput. Optim. Appl., 87 0 (1): 0 181--218, 2024 a

2024
[109]

Tran-Dinh

Q. Tran-Dinh. V ariance- R educed F ast K rasnoselkii- M ann M ethods for F inite- S um R oot- F inding P roblems. arXiv preprint arXiv:2406.02413, 2024 b

2024 arXiv
[110]

Tran-Dinh

Q. Tran-Dinh. V ariance- R educed F orward- R eflected- B ackward S plitting M ethods for N onmonotone G eneralized E quations. Forty-Second International Conference on Machine Learning (ICML), 2025

2025
[111]

Tran-Dinh and Y

Q. Tran-Dinh and Y. Luo. H alpern-type accelerated and splitting algorithms for monotone inclusions. arXiv preprint arXiv:2110.08150, 2021

2021 arXiv
[112]

Tran-Dinh and Y

Q. Tran-Dinh and Y. Luo. R andomized B lock- C oordinate O ptimistic G radient A lgorithms for R oot- F inding P roblems. Math. Oper. Res., in press, 2025

2025
[113]

Tran-Dinh, H

Q. Tran-Dinh, H. N. Pham, T. D. Phan, and M. L. Nguyen. Hybrid stochastic gradient descent algorithms for stochastic nonconvex optimization. Preprint: arXiv:1905.05920, 2019

1905 arXiv
[114]

Tran-Dinh, N

Q. Tran-Dinh, N. H. Pham, D. T. Phan, and L. M. Nguyen. A hybrid stochastic optimization framework for stochastic composite nonconvex optimization. Math. Program., 191: 0 1005--1071, 2022

2022
[115]

S. J. Wright. Optimization A lgorithms for D ata A nalysis. IAS/Park City Mathematics Series, pages 1--49, 2017

2017
[116]

J. Yang, S. Zhang, N. Kiyavash, and N. He. A catalyst framework for minimax optimization. Advances in Neural Information Processing Systems, 33, 2020

2020
[117]

Yoon and E

T. Yoon and E. K. Ryu. Accelerated algorithms for smooth convex-concave minimax problems with O (1/k^2) rate on squared gradient norm. In International Conference on Machine Learning, pages 12098--12109. PMLR, 2021

2021
[118]

Yousefian, A

F. Yousefian, A. Nedi \'c , and U. V. Shanbhag. On stochastic mirror-prox algorithms for stochastic cartesian variational inequalities: R andomized block coordinate and optimal averaging schemes. Set-Valued and Variational Analysis, 26: 0 789--819, 2018

2018
[119]

Y. Yu, T. Lin, E. V. Mazumdar, and M. Jordan. Fast distributionally robust learning with variance-reduced min-max optimization. In International Conference on Artificial Intelligence and Statistics, pages 1219--1250. PMLR, 2022

2022
[120]

Yuan and Y

Y.-X. Yuan and Y. Zhang. Symplectic E xtra-gradient type method for solving general non-monotone inclusion problem. arXiv preprint arXiv:2406.10793, 2024

2024 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Adly and H

S. Adly and H. Attouch. First-order inertial algorithms involving dry friction damping. Math. Program., pages 1--41, 2021

2021

[2] [2]

R. P. Agarwal, M. Meehan, and D. O'regan. Fixed point theory and applications, volume 141. Cambridge university press, 2001

2001

[3] [3]

Alacaoglu and Y

A. Alacaoglu and Y. Malitsky. Stochastic variance reduction for variational inequality methods. In Conference on Learning Theory, pages 778--816. PMLR, 2022

2022

[4] [4]

Alacaoglu, Y

A. Alacaoglu, Y. Malitsky, and V. Cevher. Forward-reflected-backward method with variance reduction. Comput. Optim. Appl., 80 0 (2): 0 321--346, 2021

2021

[5] [5]

J. K. Alcala, Y. T. Chow, and M. Sunkula. Moving anchor extragradient methods for smooth structured minimax problems. arXiv preprint arXiv:2308.12359, 2023

arXiv 2023

[6] [6]

Arjovsky, S

M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein generative adversarial networks. In International Conference on Machine Learning, pages 214--223, 2017

2017

[7] [7]

Attouch and A

H. Attouch and A. Cabot. Convergence of a relaxed inertial proximal algorithm for maximally monotone operators. Math. Program., 184 0 (1): 0 243--287, 2020

2020

[8] [8]

Attouch and J

H. Attouch and J. Fadili. From the R avine method to the N esterov method and vice versa: A dynamical system perspective. SIAM J. Optim., 32 0 (3): 0 2074--2101, 2022

2022