REVIEW 3 major objections 3 minor 17 references
Solving Convex-Concave Problems with $\tilde{\mathcal{O}}(\epsilon^{-4/7})$ Second-Order Oracle Complexity
T0 review · 3 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper proves a triple-loop accelerated Newton-type method that solves convex-concave minimax problems with at most $\tilde{O}(D_x^{6/7}D_y^{6/7}(\rho/\epsilon)^{4/7})$ second-order oracle calls, improving the previously conjectured…
desk verdict A genuine and likely correct improvement of the second-order oracle bound for convex-concave minimax, from epsilon^{-2/3} to tilde epsilon^{-4/7}, though the lazy-Hessian extension and a few precision details deserve referee attention. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the $(\delta,\gamma)$-inexact second-order proximal oracle: for a function $h$, it returns $(z,u)$ with $u \in \partial I_Z(z)$ such that $\|\nabla h(z) + u + \lambda(z-\bar z)\| \le (\lambda/2)\|z-\bar z\| + \delta$, where $\lambda = \gamma\|z-\bar z\|$. Algorithm 1 (AIPE) runs the search-free accelerated hybrid proximal extragradient recursion with this oracle, with tolerance $\delta \le \mu\epsilon^4/(144D^2)$ and per-restart complexity $T = O((\gamma/\mu)^{2/7})$. The nesting is: the outer loop minimizes the primal function $\Phi(x) = \max_{y \in Y} f(x,y)$; the middle loop maximizes the dual function $\Psi(y;\bar x) = \min_{x \in X}[f(x,y) + (\gamma/3)\|x-\bar x\|^3]$; and the inner loop solves the cubic-regularized saddle subproblem $\min_{x \in X}\max_{y \in Y}[f(x,y) + (\gamma/3)\|x-\bar x\|^3 - (\gamma/3)\|y-\bar y\|^3]$, which is $(\gamma/2)$-uniformly convex-concave. The continuity estimate of Lemma 5.3, $\|y^*(x_1)-y^*(x_2)\|^2 \le (\ell/\mu_y)\|x_1-x_2\|$, transfers distance errors between the $x$-solver and the $y$-solver, and setting $\gamma = \rho$ balances the inner $O((\rho/\mu)^{2/3})$ NPE-restart rate with the outer $O((\gamma/\mu)^{2/7})$ AIPE-restart rate.
What would settle it
Construct a convex-concave instance with fixed diameters $D_x, D_y$ and fixed $\rho, \ell$ for which every algorithm using $T$ second-order oracle calls of $f$ has duality gap at least $c T^{-3/2}$ with $c$ independent of $T$; that would force $T \ge \epsilon^{-2/3}$ and contradict the claimed $\epsilon^{-4/7}$ upper bound. A less formal check is to implement Algorithm 3 on a cubic-regularized bilinear test problem and count CRN oracle calls to reach a fixed gap: an observed exponent closer to $2/3$ than $4/7$ would indicate that the required precision tolerances are not being met.
Extended reading notes
Core claim
The paper's central claim is that the near-optimal second-order method for convex minimization, when restarted and nested inside itself, solves convex-concave minimax problems faster than any Newton proximal-extragradient variant proposed before. Concretely, Theorem 5.5 states that under convexity-concavity, bounded domains, Lipschitz gradients, and $\rho$-Lipschitz Hessians, Algorithm 3 with $\gamma = \rho$, $M_{\min} = \text{ANPE-restart}$, and $M_{\text{saddle}} = \text{NPE-restart}$ outputs an $\epsilon$-solution using at most $\tilde{O}(D_x^{6/7}D_y^{6/7}(\rho/\epsilon)^{4/7})$ second-order oracle calls, where $D_x, D_y$ are the domain diameters. The mechanism is to add cubic regularizers with parameters $\mu_x = \epsilon/(2D_x^3)$ and $\mu_y = \epsilon/(2D_y^3)$, solve the regularized problem by the triple loop, and then invoke Lemma 5.1, which says an $(\epsilon/3)$-solution of the regularized problem is an $\epsilon$-solution of the original one.
Load-bearing premise
The whole rate depends on being able to solve the inner cubic-regularized saddle subproblems accurately enough, with errors no larger than about $\mu\epsilon^4/D^2$, without spending more iterations than the stated counts, and the paper leaves the polynomial dependence of the precision parameters $\zeta_2, \zeta_3$ implicit in Theorems 5.2 and 5.3.
Editorial extensions
If this is right
- Standard NPE-style Newton solvers for convex-concave minimax now have a provable oracle count $\tilde{O}(\epsilon^{-4/7})$, improving the previous $O(\epsilon^{-2/3})$ for every $\epsilon < 1$.
- The lazy-Hessian variant improves LEN from $O(m + m^{2/3}\epsilon^{-2/3})$ to $\tilde{O}(m + m^{5/7}\epsilon^{-4/7})$ while querying Hessians only once every $m$ iterations.
- The reduction in Lemma 5.1 means any $\epsilon$-solver for uniformly convex-concave cubic-regularized saddle problems automatically yields an $\epsilon$-solver for the original problem, so the extra Lipschitz assumptions are mild corollaries of Hessian Lipschitzness on compact domains.
- Existing attempted proofs of an $\Omega(\epsilon^{-2/3})$ lower bound do not apply here because their hard instances let the diameters $D_x, D_y$ grow with $T$; the paper argues this in Appendix A.
- The framework is a second-order analogue of Catalyst: any linearly convergent globally convergent saddle solver satisfying Assumption 5.3 can be plugged into the inner loop and inherit the accelerated outer rate.
Reading between the lines
- If the $\epsilon^{-4/7}$ rate is correct, a tight lower bound for second-order convex-concave minimax would have to come from a zero-chain coupled across both variables with constant diameters, and the natural target exponent would be $4/7$, up to logarithmic factors.
- The same nesting recipe, with a $p$-th-order tensor proximal step replacing the cubic-regularized Newton step, is a plausible route to improved rates for $p \ge 3$ smooth minimax problems, but the paper only proves the implementable case $p=2$ and leaves the tensor-step implementation open.
- Because the framework accelerates any linearly convergent saddle solver, stochastic or quasi-Newton saddle solvers could inherit the improved oracle-count dependence, provided their own error tolerances can be propagated through the loops at polynomial cost.
- The logarithmic factor $O(\epsilon^{-4/7}\log^3(1/\epsilon))$ appears removable by the same search-free techniques used for strongly convex-strongly concave problems, which the authors list as future work.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies convex-concave minimax problems with second-order (CRN) oracles. It proposes Minimax-AIPE, a triple-loop algorithm: an outer AIPE-restart minimizes the primal function Φ, a middle AIPE-restart maximizes the dual function Ψ, and an inner loop implements the resulting proximal oracle by a linearly convergent saddle-point solver. The main result, Theorem 5.5, claims \tilde O(D_x^{6/7}D_y^{6/7}(ρ/ε)^{4/7}) CRN oracle calls under Assumptions 3.1–3.5, improving the previous O(ε^{-2/3}) NPE bound. A lazy-Hessian variant is developed in Theorems 5.6–5.7. Appendix B contains a detailed proof of the AIPE error tolerance, Appendix E.1 gives the NPE restart analysis, and Appendix A discusses why existing Ω(ε^{-2/3}) lower bounds do not apply.
Significance. If the claims are correct, this is a substantial contribution: it breaks the speculated second-order minimax barrier, transfers Monteiro–Svaiter-type acceleration to saddle-point problems, and provides a Catalyst-style framework for second-order methods. The paper has genuine strengths: the AIPE error-budget proof in Appendix B is written out in detail, the NPE restart proof is self-contained, and the lower-bound discussion carefully identifies the diameter dependence in prior hard instances. The main reasons I cannot recommend acceptance now are the unverified precision accounting in the nested loops and the delegated constrained-case LEN/ALEN proofs, both of which are load-bearing for the advertised complexity.
major comments (3)
- [Theorems 5.2–5.3 and Appendix D] The precision statements in Theorems 5.2 and 5.3 are stated as ζ2 = Ω(1/poly(ρ, ℓ, L, D, γ, μx^{-1}, μy^{-1}, ζ1^{-1})) and ζ3 = Ω(1/poly(...)), but the proofs in Appendix D require ζ2 and ζ3 to be sufficiently small so that the residuals in Eq. (21) and Eq. (22) are bounded by δ ≲ μxζ1^4/D^2 and δ ≲ μyζ2^4/D^2, respectively. That is an upper-bound requirement, so the Ω notation appears to have the wrong direction. The explicit exponents and constants are never supplied, and Eq. (21) mixes coefficients in ρ and γ. Because Theorem 5.5's ε-exponent depends on this nested precision chain being polynomial in ζ1 (and ultimately in ε), the manuscript should state explicit feasible choices, e.g. ζ2 ≤ c·(μx ζ1^4 / ((ℓ+γD)^4 ...)) and the analogous bound for ζ3, and verify that the induced log factors are poly-logarithmic in 1/ε. As written, the central \tilde O(ε^{-4/7}) claim cannot be fully checked.
- [Appendix E.2, Theorems E.3–E.4] The constrained-case LEN and ALEN convergence results are not proved in this manuscript. Theorem E.3 says the proof is 'essentially the same' as the unconstrained analysis in Chen et al. (2025b), and Theorem E.4 says the constrained version is 'readily extended' from Chen et al. (2025a). Since Theorem 5.7 and the LEN row of Table 1 depend on these theorems, this is a load-bearing gap. The constrained setting introduces projection/normal-cone terms in the optimality conditions (Eqs. (24)–(25)) and in the lazy-CRN oracle; the authors should include the actual constrained proof or explicitly declare these as assumptions rather than theorems.
- [Section 5.3, Theorem 5.5] Theorem 5.5 applies the analysis to the regularized objective from Lemma 5.1, but the regularization adds cubic terms whose Hessians are (2μx, 2μy)-Lipschitz. The Hessian Lipschitz constant of the regularized problem is therefore ρ+O(μx+μy)=ρ+O(ε/D^3), not ρ. The proof should either carry out the analysis with this effective constant or show explicitly that replacing ρ by ρ+cε/D^3 only changes the stated \tilde O(D_x^{6/7}D_y^{6/7}(ρ/ε)^{4/7}) bound by constants and logarithmic factors. In particular, the choice γ=ρ should be re-justified for the regularized objective.
minor comments (3)
- [Assumptions 3.3–3.5] The norms in Assumptions 3.3, 3.4 and 3.5 are written as ∥x−y∥ with x∈X and y∈Y, which is dimensionally inconsistent; they should be ∥(x,y)−(x′,y′)∥. The same issue appears in several places in the text.
- [Algorithm 1, line 23] There is a typographical error in line 23 of Algorithm 1: the expression contains a stray '⟩' after the quadratic term, making the update rule hard to parse.
- [Theorem 5.5] The final bound in Theorem 5.5 omits L and ℓ, which enter through the choice of ζ1 and the extragradient step in Algorithm 3. If these constants enter only through polylog factors, the paper should say so explicitly; currently the \tilde O notation hides an unspecified dependence on these quantities.
Circularity Check
No significant circularity: the ε^{-4/7} rate is derived from external A-HPE/NPE results and the paper's own error-propagation bounds; the only self-citations (Chen et al. 2025a,b) back the lazy-Hessian extension and are not load-bearing.
full rationale
The main derivation chain is self-contained against external benchmarks. Theorem 5.5 is obtained by: (i) the cubic-regularization reduction (Lemma 5.1) setting μx=ε/(2D_x^3) and μy=ε/(2D_y^3); (ii) the inexact AIPE-restart guarantee (Theorem 4.1), whose proof in Appendix B reproduces the Carmon et al. (2022a) A-HPE argument with an explicit inexactness budget δ≤με^4/(144D^2); and (iii) the nested-oracle error bounds (Theorems 5.1-5.3), where Eq. (21) and Eq. (22) express the proximal-oracle residuals as explicit polynomials in ζ2 and ζ3 and then set these precisions small enough. No step defines a quantity in terms of the target, and no fitted parameter is renamed as a prediction. The stated Ω(1/poly(...)) precisions in Theorems 5.2 and 5.3 are not enumerated, which is a completeness and rigor gap in the proof, but those are derived inequalities rather than assumed conclusions. The only self-citations are to Chen et al. (2025a,b) for the lazy-Hessian rates T_saddle=m+m^{2/3}(ρ/μ)^{2/3} and T_min=m+m^{5/7}(ρ/μ)^{2/7} used in Theorem 5.7 and Appendix E.2; these are prior work by overlapping authors and are not needed for the central ε^{-4/7} claim, which uses NPE (Monteiro and Svaiter, 2012) and A-NPE (Carmon et al., 2022a). Consequently, no circular step can be quoted from the paper's own equations.
Assumptions & free parameters
free parameters (3)
- gamma =
rho (Theorem 5.5), rho/sqrt(m) (Theorem 5.7)
- mu_x =
epsilon/(2 D_x^3)
- mu_y =
epsilon/(2 D_y^3)
assumptions (6)
- domain assumption Convex-concavity, compact convex domains, Lipschitzness, gradient Lipschitzness, and Hessian Lipschitzness (Assumptions 3.1 to 3.5).
- standard math Extragradient gradient-norm bound, Lemma 3.1, imported from Cai et al. 2022.
- standard math The CRN oracle implements a (0, rho)-proximal oracle, Lemma 4.1.
- standard math NPE regret bound in Eq. (23).
- domain assumption Linear convergence of subroutines Mmin and Msaddle, Assumptions 5.2 and 5.3.
- ad hoc to paper Constrained-case LEN and ALEN convergence results, Theorems E.3 and E.4.
Cite this review
Pith. "Pith review of Solving Convex-Concave Problems with $\tilde{\mathcal{O}}(\epsilon^{-4/7})$ Second-Order Oracle Complexity." pith.science (2026). https://pith.science/paper/2EHXGRU2
@misc{pith2026250608362,
author = {Pith},
title = {Pith review of: Solving Convex-Concave Problems with $\tilde\mathcalO(\epsilon^-4/7)$ Second-Order Oracle Complexity},
year = {2026},
howpublished = {\url{https://pith.science/paper/2EHXGRU2}},
note = {Machine review of arXiv:2506.08362}
}
abstract
Previous algorithms can solve convex-concave minimax problems $\min_{x \in \mathcal{X}} \max_{y \in \mathcal{Y}} f(x,y)$ with $\mathcal{O}(\epsilon^{-2/3})$ second-order oracle calls using Newton-type methods. This result has been speculated to be optimal because the upper bound is achieved by a natural generalization of the optimal first-order method. In this work, we show an improved upper bound of $\tilde{\mathcal{O}}(\epsilon^{-4/7})$ by generalizing the optimal second-order method for convex optimization to solve the convex-concave minimax problem. We further apply a similar technique to lazy Hessian algorithms and show that our proposed algorithm can also be seen as a second-order ``Catalyst'' framework (Lin et al., JMLR 2018) that could accelerate any globally convergent algorithms for solving minimax problems.
Reference graph
Works this paper leans on
-
[1]
Optimal methods for higher-order smooth monotone variational inequalities
Deeksha Adil, Brian Bullins, Arun Jambulapati, and Sushant Sachdeva. Optimal methods for higher-order smooth monotone variational inequalities. arXiv preprint arXiv:2205.06167,
-
[4]
Line 2 in Algorithm 3 finds ∥ ˆy − y∗( ˆx)∥ ≤ζ1, where y∗(x) = arg maxy∈Y f (x, y). Then ∥ ˆy − y∗∥ ≤ ∥ˆy − y∗( ˆx)∥ + ℓ1/2 µ1/2 y p ∥ ˆx − x∗∥, where we use Lemma 5.3 and y∗(x∗) = x∗ in the above inequality. It means ∥( ˆx, ˆy) − (x∗, y∗)∥ ≤3 s ℓζ1 µy Then from Lemma 3.1 we know Line 3 in Algorithm 3 outputs zout = (xout, yout) such that F (zout) + cout ...
work page 2012
-
[6]
It is known (Monteiro and Svaiter, 2012; Adil et al., 2022; Bullins and Lai,
work page 2012
-
[7]
can solve a µ-strongly monotone variational inequality problem in O((γ/µ)2/3 log ϵ−1) iteration complexity, as stated in the following theorem. Theorem E.1 (NPE-restart) Under Assumption 5.1 with µx = µy = µ and Assumption 3.5, running Algorithm 7 with γ = 2ρ and T = O((γ/µ)2/3) and S = O(log(d0/ϵ)) returns a point z(S) such that ∥z(S) − z∗∥ ≤ϵ, where z∗ ...
work page 2012
-
[8]
Mitigating unwanted biases with adversarial learning
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335–340,
work page 2018
-
[9]
Discussions on the Existing Ω(ϵ−2/3) Lower Bound Adil et al
18 SOLVING CONVEX -C ONCAVE PROBLEMS WITH ˜O(ϵ−4/7) SECOND -O RDER ORACLE COMPLEXITY Appendix A. Discussions on the Existing Ω(ϵ−2/3) Lower Bound Adil et al. (2022) provided anΩ(ϵ−2/3) lower bound on the second-order oracle of the primal function Φ(x) = max y∈Y f (x, y). However, practical algorithms (including our algorithm and Adil et al. (2022)’s algor...
work page 2022
-
[10]
to establish a lower bound with the second-order oracle of f (x, y). They showed that there exists a ρ-Hessian smooth function, such that the output of any second-order method that queries no more than T second-order oracles of f (x, y) must has duality gap of Ω(ρDxD2 y/T 3/2) (Lin and Jordan, 2024, Theorem 3.11). However, the diameters of sets X and Y in...
work page 2024
-
[11]
Therefore, we have that min{zTϵ, ¯zTϵ}} ≤µϵ3/3
to show that the algorithm finds Et ≤ µϵ3/3 in Tϵ = O γd3 0/(µϵ3) 2/7 iterations, where d0 = ∥z0 − z∗∥. Therefore, we have that min{zTϵ, ¯zTϵ}} ≤µϵ3/3. This further implies h(zout) ≤ ˜h(zout) + δ ≤ ˜h(zTϵ) + δ ≤ h(zTϵ) + 2δ. If 2δ ≤ µϵ3/3 and we know from Et ≤ µϵ3/3 that h(zout) − h(z∗) ≤ 2µϵ3/3. By Lemma 3.2, this implies that ∥zout − z∗∥ ≤2ϵ. The last t...
work page 2020
Show all 17 references
-
[15]
that Algorithm 6 ensures Reget := 1PT −1 t=0 ηt T −1X t=0 ηt⟨F (zt+1/2), zt+1/2 − z∗⟩ = O γ∥z0 − z∗∥3 T 3/2 . (23) We further use Lemma C.1, the convexity of∥ · ∥3 and Jensen’s inequality to derive that ∥zout − z∗∥3 ≤ 1PT −1 t=0 ηt T −1X t=0 ηt∥zt+1/2 − z∗∥3 ≤ 3 2µ Regret, whi...
2025
-
[16]
The proof is essentially the same as (Chen et al., 2025b, Theorem 4.1), except here we consider the constrained case
under the convex-concave setting. The proof is essentially the same as (Chen et al., 2025b, Theorem 4.1), except here we consider the constrained case. Below, we show that all the proofs in (Chen et al., 2025b, Theorem 4.1) also hold under the constrained case. Using the first...
2023
-
[17]
as a subroutine, while the proof of lazy CRN can be readily extended to the constrained case (Doikov et al., 2023, Appendix F). 31
2023
-
[2000]
The oracle complexity of simplex-based matrix games: Linear separability and nash equilibria
Guy Kornowski and Ohad Shamir. The oracle complexity of simplex-based matrix games: Linear separability and nash equilibria. arXiv preprint arXiv:2412.06990,
-
[2019]
An approximation-based regularized extra-gradient method for monotone variational inequalities
Kevin Huang and Shuzhong Zhang. An approximation-based regularized extra-gradient method for monotone variational inequalities. arXiv preprint arXiv:2210.04440,
-
[2020]
Tianyi Lin, Panayotis Mertikopoulos, and Michael I. Jordan. Explicit second-order min-max optimization methods with optimal convergence guarantee. arXiv preprint arXiv:2210.12860,
-
[2021]
High-order reduced-gradient methods for composite variational inequalities
Yurii Nesterov. High-order reduced-gradient methods for composite variational inequalities. arXiv preprint arXiv:2311.15154, 2023a. Yurii Nesterov. Inexact accelerated high-order proximal-point methods. Mathematical Programming, pages 1–26, 2023b. Yurii Nesterov and Boris T Po...
-
[2022]
Regularized newton methods for monotone variational inequalities with h\” older continuous jacobians
16 SOLVING CONVEX -C ONCAVE PROBLEMS WITH ˜O(ϵ−4/7) SECOND -O RDER ORACLE COMPLEXITY Chengchang Liu and Luo Luo. Regularized newton methods for monotone variational inequalities with h\” older continuous jacobians. arXiv preprint arXiv:2212.07824,
-
[2024]
Computationally faster newton methods by lazy evaluations
Lesi Chen, Chengchang Liu, Luo Luo, and Jingzhao Zhang. Computationally faster newton methods by lazy evaluations. arXiv preprint arXiv:2501.17488, 2025a. Lesi Chen, Chengchang Liu, and Jingzhao Zhang. Second-order min-max optimization with lazy hessians. In ICLR, 2025b. Sebas...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.