REVIEW 2 major objections 6 minor 37 references
Anderson acceleration of the proximal point method: the exact adaptive minimax, a spectral phase transition, and optimal safeguarding
T0 review · 2 major / 6 minor · reviewed 2026-07-31 · grok-4.5
Pith's one-line read Any adaptive residual-polynomial acceleration of the proximal point method is minimax-limited to residual d0/(K+1) after K resolvent calls, matching a one-line averaged-reflection estimator.
desk verdict Exact adaptive minimax and spectral phase transition check out; the safeguarding "factor two is optimal" claim overreaches on amortization. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The extremal spectral measure on the roots of u^{K+1}=-1 with masses proportional to csc-squared, dual via Christoffel functions to the circle Chebyshev problem min |(u-1)P(u)| = 2/(K+1). Cauchy–Schwarz on the Lagrange basis at those nodes forces the 1/(K+1) barrier and identifies the Fejér kernel as the unique optimizer; Jackson kernels then give the escape side when the spectral floor clears 1/K.
What would settle it
On the paper’s explicit extremal instance, compute the residual of every degree-K polynomial method (or full-memory Anderson) and check whether any falls strictly below 1/(K+1), or whether the averaged-reflection residual fails to land on the claimed per-step floor 1/sqrt((K+1)(k+1)) to machine precision.
Extended reading notes
Core claim
Over the full class of residual-polynomial methods (any memory, adaptivity, or safeguarding), the minimax fixed-point residual after K resolvent evaluations is exactly d0/(K+1). The upper bound is elementary: the averaged reflection of the orbit attains the bound for every maximal monotone operator. The matching lower bound is realized by one explicit skew-adjoint instance whose resolvent eigenvalues sit at the roots of u^{K+1}=-1 with csc-squared masses; on that instance every degree-K polynomial method satisfies residual at least 1/(K+1), the unique optimizer is the Fejér kernel, and the same instance supplies a per-step floor.
Load-bearing premise
The matching lower bound and phase transition are proved only inside methods that output affine combinations of resolvent values, via a linear polynomial reduction that fails for genuinely nonlinear trajectories.
Editorial extensions
If this is right
- On linear worst-case spectra, adaptivity and memory buy nothing over the offline averaged-reflection estimator.
- Anderson acceleration of PPM needs no safeguarding on linear problems; residuals decrease automatically.
- Certifying an O(1/k) envelope on nonlinear problems requires exactly two resolvent evaluations per iteration; history-only certificates are impossible.
- When the resolvent spectrum clears distance s with sK→∞, Jackson/Chebyshev schedules beat the 1/K envelope; at s≍1/K the barrier is exact.
- Under Hölderian growth the correct residual rates are the stated trichotomy (superlinear / linear / power-law), sharp for f(x)=|x|^q/q.
Reading between the lines
- Practitioners running Anderson on splitting methods should treat dense near-fixed spectra as unaccelerable and default to averaged reflection or Halpern-type schedules rather than large-memory AA.
- An online spectral-floor diagnostic that switches between Fejér and Jackson regimes would turn the phase-transition map into a practical algorithm without prior knowledge of s.
- The same Christoffel–Chebyshev duality may pin exact adaptive minimax values for other firmly nonexpansive fixed-point iterations beyond PPM.
- Stochastic or inexact resolvents will require coefficient control (ridge or capping) before any variance-aware certificate can inherit the deterministic 1/(K+1) map.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies residual-polynomial acceleration of the proximal point method, with Anderson acceleration as the prototypical adaptive scheme, and makes three headline claims. (i) The minimax residual over the adaptive class A∞ after K resolvent evaluations is exactly d0/(K+1): the averaged-reflection estimator attains it for every maximal monotone M by a telescoping identity (Thm 3.1), and an explicit skew-adjoint extremal instance — resolvent eigenvalues (1+ω_j)/2 at the roots of u^{K+1}=−1 with csc² masses — defeats every degree-K polynomial method, with the Fejér kernel as unique optimizer and a per-step floor (Thm 4.4, Cor 4.5, Thm 4.7). (ii) A spectral phase transition at s≍1/K: Jackson-kernel escape O(d0/(K²s)) for sK→∞, exact barrier 1/(K+1) at the critical floor (Thms 5.1–5.2), extended to normal operators and to the nonlinear family M=S+N_C (Thms 6.1, 6.4). (iii) Safeguarding: none needed on linear problems (Cor 2.5); on nonlinear problems a certified scheme at two evaluations per iteration (Thm 7.1), with a claimed matching optimality of the factor two (Prop 7.2, Cor 7.3). Auxiliary sections correct structured results (finite termination, strong monotonicity, Hölderian growth trichotomy) and report numerics. I verified the core derivations (Thms 3.1, 4.4, 4.7, 4.11, 5.1) line by line; they are sound, and the csc²/Cauchy–Schwarz/aliasing computations check out.
Significance. If the results hold, this is a strong contribution. The minimax value d0/(K+1) is proved with an exact closed-form extremal spectral measure (roots of u^{K+1}=−1 with csc² masses), a unique extremal polynomial (Fejér), simultaneous per-step floors on one fixed instance, and a no-gap Christoffel–Chebyshev duality — genuinely complementary to Park–Ryu's span-oracle result, which gives the value but not the spectral characterization. The upper bound is a one-line telescoping identity valid for every maximal monotone operator. The spectral phase transition at s≍1/K with exact constants, its extension to normal operators, and the honest treatment of the nonlinear family and of stochastic caveats add further value. Numerics verify exact constants to machine precision. The paper is careful about what is proved versus open (Remarks 6.5, 8.5, §10).
major comments (2)
- [§7.2, Corollary 7.3] §7.2, Corollary 7.3 (and the abstract): the necessity half of 'the factor two is optimal' is not established as stated. Proposition 7.2 shows only that a probe's residual cannot be certified from window history, hence the probe must be evaluated. Corollary 7.3 additionally asserts that one must 'advance a certified chain at an evaluated point' at *every* iteration, but nothing forces the probe evaluation and the chain advance to coincide in time. Counterexample within the paper's own framework: alternate — odd iterations spend the single oracle call advancing the certified chain c_{j+1}=Jc_j, even iterations spend it evaluating the probe. Against an adversary that rejects all probes, the chain has advanced floor(K/2) times after K evaluations, certifying ∥r∥≤d0/√(floor(K/2)+1) — exactly the guarantee Theorem 7.1(i) itself advertises ('never worse than PPM run at half the oracle budget'),
- [§8.1, Theorem 8.1] §8.1, Theorem 8.1: the proof asserts that taking q of degree ν vanishing on the spectrum of L off {1} gives (L−I)q(L)=0 on R^d. This requires L diagonalizable (or q divisible by the minimal polynomial). Firmly nonexpansive resolvents of linear maximal monotone operators can have Jordan blocks off the eigenvalue 1: e.g. M=[[1,−2],[0,1]] is monotone (⟨Mx,x⟩=(x1−x2)²≥0) with L=(I+M)^{−1}=[[1/2,1/2],[0,1/2]], a nontrivial Jordan block at 1/2; a polynomial vanishing at the distinct eigenvalues does not annihilate it. The statement should use the degree of the minimal polynomial of L (restricted away from ker(I−L)) rather than the number ν of distinct eigenvalues; the GMRES-termination argument then goes through unchanged. Local fix, but the theorem as proved covers only the diagonalizable case.
minor comments (6)
- [§4.2, Theorem 4.4(ii)] Thm 4.4(ii): the Cauchy–Schwarz equality case is q_j = conj(ℓ_j(1)), not ℓ_j(1); the final chain '2/(n(1−ω_j)) = ℓ_j(1)' should read '= conj(ℓ_j(1))' (the value is correct since conj(ℓ_j(1)) = 2/(n(1−ω_j))).
- [§4.3, Remark 4.13] Remark 4.13 claims full-memory AA attains the instance minimax at every step on every linear instance. By Lemma 2.4, AA minimizes the surrogate ∥Σγ_i r_i∥ while the true candidate residual is ∥LΣγ_i r_i∥; these coincide only when L acts (block-wise) isometrically on the relevant subspace. Please qualify the remark (e.g. to normal/conformal L, as in the skew-adjoint instances where Experiment 2 confirms it) or add a justification.
- [Abstract] Abstract vs. Theorem 7.1: the abstract speaks of certifying 'the O(1/k) envelope', but the certified floor in Thm 7.1(i) is the PPM O(d0/√k) rate. Please align the wording.
- [Abstract / §4.3] Abstract and Corollary 4.5: 'the minimax complexity over all adaptive methods' is proved over the residual-polynomial class A∞ (Lemma 2.3); the extension to all deterministic methods rests on Park–Ryu. Suggest 'all adaptive residual-polynomial methods' in the abstract, with the Park–Ryu complementarity stated explicitly there as it is in §1.2.
- [§4–§5] Notation: n = K+1 in §4.1 (Definition 4.1) but N = K+1 in §4.4 (Definition 4.9), and N is reused for the Jackson half-degree in Theorem 5.1. A uniform symbol would ease cross-referencing.
- [§9] §9: 'Code is available from the author upon request.' Given that the numerics verify exact constants to machine precision (Tables 2, 5, 6), a public repository with the Gram-system solver and resolvent routines would materially strengthen reproducibility.
Circularity Check
No significant circularity: the exact minimax, phase transition, and Fejér uniqueness are derived from explicit spectral constructions and classical identities, not from fitted inputs or load-bearing self-citation.
full rationale
The paper’s central chain is self-contained. The upper bound (Thm 3.1) is an elementary telescoping identity for the averaged-reflection estimator that holds for every maximal monotone M. The matching lower bound (Thm 4.4) is proved on an explicitly constructed skew-adjoint instance (roots of u^{K+1}=-1 with csc² masses) by Lagrange interpolation identities, the classical sum Σ csc² = n², and Cauchy–Schwarz; equality forces the Fejér kernel uniquely. That construction is not fitted to residual data and does not presuppose the claimed rate. Park–Ryu is cited only as complementary (span/deterministic classes), not as a hidden premise that forces the spectral measure. The phase-transition constants follow from the same instance plus a Jackson-kernel estimate; the nonlinear-family minimax (Thm 6.4) holds by inclusion of the linear worst case. Safeguarding necessity (Prop 7.2–Cor 7.3) is an independent oracle argument, not a circular reduction. No self-definitional loop, no fitted-parameter-as-prediction, and no load-bearing uniqueness imported from the authors’ prior work. Residual concerns (scope of A∞, amortized safeguarding) are correctness/scope issues, not circularity.
Assumptions & free parameters
assumptions (5)
- domain assumption Maximal monotonicity of M implies J and R are (firmly) nonexpansive, zer M = Fix J = Fix R, and the resolvent identity used in residual bounds.
- domain assumption On linear M, every method in A∞ produces y_k = Q_k(L)y_0 with Q_k(1)=1 and deg Q_k ≤ k (polynomial reduction).
- standard math Classical sum ∑_{j=0}^{n-1} csc²((2j+1)π/(2n)) = n² and Lagrange identity ℓ_j(1) = −2ω_j/(n(1−ω_j)) on roots of u^n = −1.
- standard math Bernstein inequality on the circle and the circle Chebyshev value min_{P(1)=1} max_{|u|=1} |(u−1)P(u)| = 2/(K+1).
- ad hoc to paper Complexity is measured inside residual-polynomial methods A_m / A∞ (affine combinations of resolvent outputs with history-dependent coefficients).
invented entities (1)
-
Extremal skew-adjoint instance M★_K with resolvent eigenvalues (1+ω_j)/2 at roots of u^{K+1}=−1 and masses w_j ∝ csc²((2j+1)π/(2K+2))
independent evidence
Cite this review
Pith. "Pith review of Anderson acceleration of the proximal point method: the exact adaptive minimax, a spectral phase transition, and optimal safeguarding." pith.science (2026). https://pith.science/paper/KUGFCKQZ
@misc{pith2026260724643,
author = {Pith},
title = {Pith review of: Anderson acceleration of the proximal point method: the exact adaptive minimax, a spectral phase transition, and optimal safeguarding},
year = {2026},
howpublished = {\url{https://pith.science/paper/KUGFCKQZ}},
note = {Machine review of arXiv:2607.24643}
}
abstract
\noindent We study residual-polynomial acceleration of the proximal point method (PPM) for maximal monotone inclusions, with Anderson acceleration (AA) as the prototypical adaptive scheme. We answer three questions exactly. (i)~The minimax complexity over all adaptive methods is precisely $d_0/(K+1)$ per $K$ resolvent evaluations. The upper bound is attained by the averaged-reflection estimator; the matching lower bound uses an explicit skew-adjoint instance with resolvent eigenvalues at the roots of $u^{K+1}=-1$ and $\csc^2$-distributed masses, on which every degree-$K$ polynomial method satisfies $\|r(y_K)\|\ge 1/(K+1)$. The optimal polynomial is uniquely the Fej\'er kernel, and the same instance certifies a per-step floor. (ii)~A sharp phase transition separates regimes: Jackson-kernel polynomials achieve $O(d_0/(K^2 s))$ when the spectral floor $s$ satisfies $sK\to\infty$, while at the critical scale $s\asymp 1/K$ the barrier is exactly $1/(K+1)$. The picture extends to normal operators and the nonlinear family $M=S+N_C$. (iii)~On linear problems AA-PPM needs no safeguarding; on nonlinear problems certification of the $O(1/k)$ envelope requires exactly two oracle evaluations per iteration, and this factor is optimal. We also correct and complete the theory for structured problems---affine, strongly monotone, piecewise-affine, and H\"olderian growth---and confirm all predictions numerically.
Figures
Reference graph
Works this paper leans on
-
[1]
N. I. Achieser,Theory of Approximation, Dover, 1992
1992
-
[2]
D. G. Anderson, Iterative procedures for nonlinear integral equations,J. ACM12 (1965), 547–560
1965
-
[3]
Attouch and J
H. Attouch and J. Bolte, On the convergence of the proximal algorithm for nonsmooth functions involving analytic features,Math. Program.116 (2009), 5–16
2009
-
[4]
H. H. Bauschke and P. L. Combettes,Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd ed., Springer, 2017
2017
-
[5]
Bolte, A
J. Bolte, A. Daniilidis, and A. Lewis, The Lojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems,SIAM J. Optim.17 (2007), 1205–1223
2007
-
[6]
Bravo and R
M. Bravo and R. Cominetti, Sharp convergence rates for averaged nonexpansive maps,Israel J. Math.232 (2019), 279–303
2019
-
[7]
Br´ ezis and P.-L
H. Br´ ezis and P.-L. Lions, Produits infinis de r´ esolvantes,Israel J. Math.29 (1978), 329–345
1978
-
[8]
Cominetti, J
R. Cominetti, J. A. Soto, and J. Vaisman, On the rate of convergence of Krasnosel’ski ˘ ı–Mann iterations and their connection with sums of Bernoullis,Israel J. Math.199 (2014), 757–772
2014
Show all 37 references
-
[9]
J. P. Contreras and R. Cominetti, Optimal error bounds for non-expansive fixed-point iterations, SIAM J. Optim.32 (2022), 1416–1435
2022
-
[10]
Diakonikolas, Halpern iteration for near-optimal and parameter-free monotone inclusion and strong solutions to variational inequalities,Proc
J. Diakonikolas, Halpern iteration for near-optimal and parameter-free monotone inclusion and strong solutions to variational inequalities,Proc. COLT(2020), 1428–1451
2020
-
[11]
Eckstein and D
J. Eckstein and D. P. Bertsekas, On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators,Math. Program.55 (1992), 293–318
1992
-
[12]
Eiermann, X
M. Eiermann, X. Li, and R. S. Varga, On hybrid semi-iterative methods,SIAM J. Numer. Anal.26 (1989), 152–168
1989
-
[13]
Evans, S
C. Evans, S. Pollock, L. G. Rebholz, and M. Xiao, A proof that Anderson acceleration improves the convergence rate in linearly converging fixed-point methods (but not in those converging quadratically),SIAM J. Numer. Anal.58 (2020), 788–810. 24
2020
-
[14]
Gu and J
G. Gu and J. Yang, Tight sublinear convergence rate of the proximal point algorithm,J. Optim. Theory Appl.185 (2020), 1019–1028
2020
-
[15]
Halpern, Fixed points of nonexpanding maps,Bull
B. Halpern, Fixed points of nonexpanding maps,Bull. Amer. Math. Soc.73 (1967), 957–961
1967
-
[16]
Joubert, On the convergence behavior of the restarted GMRES algorithm for solving nonsymmetric linear systems,Numer
W. Joubert, On the convergence behavior of the restarted GMRES algorithm for solving nonsymmetric linear systems,Numer. Linear Algebra Appl.1 (1994), 427–447
1994
-
[17]
Kim, Accelerated proximal point method for maximally monotone operators,Math
D. Kim, Accelerated proximal point method for maximally monotone operators,Math. Program. 190 (2021), 57–87
2021
-
[18]
Kurdyka, On gradients of functions definable in o-minimal structures,Ann
K. Kurdyka, On gradients of functions definable in o-minimal structures,Ann. Inst. Fourier48 (1998), 769–783
1998
-
[19]
Lee and D
S. Lee and D. Kim, Fast extra gradient methods for smooth structured nonconvex-nonconcave minimax problems,Proc. NeurIPS(2021)
2021
-
[20]
Leventhal, Metric subregularity and the proximal point method,J
D. Leventhal, Metric subregularity and the proximal point method,J. Math. Anal. Appl.360 (2009), 681–688
2009
-
[21]
Li and T
G. Li and T. K. Pong, Calculus of the exponent of Kurdyka– Lojasiewicz inequality and its applications to linear convergence of first-order methods,Found. Comput. Math.18 (2018), 1199–1232
2018
-
[22]
Lieder, On the convergence rate of the Halpern-iteration,Optim
F. Lieder, On the convergence rate of the Halpern-iteration,Optim. Lett.15 (2021), 405–418
2021
-
[23]
A. Nemirovski, Prox-method with rate of convergence O(1/t) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems, SIAM J. Optim.15 (2004), 229–251
2004
-
[24]
Ouyang and Y
Y. Ouyang and Y. Xu, Lower complexity bounds of first-order methods for convex-concave bilinear saddle-point problems,Math. Program.185 (2021), 1–35
2021
-
[25]
Park and E
J. Park and E. K. Ryu. Exact optimal accelerated complexity for fixed-point iterations. In Proceedings of the 39th International Conference on Machine Learning (ICML), volume 162, 2022
2022
-
[26]
Pollock and L
S. Pollock and L. G. Rebholz, Anderson acceleration for contractive and noncontractive operators,IMA J. Numer. Anal.41 (2021), 2841–2872
2021
-
[27]
Q. I. Rahman and G. Schmeisser,Analytic Theory of Polynomials, Oxford University Press, 2002
2002
-
[28]
R. T. Rockafellar, Monotone operators and the proximal point algorithm,SIAM J. Control Optim.14 (1976), 877–898
1976
-
[29]
R. T. Rockafellar, Augmented Lagrangians and applications of the proximal point algorithm in convex programming,Math. Oper. Res.1 (1976), 97–116
1976
-
[30]
Saad and M
Y. Saad and M. H. Schultz, GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems,SIAM J. Sci. Statist. Comput.7 (1986), 856–869
1986
-
[31]
Scieur, A
D. Scieur, A. d’Aspremont, and F. Bach, Regularized nonlinear acceleration,Math. Program. 179 (2020), 47–83. 25
2020
-
[32]
Toth and C
A. Toth and C. T. Kelley, Convergence analysis for Anderson acceleration,SIAM J. Numer. Anal.53 (2015), 805–819
2015
-
[33]
Tran-Dinh and Y
Q. Tran-Dinh and Y. Luo, Halpern-type accelerated and splitting algorithms for monotone inclusions, arXiv:2110.08150 (2021)
2021 arXiv
-
[34]
H. F. Walker and P. Ni, Anderson acceleration for fixed-point iterations,SIAM J. Numer. Anal. 49 (2011), 1715–1735
2011
-
[35]
Wittmann, Approximation of fixed points of nonexpansive mappings,Arch
R. Wittmann, Approximation of fixed points of nonexpansive mappings,Arch. Math.58 (1992), 486–491
1992
-
[36]
Yoon and E
T. Yoon and E. K. Ryu, Accelerated algorithms for smooth convex-concave minimax problems withO(1/k 2) rate on squared gradient norm,Proc. ICML(2021), 12098–12109
2021
-
[37]
Zhang, B
J. Zhang, B. O’Donoghue, and S. Boyd, Globally convergent type-I Anderson acceleration for nonsmooth fixed-point iterations,SIAM J. Optim.30 (2020), 3170–3197. 26
2020
Reviewed July 31, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.