Pith. sign in

REVIEW 3 major objections 6 minor 40 references

Numerical Design of Optimized First-Order Algorithms

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper's central claim: a sequential linearization method designs step-size schedules that beat a branch-and-bound benchmark and accelerate several first-order descent methods beyond classical rates.

desk verdict A genuinely useful numerical design method (SLM) for fixed-step first-order algorithms, with exciting but unproven acceleration claims and a load-bearing benchmark comparison that needs code and data before it can be believed. read the letter →

arxiv 2507.20773 v1 pith:C2DUP6UJ submitted 2025-07-28 math.OC

Yassine Kamri , Julien M. Hendrickx , François Glineur This is my paper · ORCID
classification math.OC MSC 90C2590C2265K05
keywords PerformanceEstimationProblemfirst-ordermethodsstep-sizeoptimizationsequentiallinearizationsemidefiniteprogrammingcoordinatedescentinexactgradientacceleration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the worst-case behavior of fixed-step first-order optimization algorithms can be turned into a design tool: solve a sequence of semidefinite programs that linearize the algorithm-design problem, and the resulting step-size schedules are better than those produced by the branch-and-bound benchmark. On the benchmark task of memoryless gradient descent over smooth convex functions, the sequential linearization method (SLM) matches or beats the published BNB-PEP values, which the paper reads as evidence that those published values are not truly optimal. The same pipeline applied to cyclic coordinate descent, inexact gradient descent, and cyclic gradient descent produces worst-case upper bounds that the paper fits to accelerated rates, such as $O(1/N^{1.145})$ for two-block memoryless cyclic coordinate descent. If these bounds are correct, careful step-size design alone can accelerate first-order methods in settings where only $O(1/N)$ or a slower linear rate was previously known.

What carries the argument

The central machinery is the Performance Estimation Problem (PEP), a reformulation of worst-case convergence analysis as a semidefinite program in the Gram matrix of iterates and gradients, together with the sequential linearization method (SLM). PEP replaces the infinite-dimensional choice of a worst-case function by interpolation conditions on a finite set of points, and the design problem is written as a min-max problem whose inner maximization is dualized into a nonlinear conic constraint $A(\tau,\lambda,\alpha)\succeq 0$. SLM iteratively linearizes that constraint around the current step sizes and dual multipliers, solves the resulting tangent SDP with a trust-region bound on the update, and accepts or rejects the update according to the ratio of actual to predicted improvement. This machinery converts algorithm design into a sequence of convex problems, which is why it can be applied to any SDP-representable function class.

What would settle it

Solve the PEP for the SLM-designed two-block memoryless cyclic coordinate descent at much larger $N$, say $N=200$, and for the cyclic gradient descent at many multiples of the 4-step cycle; if the coordinate upper bounds stop tracking $1/N^{1.145}$ or the per-cycle contraction factor rises above $0.14239$, the claimed acceleration is refuted.

Watch

Extended reading notes

Core claim

The central discovery is that a sequential linearization method can solve the nonconvex algorithm-design problem that arises when one dualizes a Performance Estimation Problem: at each iteration one linearizes the nonlinear matrix inequality $A(\tau,\lambda,\alpha)\succeq 0$ around the current step sizes and dual variables and solves a tangent semidefinite program inside a trust region. The paper reports that on the benchmark of optimizing memoryless gradient descent step sizes over L-smooth convex functions, SLM produces worst-case upper bounds that are at most the BNB-PEP values at every listed $N$ up to 50, including $f(x_{19})-f_*\le 0.004869$ against BNB-PEP's $0.004941$, and therefore that BNB-PEP's claimed optima up to $N=25$ are not optimal. For new settings, SLM-designed steps give fits $O(1/N^{1.145})$ for two-block memoryless cyclic coordinate descent, $O(1/N^{1.47})$ for full cyclic coordinate descent, rates from $O(1/N^{1.018})$ to $O(1/N^{1.116})$ for memoryless inexact gradient descent on the squared gradient norm, and a cyclic gradient descent that satisfies $\|x_{4N}-x_*\|^2\le 0.14239^N\|x_0-x_*\|^2$ on $1$-smooth $0.1$-strongly convex functions, improving the $0.2008^N$ bound of the optimal constant step size.

Load-bearing premise

The load-bearing premise is that the fitted formulas over a short range of iteration counts reveal the true long-run convergence rates; the paper also assumes, on numerical evidence only, that its reformulated design problems give exact worst-case values rather than loose relaxations.

Editorial extensions

If this is right

  • For memoryless gradient descent over smooth convex functions, SLM produces worst-case upper bounds that beat BNB-PEP's published values at every listed $N$ up to 50, so those published values are not genuinely optimal as claimed up to $N=25$.
  • Memoryless cyclic coordinate descent with SLM-tuned steps converges faster than $O(1/N)$ even with two blocks and no memory of past gradients; the numerical fit is $O(1/N^{1.145})$.
  • On $1$-smooth $0.1$-strongly convex functions, cyclic gradient descent with SLM steps contracts the squared error by a factor $0.14239$ per 4-step cycle, against $0.2008$ per 4 steps for the optimal constant step size.
  • Inexact gradient descent with optimized steps beats the $O(1/N)$ bound of constant-step inexact gradient descent for the squared gradient norm when inexactness is small, and using past inexact gradients increases the fitted exponent further.
  • The design pipeline applies to any SDP-representable function class, so the same numerical method can be reused to tune fixed-step algorithms for other oracle models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the finite-$N$ fits reveal true asymptotics, momentum-free acceleration is a general step-size phenomenon rather than a quirk of gradient descent; the same pipeline should be tried on proximal, randomized, and decentralized variants to see where the acceleration survives.
  • The paper's Table 1 suggests that published BNB-PEP optimal values are systematically slightly too high; re-running other BNB-PEP-designed algorithms with SLM could uncover similar gaps in published benchmark tables.
  • The SLM step sizes in Table 2 resemble but do not match the composed schedules discussed in [4,20], so finding an analytic description of these schedules could turn a numerical discovery into a proof.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This manuscript studies the computer-assisted design of fixed-step first-order optimization methods through the Performance Estimation Problem (PEP) framework. It develops a generic SDP formulation for evaluating a fixed algorithm and a nonconvex design formulation, then proposes three numerical heuristics: Alternating Minimization, a first-order method, and a Sequential Linearization Method (SLM). The methods are benchmarked on the task of optimizing step sizes for memoryless gradient descent over smooth convex functions, with Table 1 claiming that SLM outperforms the published BNB-PEP values on many horizons and is computationally more efficient. The paper then applies SLM to design step sizes for cyclic coordinate descent, inexact gradient descent, full-memory variants, and cyclic gradient descent on strongly convex functions, reporting accelerated numerical rates such as O(1/N^1.145) for two-block memoryless cyclic coordinate descent and a linear contraction factor of 0.14239^N for a four-step cyclic gradient method. The authors are explicit that the rates in Section 5 are numerical fits over finite N and label several of them as conjectures, but the abstract and Section 4 make stronger claims of superiority over BNB-PEP.

Significance. If the results are correct, the paper offers a flexible numerical toolbox for algorithm design that goes beyond known analytical constructions and can produce nontrivial step-size schedules for settings where no analytic optimal schedule is known. The PEP upper bounds for smooth strongly convex functions are grounded in exact interpolation conditions, and the manuscript is honest in labeling the Section 5 rates as fitted numerical evidence rather than proven theorems. The applications to cyclic coordinate descent and inexact gradient descent are interesting and likely to stimulate further work. However, the strongest claims are numerical and are not independently reproducible: no code, data, solver settings, or certificates are provided, and the Section 4 comparison against BNB-PEP is based on published values rather than a re-run. The reported upper bounds are also the same quantities minimized during design, which does not invalidate them as bounds but means the experiments do not independently certify optimality. These issues must be addressed before the paper's more ambitious claims can be accepted.

major comments (3)
  1. [Section 4, Table 1 and following paragraph] The claim that SLM outperforms BNB-PEP and that BNB-PEP's published values are not optimal is load-bearing but is not supported by reproducible evidence. No code, data, solver versions, tolerances, or certificates are provided, and BNB-PEP is not re-run; Table 1 compares an unreleased SLM implementation against values read from [21]. If both methods solve the same exact PEP, a local method cannot beat a correctly solved global method, so the discrepancy must be explained; if the formulations differ, the equivalence must be established. The manuscript attributes the discrepancy to 'numerical issues/bugs due to Gurobi branch-and-bound' without evidence. To make the claim stand, the authors should provide reproducibility artifacts and either a primal SDP certificate for the SLM bound below the BNB-PEP value or a corrected BNB-PEP run with reported tolerances.
  2. [Section 5, Remark 12 and Tables 6-9] The accelerated-rate claims (O(1/N^1.145) for memoryless cyclic coordinate descent, the rates in Tables 6-9, and the cyclic contraction 0.14239^N) are obtained by fitting the model 1/(alpha N^nu + beta) to PEP upper bounds over a small finite range of N, with no error bars, no sensitivity analysis, and no asymptotic verification. A three-parameter fit over N up to 30 or 40 can readily produce nu > 1 even when the true asymptotic rate is O(1/N), possibly with logarithmic factors. Since these rates are the main conclusions of Section 5, the authors should either provide evidence that the fitted exponents are stable as N grows and across fitting windows, or explicitly reframe the claims as finite-horizon numerical observations rather than convergence rates. This also affects the abstract's statement that accelerated convergence rates are reported in all cases.
  3. [Section 4, Tables 3-5 and Figure 3] The runtime and efficiency comparison conflates BNB-PEP with its local IPOPT first step. Tables 3-5 and Figure 3 compare SLM against the local QCQP step of BNB-PEP using IPOPT, not against the full branch-and-bound procedure, while the conclusion describes the comparison as being with the BNB-PEP approach. The full BNB-PEP runtimes cited from [21] are not measured in this paper, so the claim of computational advantage over BNB-PEP is not established by these experiments. The tables should be labeled as a local-solver comparison, and the runtime-related conclusions should be narrowed accordingly.
minor comments (6)
  1. [Section 3, Theorem 3.2] The gradient formula uses G, which Theorem 3.1 identifies as a solution of Problem (SDP-PEP-Generic), but the text says 'the unique (see remark 7) solution of Problem (dual-PEP-generic)'; this appears to be a typo and should be corrected.
  2. [Figures 6 and 7] The legends refer to 'Theorem 5.19' but the relevant statement is Theorem 5.3; the numbering should be fixed.
  3. [Algorithm 1] The output is written as alpha^{(N)}_{i,k} although the loop runs for T iterations; the output should be alpha^{(T)}_{i,k}.
  4. [Section 3 and Section 4] Remark 12 uses the symbol alpha both for the step sizes and for the coefficient in the fitted model 1/(alpha N^nu + beta); a different symbol, such as a or c, would avoid ambiguity.
  5. [Section 4 and Section 5] There are several typographical errors: 'initilizations' in the discussion of AM/FOM, 'T able' in several table captions, 'four our' in the full inexact gradient descent subsection, and 'forgoing' should be 'foregoing' in Remark 10.
  6. [Theorem 2.4] The statement says 'For any d in Z', but the dimension d of the ambient space should be a positive integer; if the bound is intended to be dimension-independent, that should be stated explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported rates are explicit numerical fits, and the design/evaluation loop is transparent rather than a derivation that reduces to its own inputs.

full rationale

The paper's derivation chain is a standard optimization loop: worst-case performance is formulated as a convex SDP (Theorem 2.4), dualized (dual-PEP-generic), and step sizes are designed by minimizing the dual upper bound (Algo-design-generic), solved by SLM. The same PEP bound is later used to evaluate the designed steps. This is self-referential only in the weak sense that the evaluation metric is the same as the design objective; it is not circular, because the PEP bound is a valid worst-case upper bound for the algorithm, and comparing it with the PEP bound of a classical schedule is a comparison of two legitimate bounds. The accelerated-rate claims (e.g., O(1/N^1.145) for cyclic coordinate descent) are not presented as first-principles predictions: Remark 12 explicitly states that the rates are obtained by solving the PEP over a range of N and fitting a model of the form 1/(αN^ν + β), and the statements are labeled as Conjectures 5.1 and 5.2 rather than theorems. The Section 4 claim that SLM outperforms BNB-PEP and that BNB-PEP's published values are therefore not optimal is not circular, but it depends on an unreproduced comparison against published numbers and on an asserted Gurobi branch-and-bound bug; that is a reproducibility/correctness risk, not a circularity. Self-citations to [38] and [23] supply background PEP interpolation conditions and coordinate-descent PEP formulations; they are technical tools with independent published or stated content, and no load-bearing conclusion reduces to an unverified self-citation. The paper also acknowledges its limitations (Remarks 1, 2, 3, 6), including that the SDP may be a relaxation, that strong duality is only observed experimentally, and that local methods have no global convergence guarantee, which further supports that no hidden circular construction is being used.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on optimized step sizes (free parameters) and on curve-fitted rate exponents. No new mathematical entities are postulated. The main assumptions are exactness of the PEP SDP for the classes studied, unproved strong duality, and the validity of extrapolating finite-N fits to asymptotic rates.

free parameters (4)
  • Optimized step sizes alpha_{i,k} = Tables 2 and optimized schedules for Sections 5
    Design variables chosen by numerical optimization (SLM) to minimize the PEP worst-case bound. No analytical closed form is given.
  • Fitted convergence exponents nu = 1.145, 1.47, 1.116, 1.384, 1.171, 1.450, 1.342, 1.190, 1.052, 1.275, 1.151, etc.
    Exponents from fitting 1/(alpha N^nu + beta) to finite-N PEP upper bounds. These exponents are the basis for the reported accelerated rates.
  • Fitted model coefficients alpha and beta = Not reported
    The curve fits use coefficient values that are not listed, so the rates cannot be independently reconstructed from the paper.
  • Cycle length N* = 4
    Hand-chosen horizon for the cyclic gradient descent experiment; the reported contraction rate depends on this choice.
assumptions (4)
  • domain assumption The functional classes considered are SDP-representable and the interpolation conditions used give exact worst-case PEP values when the paper claims improved performance.
    Definition 2.3 allows necessary-only conditions, and Remark 1 notes the resulting SDP may be a relaxation. Sections 4 and 5 treat the PEP values as exact worst-case bounds for the algorithms being compared.
  • domain assumption Strong duality holds for every explicit PEP formulation used.
    Remark 3 states strong duality holds experimentally, not by proof. The dual-based algorithm design problem relies on this to avoid further relaxation.
  • ad hoc to paper Finite-N PEP upper bounds follow the model 1/(alpha N^nu + beta), so fitted exponents represent asymptotic convergence rates.
    Remark 12 defines the fitting procedure. The conjectured rates in Tables 6-9 and Figures 4-9 depend on this extrapolation.
  • domain assumption For cyclic gradient descent, the contraction bound computed over one optimized cycle transfers to every subsequent cycle under the same schedule.
    Section 5, Algorithm 9 and the concluding formulas assume the worst-case per-cycle contraction c(N*) applies uniformly to all starts, giving c(N*)^k over k cycles.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Numerical Design of Optimized First-Order Algorithms." pith.science (2026). https://pith.science/paper/C2DUP6UJ

@misc{pith2026250720773,
  author       = {Pith},
  title        = {Pith review of: Numerical Design of Optimized First-Order Algorithms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C2DUP6UJ}},
  note         = {Machine review of arXiv:2507.20773}
}
read the original abstract

We derive several numerical methods for designing optimized first-order algorithms in unconstrained convex optimization settings. Our methods are based on the Performance Estimation Problem (PEP) framework, which casts the worst-case analysis of optimization algorithms as an optimization problem itself. We benchmark our methods against existing approaches in the literature on the task of optimizing the step sizes of memoryless gradient descent (which uses only the current gradient for updates) over the class of smooth convex functions. We then apply our methods to numerically tune the step sizes of several memoryless and full (i.e., using all past gradient information for updates) fixed-step first-order algorithms, namely coordinate descent, inexact gradient descent, and cyclic gradient descent, in the context of linear convergence. In all cases, we report accelerated convergence rates compared to those of classical algorithms.

Figures

Figures reproduced from arXiv: 2507.20773 by the authors.

Figure 1
Figure 1. AM iterates are not guaranteed to converge to a minimum First-order method (FOM). Our second approach is to apply a first-order method to the function w dual(α) directly. This requires computing the gradients, when well￾defined, of w dual with respect to the step sizes αi,k. Since the step sizes αi,k influence w dual(α) only through the vectors hi in the matrices Ai,j (α) (see Problem dual-PEP￾generic), computing th… view at source ↗
Figure 2
Figure 2. Worst-case convergence of memoryless gradient descent (MGD) with the optimized step-sizes given the BNB-PEP method [21], our Alternating Minimization (AM), First-Order method (FOM) and Successive Linearization method (SLM) 14 [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
Figure 3
Figure 3. Worst-case convergence of memoryless gradient descent (MGD) with the optimized step sizes obtained using the BNB-PEP method with the local solver IPOPT, and with the step sizes from our Successive Linearization Method (SLM) under the limitation of 100 optimization steps [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Upper bounds on the worst-case convergence rate of memoryless cyclic coordinate descent with classical steps 1 Lℓ (yellow) and with our optimized step-sizes (blue). Conjecture 5.1. Memoryless cyclic coordinate descent with optimized step sizes can achieve a faster conv…
Figure 5
Figure 5. Figure 5: Comparison of upper bounds on the worst-case convergence rate of our optimized full cyclic coor￾dinate descent (blue) and optimized memoryless gradient descent (red). Conjecture 5.2. Full cyclic coordinate descent with optimized step sizes can achieve a faster converge…
Figure 6
Figure 6. Figure 6: Comparison of worst-case performance of our optimized inexact gradient methods compared to the inexact gradient descent with step-size α (0) i = 1/L for three different levels of inexactness ϵ. Conjecture 5.4. Inexact gradient descent with optimized step sizes can achi…
Figure 7
Figure 7. Figure 7: Comparison of upper bounds on the worst-case performance of our optimized full inexact gradient methods compared to the inexact gradient descent with constant step sizes (upper bound from Theorem 5.3) for three different levels of inexactness ϵ. Conjecture 5.5. Full In…
Figure 8
Figure 8. Figure 8: Comparison of the worst-case performance of our optimized inexact gradient methods compared to the inexact gradient descent with constant step sizes α (0) i = 1/L for three different levels of inexactness ϵ. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_8.png]
Figure 9
Figure 9. Figure 9: Comparison of worst-case performance of our optimized full inexact gradient methods, (FIGD), compared to the inexact gradient descent with constant step sizes α (0) i = 1/L, (IGD), for three different levels of inexactness ϵ. Design of cyclic gradient descent algorithm…
Figure 10
Figure 10. Figure 10: Optimized cyclic steps for gradient descent for 1-smooth 0.1-strongly convex functions. Conclusions. Our optimized cyclic steps improve the worst-case performance of gradi￾ent descent on smooth strongly convex functions. To quantify the performance gap, we provide num…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 38 canonical work pages

  1. [21]

    Van Parys, and Ernest K

    Shuvomoy Das Gupta, Bart P.G. Van Parys, and Ernest K. Ryu. Branch-and-bound performance estimation programming: A unified methodology for constructing optimal optimization methods. Mathematical Programming, 204:567–639, 2024

    work page 2024

  2. [1]

    Abbaszadehpeivasti, E

    H. Abbaszadehpeivasti, E. de Klerk, and M. Zamani. Conditions for linear convergence of the gradient method for non-convex optimization. Optimization Letters, 17:1105–1125, 2023

    work page 2023

  3. [2]

    Katyusha: The first direct acceleration of stochastic gradient methods

    Zeyuan Allen-Zhu. Katyusha: The first direct acceleration of stochastic gradient methods. The Journal of Machine Learning Research (JMLR) , 18(1):8194–8244, 2017

    work page 2017

  4. [3]

    Altschuler and Pablo A

    Jason M. Altschuler and Pablo A. Parrilo. Acceleration by stepsize hedging i: Multi-step descent and the silver stepsize schedule, 2023

    work page 2023

  5. [4]

    Altschuler and Pablo A

    Jason M. Altschuler and Pablo A. Parrilo. Acceleration by stepsize hedging: Silver stepsize schedule for smooth convex optimization. Mathematical Programming, 2024

    work page 2024

  6. [5]

    Taylor, and Francis Bach

    Mathieu Barr´ e, Adrien B. Taylor, and Francis Bach. Principled analyses and design of first-order methods with inexact proximal operators. Mathematical Programming, 201(1- 2):185–230, 2023

    work page 2023

  7. [6]

    A fast iterative shrinkage-thresholding algorithm for linear inverse problems

    Amir Beck and Marc Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences , 2(1):183–202, 2009

    2009

  8. [7]

    Fr´ ed´ eric Bonnans and Alexander Shapiro.Perturbation Analysis of Optimization Prob- lems

    J. Fr´ ed´ eric Bonnans and Alexander Shapiro.Perturbation Analysis of Optimization Prob- lems. Springer Series in Operations Research and Financial Engineering. Springer, 2000

    work page 2000

Show all 40 references
  1. [8]

    Boyd and L

    S. Boyd and L. El Ghaoui. Alternating minimization algorithms for bilinear matrix inequalities. In Proceedings of the 1994 American Control Conference , pages 687–691. IEEE, 1994

    1994

  2. [9]

    A global algorithm for nonlinear semidefinite programming

    Rafael Correa and H´ ector Ram ´ ırez Cabrera. A global algorithm for nonlinear semidefinite programming. SIAM Journal on Optimization , 15(1):303–318, 2004

    2004

  3. [10]

    Performance estimation of the gradient method with fixed arbitrary step sizes

    Antoine Daccache. Performance estimation of the gradient method with fixed arbitrary step sizes. Master’s thesis, Universit´ e Catholique de Louvain, 2019

    2019

  4. [11]

    On the worst-case complexity of the gradient method with exact line search for smooth strongly convex functions

    Etienne De Klerk, Fran¸ cois Glineur, and Adrien B Taylor. On the worst-case complexity of the gradient method with exact line search for smooth strongly convex functions. Optimization Letters, 11:1185–1199, 2017

    2017

  5. [12]

    Yoel Drori and Adrien B. Taylor. Efficient first-order methods for convex minimization: A constructive approach. Mathematical Programming, 184:183–220, 2020

    2020

  6. [13]

    Performance of first-order methods for smooth convex minimization: A novel approach

    Yoel Drori and Marc Teboulle. Performance of first-order methods for smooth convex minimization: A novel approach. Mathematical Programming, 145(1–2):451–482, 2014

    2014

  7. [14]

    Worst-case functions for the gradient method with fixed variable step sizes

    Diego Eloi. Worst-case functions for the gradient method with fixed variable step sizes. Master’s thesis, Universit´ e Catholique de Louvain, 2022

    2022

  8. [15]

    Fares, D

    B. Fares, D. Noll, and P. Apkarian. Robust control via sequential semidefinite program- ming. SIAM Journal on Control and Optimization , 40(6):1791–1820, 2002. 29

    2002

  9. [16]

    Accelerated, parallel and proximal coordinate descent

    Olivier Fercoq and Peter Richt´ arik. Accelerated, parallel and proximal coordinate descent. SIAM Journal on Optimization , 25(4):1997–2023, 2015

    1997

  10. [17]

    On fundamental proof struc- tures in first-order optimization

    Baptiste Goujaud, Aymeric Dieuleveut, and Adrien Taylor. On fundamental proof struc- tures in first-order optimization. In 2023 62nd IEEE Conference on Decision and Control (CDC), pages 3023–3030, 2023

    2023

  11. [18]

    Taylor, and Aymeric Dieuleveut

    Baptiste Goujaud, Adrien B. Taylor, and Aymeric Dieuleveut. Optimal first-order meth- ods for convex functions with a quadratic upper bound. arXiv preprint arXiv:2205.15033, 2022

    2022 arXiv

  12. [19]

    Provably faster gradient descent via long steps

    Benjamin Grimmer. Provably faster gradient descent via long steps. SIAM Journal on Optimization, 34(3):2588–2608, 2024

    2024

  13. [20]

    Benjamin Grimmer, Kevin Shu, and Alex L. Wang. Composing optimized stepsize sched- ules for gradient descent, 2024

    2024

  14. [22]

    Uijeong Jang, Shuvomoy Das Gupta, and Ernest K. Ryu. Computer-assisted design of accelerated composite optimization methods: Optista. arXiv preprint arXiv:2305.15704 , 2023

    2023

  15. [23]

    Hendrickx, and Ion Necoara

    Yassine Kamri, Fran¸ cois Glineur, Julien M. Hendrickx, and Ion Necoara. On the worst- case analysis of cyclic block coordinate descent type algorithms, 2025

    2025

  16. [24]

    Donghwan Kim and Jeffrey A. Fessler. Optimized first-order methods for smooth convex minimization. Mathematical Programming, 159:81–107, 2016

    2016

  17. [25]

    Donghwan Kim and Jeffrey A. Fessler. Optimizing the efficiency of first-order methods for decreasing the gradient of smooth convex functions. Journal of Optimization Theory and Applications, 188(1):192–219, 2021

    2021

  18. [26]

    A universal catalyst for first-order optimization

    Hongzhou Lin, Julien Mairal, and Zaid Harchaoui. A universal catalyst for first-order optimization. In Advances in Neural Information Processing Systems (NIPS), pages 3384– 3392, 2015

    2015

  19. [27]

    Monteiro and Benar F

    Renato D. Monteiro and Benar F. Svaiter. An accelerated hybrid proximal extragradient method for convex optimization. SIAM Journal on Optimization , 23(2):1092–1125, 2013

    2013

  20. [28]

    Linear convergence of first order methods for non-strongly convex optimization

    Ion Necoara, Yurii Nesterov, and Fran¸ cois Glineur. Linear convergence of first order methods for non-strongly convex optimization. Mathematical Programming, 175:69–107, 2019

    2019

  21. [29]

    A. S. Nemirovsky and B. T. Polyak. The chebyshev iterative method for solving linear equations. USSR Computational Mathematics and Mathematical Physics , 24(4):189–194, 1984

    1984

  22. [30]

    Problem Complexity and Method Efficiency in Optimization

    Arkadii Nemirovsky and David Yudin. Problem Complexity and Method Efficiency in Optimization. Wiley, New York, 1983

    1983

  23. [31]

    A method of solving a convex programming problem with convergence rate o(1/k2)

    Yurii Nesterov. A method of solving a convex programming problem with convergence rate o(1/k2). Soviet Mathematics Doklady , 27:372–376, 1983

    1983

  24. [32]

    Efficiency of coordinate descent methods on huge-scale optimization prob- lems

    Yurii Nesterov. Efficiency of coordinate descent methods on huge-scale optimization prob- lems. SIAM Journal on Optimization , 22(2):341–362, 2012

    2012

  25. [33]

    Jongmin Park and Ernest K. Ryu. Exact optimal accelerated complexity for fixed-point iterations. In Proceedings of the 39th International Conference on Machine Learning , volume 162 of Proceedings of Machine Learning Research , pages 17420–17457. PMLR, July 2022

    2022

  26. [34]

    B. T. Polyak. Some methods of speeding up the convergence of iteration methods. USSR Computational Mathematics and Mathematical Physics , 4(5):1–17, 1964

    1964

  27. [35]

    Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization

    Shai Shalev-Shwartz and Tong Zhang. Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization. In Proceedings of the 31st International Confer- ence on Machine Learning (ICML) , pages 64–72, 2014

    2014

  28. [36]

    An optimal gradient method for smooth strongly convex minimization

    Adrien Taylor and Yoel Drori. An optimal gradient method for smooth strongly convex minimization. Mathematical Programming, 199:557–594, 2023

    2023

  29. [37]

    Adrien B. Taylor. Convex Interpolation and Performance Estimation of First-order Meth- 30 ods for Convex Optimization . PhD thesis, Universit´ e catholique de Louvain, 2017

    2017

  30. [38]

    Taylor, Julien M

    Adrien B. Taylor, Julien M. Hendrickx, and Fran¸ cois Glineur. Smooth strongly convex interpolation and exact worst-case performance of first-order methods. Mathematical Programming, 161(1-2):307–345, 2017

    2017

  31. [39]

    On the NP-hardness of solving bilinear matrix inequalities and simultaneous stabilization with static output feedback

    Onur Toker and Hitay Ozbay. On the NP-hardness of solving bilinear matrix inequalities and simultaneous stabilization with static output feedback. In 1995 American Control Conference (ACC), pages 2525–2526. IEEE, 1995

    1995

  32. [40]

    Convergence analysis of an inexact gradient method on smooth convex functions

    Pierre Vernimmen and Fran¸ cois Glineur. Convergence analysis of an inexact gradient method on smooth convex functions. In Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN) , Bruges, Belgium, 2024. 31

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.