REVIEW 4 major objections 5 minor 1 cited by
A Steepest Gradient Method with Nonmonotone Adaptive Step-sizes for the Nonconvex Minimax and Multi-Objective Optimization Problems
T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read A line-search-free steepest-gradient method with nonmonotone adaptive step sizes is claimed to reach stationary points for nonconvex minimax, and global minima or weakly efficient points under stronger convexity.
desk verdict A nicely motivated adaptive step-size scheme for minimax problems, but the convergence proof conflates the admissible step-size bound with the actual step sizes; the main theorems are unproven as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the subsidiary problem (SP): at each iterate $\theta_k$, choose $p_k$ and $\beta_k$ to minimize $\beta+\frac{1}{2}\|p\|^2$ subject to $\langle \nabla g_i(\theta_k),p\rangle+g_i(\theta_k)-\beta\le 0$ for indices in the $\delta$-active set $J_\delta(\theta_k)$. Its KKT conditions give $p_k+\sum_i u_i \nabla g_i(\theta_k)=0$ with $\sum_i u_i=1$, so if $p_k\to 0$ then $0$ lies in the convex hull of the active gradients, which is the Clarke stationary condition for $G(\theta)=\max_i g_i(\theta)$. The second mechanism is the step-size rule: a success (sufficient descent with tolerance $\varepsilon$) increases $\alpha_k$ by $\eta_k\sigma^s$, while a failure multiplies it by $\sigma$ and the counter $s$ advances. The summability of $\eta_k$ is used to show failures occur only finitely often, so $\alpha_k$ cannot be driven to zero; the proof then uses this non-vanishing of the step size to conclude that the directions $p_k$ vanish and stationarity follows.
What would settle it
Check Lemma 3.2 directly: the proof shows that the infimum of the allowable step sizes $\alpha_k$ is positive, which only gives an upper bound on $\|p_k\|$, and then asserts this means $\|p_k\|^2\to 0$. Exhibit any sequence of iterates satisfying the subsidiary problem and the step rule with $\|p_k\|$ constant, for instance directions alternating between $e_1$ and $-e_1$, while $\alpha_k$ stays bounded below; then the lemma's asserted implication fails and the stationarity theorem lacks its key premise.
Extended reading notes
Core claim
The paper's central claim is that the search directions $p_k$ produced by the subsidiary quadratic program tend to zero, and that this forces every accumulation point $\theta^*$ of the iterates to satisfy the stationarity system $\sum_{i=1}^m u_i \nabla g_i(\theta^*)=0$ with $\sum_{i=1}^m u_i=1$ and $u_i(g_i(\theta^*)-G(\theta^*))=0$. In the quasiconvex case the argument is meant to show the full sequence converges to a stationary point, and in the pseudoconvex case the stationarity condition upgrades to $G(\theta^*)\le G(\theta)$ for all $\theta$, i.e. a global minimum of the minimax problem. Via the Tchebycheff scalarization $\min_\theta \max_i (g_i(\theta)-v_i)/d_i$, these same conclusions translate into Pareto criticality or weak efficiency for the multiobjective problem. The paper presents the nonmonotone step-size rule not as a heuristic but as a provable replacement for line search inside a classical steepest-descent minimax method.
Load-bearing premise
The convergence argument rests on the step from “the step size stays bounded away from zero” to “the search directions shrink to zero,” and the second claim does not follow from the first by itself.
Editorial extensions
If this is right
- Each iteration costs one function evaluation for the step-size test, removing line-search overhead from a classical steepest-descent minimax algorithm.
- For differentiable nonconvex components, any accumulation point of the iterates satisfies the Clarke stationary condition for the max function $G$.
- For quasiconvex components, the full sequence converges to a stationary point rather than merely having stationary cluster points.
- For pseudoconvex components, the limit is a global minimum of the minimax problem and a weakly efficient point of the corresponding multiobjective problem.
- Sweeping the reference vector in the Tchebycheff formulation yields a family of such points, giving a reference-based approximation of the Pareto front.
Reading between the lines
- Editorial inference: if the step-size rule does force $p_k\to 0$, the same rule could plausibly be adapted to nonsmooth objectives, since the Clarke calculus used in the paper already accommodates locally Lipschitz max functions; the paper lists nonsmooth extension as future work.
- Editorial inference: the method's single-evaluation step update resembles heuristics used in large-scale training, but the paper's analysis is deterministic; testing the rule on stochastic or mini-batch losses would show whether the convergence guarantee survives noise.
- Editorial inference: the proof gap in Lemma 3.2 suggests a concrete experiment—run the algorithm while recording $\alpha_k$ and $\|p_k\|$ on a nonconvex problem; if $\|p_k\|$ fails to tend to zero while $\alpha_k$ stays bounded below, the stationarity theorem needs an additional assumption or a modified step rule.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a line-search-free steepest descent method with a nonmonotone adaptive step size (Algorithm 1) for the finite minimax problem (MP), and a reference-based extension (Algorithm 2) to multiobjective optimization via Tchebycheff scalarization. It claims convergence to a stationary point under nonconvex and quasiconvex assumptions, to a global minimum under pseudoconvexity, and correspondingly to Pareto critical or weakly efficient points of (MOP). Assumptions A1-A3 are stated, and numerical experiments on three small problems are reported.
Significance. If valid, this would be a useful contribution: a line-search-free adaptive step-size rule with convergence guarantees is attractive for large-scale and machine-learning settings, and approximating the Pareto front by solving a series of minimax problems is practical. The authors state their assumptions up front, do not fit parameters to the numerical examples, and the experiments illustrate the intended behavior. However, the proof of the central convergence theorem is not sound, so the advertised guarantees are not established.
major comments (4)
- [Section 3.3, Lemma 3.2] This lemma is load-bearing, and its proof conflates two different objects. The proof defines the allowable upper bound alpha_k = min{1, delta/(||p_k||(K + 1/2 ||p_k||)), (1/2 - epsilon)/L} and shows that inf_k alpha_k > 0 under the stated assumptions. That is a fact about an admissible step-size bound, not about the step-size sequence actually generated by Algorithm 1. The concluding sentence "alpha_k is not converge to 0, which means ||p_k||^2 goes to 0" is a non-sequitur: a positive lower bound on the allowable step does not force the actual step sizes away from zero, since a failure halves alpha, and it does not force p_k to vanish, since a bounded sequence p_k with constant nonzero norm yields a positive infimum for this allowable bound. The assertion that existence of a solution to (MP) implies ||p_k|| not -> infinity is also not justified. Consequently, the basis for Lemma 3.3 and for the conclusion p_k -> 0 in Theorems 3.5-3.7 is missing.
- [Section 3.3, Lemma 3.3] The induction proving that only finitely many iterations fail is algebraically wrong. In the success case the proof derives alpha_{l+1} <= sigma^l(alpha_0 + sum_{i=0}^l eta_i) but then claims the right-hand side is bounded by sigma^{l+1}(alpha_0 + sum_{i=0}^l eta_i); since sigma in (0,1), the latter is strictly smaller, so the inequality does not follow. If, instead, the index l is meant to be the number of failures rather than an iteration counter, then the induction step does not handle a success at all, because a success does not increase the failure count. Under either reading, the lemma's conclusion that sum I(...) < infinity is unsupported. The proof also uses s both as an iteration count and as a failure count without clarification.
- [Section 3.3, Theorem 3.5] The proof does not establish stationarity. Even assuming the previously unproved fact alpha_k ||p_k|| -> 0, the claim that {theta_k} is Cauchy does not follow; consecutive increments tending to zero do not imply that ||theta_m - theta_k|| < epsilon for all large m,k. The proof then simply asserts that the limit point theta* is stationary without using the KKT conditions of the subsidiary problem (SP) from Lemma 3.4 or any stationarity criterion. This is the main nonconvex convergence claim, and it is not proven.
- [Sections 3.3 and 4.5, Theorems 3.6, 3.7, 4.6] The quasiconvex, pseudoconvex, and multiobjective results all inherit the dependence on the unproved p_k -> 0 and on Lemma 3.3's finite-failure conclusion. Theorem 3.6, for instance, uses Lemma 3.3 to infer sum alpha_k epsilon ||p_k||^2 < infinity; since Lemma 3.3 is not established, the summability argument collapses. Theorem 4.6 then transfers these unsupported conclusions to Pareto criticality and weak efficiency. Thus the central claims of the paper are not supported by the written proofs.
minor comments (5)
- [Algorithm 1, lines 3-8] The scope of the statement 's = s + 1' is ambiguous; if it is meant to execute on every iteration, it should be placed after 'end if', while if it belongs to the else branch, that should be made explicit. The proof's counting of s must match the pseudocode.
- [Algorithm 1, Step 2] The subsidiary problem (SP) is not fully specified: it should state that the minimization is over (beta, p) in R x R^n, and the variables in the constraint should be written consistently.
- [Notation in Lemmas 3.2 and 3.3] alpha_k denotes both the actual step size and the allowable bound in Lemma 3.2, and k denotes both an iteration index and a failure count in Lemma 3.3; this overloading makes the proofs difficult to follow.
- [Theorem 4.6 proof] The text 'From Lemma 3.7' should be 'From Theorem 3.7', since there is no Lemma 3.7 in the manuscript.
- [Section 5, Numerical Results] Figures 1-3 alone do not verify the convergence claims; the section should report objective values over iterations, stopping criteria, and comparisons with existing methods.
Circularity Check
No circularity: the convergence claims do not reduce to fitted constants or to self-citations; the main defect is a non-sequitur in Lemma 3.2, which is a correctness gap, not circular reasoning.
full rationale
The paper's derivation chain is input-driven rather than output-equivalent. Assumptions (A1)-(A3) are stated up front: existence of a solution, bounded level set, and Lipschitz gradients; these are not disguised forms of the stationarity conclusion. The step-size rule in Algorithm 1 is adaptive, but no parameter is fitted to the data used in the numerical experiments, and the examples do not back-propagate into the proofs. The only self-citation is [23], used in a background sentence about Tchebycheff scalarization with directional concessions; it is not invoked in Lemma 3.2, Theorem 3.5, Theorem 3.6, or Theorem 3.7, so it is not load-bearing. The serious internal problem is a logical gap in Lemma 3.2: after proving a positive infimum for the allowable bound alpha_k := min{1, delta/(||p_k||(K+||p_k||/2)), (1/2-epsilon)/L}, the proof asserts 'alpha_k is not converge to 0, which means ||p_k||^2 goes to 0'. A positive lower bound on an admissible step-size sequence does not imply the actual search directions tend to zero, so the stationarity conclusion is unsupported. Similarly, Theorem 3.5's Cauchy-type argument does not by itself establish 0 in the generalized subdifferential of G at theta*. These are non-sequiturs and correctness risks, but they are not circular steps: the theorems do not define stationarity in terms of alpha_k or p_k, nor do they fit a parameter to the target conclusion. Consequently, no step in the paper reduces by construction to its own inputs, and the appropriate circularity score is 0.
Assumptions & free parameters
free parameters (4)
- epsilon =
0.4 in experiments; in (0,1/2) in theory
- sigma =
0.9 in experiments; in (0,1) in theory
- eta_k =
0.01^k in experiments; positive summable sequence in theory
- delta =
not specified numerically
assumptions (5)
- domain assumption Assumption A1: Problem (MP) has a solution; A2: the level set Omega is bounded; A3: gradients are Lipschitz on Omega.
- standard math Theorem 2.6 from Mifflin: the Clarke subdifferential of a max of locally Lipschitz functions is contained in, or equal to, the convex hull of active subdifferentials under semiconvexity or quasidifferentiability.
- standard math Proposition 2.3: quasiconvex locally Lipschitz functions are Clarke-subdifferentially quasiconvex.
- domain assumption The subsidiary problem (SP) is solvable at every iteration and its KKT conditions hold with multipliers u_i.
- ad hoc to paper The proof requires that s in Algorithm 1 counts failed iterations only.
Cite this review
Pith. "Pith review of A Steepest Gradient Method with Nonmonotone Adaptive Step-sizes for the Nonconvex Minimax and Multi-Objective Optimization Problems." pith.science (2026). https://pith.science/paper/DVFY5ZWT
@misc{pith2026250202010,
author = {Pith},
title = {Pith review of: A Steepest Gradient Method with Nonmonotone Adaptive Step-sizes for the Nonconvex Minimax and Multi-Objective Optimization Problems},
year = {2026},
howpublished = {\url{https://pith.science/paper/DVFY5ZWT}},
note = {Machine review of arXiv:2502.02010}
}
read the original abstract
This paper proposes a new steepest gradient descent method for solving nonconvex finite minimax problems using non-monotone adaptive step sizes and providing proof of convergence results in cases of the nonconvex, quasiconvex, and pseudoconvex differentiate component functions. The proposed method is applied using a referenced-based approach to solve the nonconvex multiobjective programming problems. The convergence to weakly efficient or Pareto stationary solutions is proved for pseudoconvex or quasiconvex multiobjective optimization problems, respectively. A variety of numerical experiments are provided for each scenario to verify the correctness of the theoretical results corresponding to the algorithms proposed for the minimax and multiobjective optimization problems.
Figures
Forward citations
Cited by 1 Pith paper
-
Barzilai-Borwein Diagonal Quasi-Newton Method for Nonconvex Multiobjective Optimization Problems
A diagonal Barzilai-Borwein quasi-Newton method with one shared scalar parameter converges to Pareto-critical points for smooth nonconvex multiobjective problems and scales to dimension 2000 in tests.
Reference graph
Works this paper leans on
-
[1]
Advances in cone-based preference modeling for decision making with mul- tiple criteria
Wiecek MM. Advances in cone-based preference modeling for decision making with mul- tiple criteria. Decision Making in Manufacturing and Services. 2007;1:153–173. 18
work page 2007
-
[2]
Finance-based scheduling multi-objective opti- mization: Benchmarking of evolutionary algorithms
El-Abbasy MS, Elazouni A, Zayed T. Finance-based scheduling multi-objective opti- mization: Benchmarking of evolutionary algorithms. Automation in Construction. 2020; 120:103392
work page 2020
-
[3]
Robust multiobjective optimization & applications in portfolio opti- mization
Fliege J, Werner R. Robust multiobjective optimization & applications in portfolio opti- mization. European Journal of Operational Research. 2014;234(2):422–433
work page 2014
-
[4]
Multicriteria design optimization: procedures and applications
Eschenauer H, Koski J, Osyczka A. Multicriteria design optimization: procedures and applications. Springer Science & Business Media; 2012
work page 2012
-
[5]
Jimenez H, Mavris D. Pareto-optimal aircraft technology study for environmental benefits with multi-objective optimization. Journal of Aircraft. 2017;54(5):1860–1876
work page 2017
-
[6]
Numerical methods in extremal problems
Pshenichny BN, Danilin YM. Numerical methods in extremal problems. Mir Publishers; 1978
work page 1978
-
[7]
An efficient method to solve the minimax problem directly
Charalambous C, Conn A. An efficient method to solve the minimax problem directly. SIAM Journal on Numerical Analysis. 1978;15(1):162–187
work page 1978
-
[8]
A globally convergent qp-free algorithm for inequality constrained minimax optimization
Jian J, Ma G. A globally convergent qp-free algorithm for inequality constrained minimax optimization. Acta Mathematica Scientia. 2020;40(6):1723–1738
work page 2020
Show all 32 references
-
[9]
Primal interior-point method for large sparse minimax optimization
Lukˇ san L, Matonoha C, Vlcek J. Primal interior-point method for large sparse minimax optimization. Technical report; 2005
2005
-
[10]
Interior-point method for non-linear non-convex opti- mization
Lukˇ san L, Matonoha C, Vlˇ cek J. Interior-point method for non-linear non-convex opti- mization. Numerical linear algebra with applications. 2004;11(5-6):431–453
2004
-
[11]
An interior-point algorithm for nonconvex nonlinear program- ming
Vanderbei RJ, Shanno DF. An interior-point algorithm for nonconvex nonlinear program- ming. Computational Optimization and Applications. 1999;13:231–252
1999
-
[12]
A non-monotone conjugate subgradient type method for minimization of convex functions
Konnov I. A non-monotone conjugate subgradient type method for minimization of convex functions. Journal of Optimization Theory and Applications. 2020;184(2):534–546
2020
-
[13]
On solving large-scale finite minimax problems using exponential smoothing
Pee E, Royset JO. On solving large-scale finite minimax problems using exponential smoothing. Journal of optimization theory and applications. 2011;148(2):390–421
2011
-
[14]
An algorithm based on active sets and smoothing for discretized semi-infinite minimax problems
Polak E, Womersley R, Yin H. An algorithm based on active sets and smoothing for discretized semi-infinite minimax problems. Journal of Optimization Theory and Appli- cations. 2008;138:311–328
2008
-
[15]
On the convergence of the pshenichnyi-pironneau-polak minimax algorithm with an active set strategy
Polak E. On the convergence of the pshenichnyi-pironneau-polak minimax algorithm with an active set strategy. Journal of optimization theory and applications. 2008;138:305–309
2008
-
[16]
Steepest descent methods for multicriteria optimization
Fliege J, Svaiter BF. Steepest descent methods for multicriteria optimization. Mathemat- ical methods of operations research. 2000;51:479–494
2000
-
[17]
Nonlinear conjugate gradient methods for vector opti- mization
Lucambio P´ erez L, Prudente L. Nonlinear conjugate gradient methods for vector opti- mization. SIAM Journal on Optimization. 2018;28(3):2690–2720
2018
-
[18]
Extended newton methods for multiobjective op- timization: majorizing function technique and convergence analysis
Wang J, Hu Y, Wai Yu CK, et al. Extended newton methods for multiobjective op- timization: majorizing function technique and convergence analysis. SIAM Journal on Optimization. 2019;29(3):2388–2421
2019
-
[19]
Convergence of the projected gradient method for quasiconvex multiobjective optimization
Cruz JB, P´ erez LL, Melo J. Convergence of the projected gradient method for quasiconvex multiobjective optimization. Nonlinear Analysis: Theory, Methods & Applications. 2011; 74(16):5268–5273
2011
-
[20]
A projected gradient method for vector optimization prob- lems
Drummond LG, Iusem AN. A projected gradient method for vector optimization prob- lems. Computational Optimization and applications. 2004;28:5–29
2004
-
[21]
A new nonmonotone line search method for nonsmooth nonconvex optimization
Akbari Z. A new nonmonotone line search method for nonsmooth nonconvex optimization. Optimization. 2024;73(2):429–441
2024
-
[22]
Interactive multiple criteria decision making based on preference driven evolutionary multiobjective optimization with controllable accuracy
Kaliszewski I, Miroforidis J, Podkopaev D. Interactive multiple criteria decision making based on preference driven evolutionary multiobjective optimization with controllable accuracy. European Journal of Operational Research. 2012;216(1):188–199
2012
-
[23]
A monotonic optimization approach for solving strictly quasiconvex multiobjective programming problems
Thang TN, Solanki VK, Dao TA, et al. A monotonic optimization approach for solving strictly quasiconvex multiobjective programming problems. Journal of Intelligent & Fuzzy Systems. 2020;38(5):6053–6063
2020
-
[24]
Generalized convexity of functions and generalized monotonicity of set-valued maps
Penot JP, Quang P. Generalized convexity of functions and generalized monotonicity of set-valued maps. Journal of Optimization Theory and Applications. 1997;92:343–356
1997
-
[25]
Semismooth and semiconvex functions in constrained optimization
Mifflin R. Semismooth and semiconvex functions in constrained optimization. SIAM Jour- nal on Control and Optimization. 1977;15(6):959–972. 19
1977
-
[26]
Disciplined quasiconvex programming
Agrawal A, Boyd S. Disciplined quasiconvex programming. Optimization Letters. 2020; 14:1643–1657
2020
-
[27]
Iterative algorithms for nonlinear operators
Xu HK. Iterative algorithms for nonlinear operators. Journal of the London Mathematical Society. 2002;66(1):240–256
2002
-
[28]
Efficient computation of the search re- gion in multi-objective optimization
D¨ achert K, Klamroth K, Lacour R, et al. Efficient computation of the search re- gion in multi-objective optimization. European Journal of Operational Research. 2017; 260(3):841–855
2017
-
[29]
Multiobjective optimization through a series of single- objective formulations
Audet C, Savard G, Zghal W. Multiobjective optimization through a series of single- objective formulations. SIAM Journal on Optimization. 2008;19(1):188–210
2008
-
[30]
Multicriteria optimization
Ehrgott M. Multicriteria optimization. Vol. 491. Springer Science & Business Media; 2005
2005
-
[31]
Convergence of a nonmonotone projected gradient method for nonconvex multiobjective optimization
Zhao X, Jolaoso LO, Shehu Y, et al. Convergence of a nonmonotone projected gradient method for nonconvex multiobjective optimization. J Nonlinear Var Anal. 2021;5:441–457
2021
-
[32]
Pareto multi-task learning
Lin X, Zhen HL, Li Z, et al. Pareto multi-task learning. Advances in neural information processing systems. 2019;32. 20
2019
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.