REVIEW 3 major objections 5 minor 28 references
Hager-Zhang Conjugate Gradient Method for Set Optimization with Set-Valued Objective Map of Finite Cardinality
T0 review · 3 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read An HZ-type conjugate gradient method converges to stationarity for set optimization with finite-cardinality set-valued objectives, without requiring a finitely generated ordering cone or regularity at the solution.
desk verdict Useful extension of HZ-CG to set optimization, with genuinely proved descent results, but the main global-convergence theorem is deferred to another paper and misses a load-bearing boundedness step; warrants major revision rather than acceptance. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The scalarization functional φ(y) = sup{w^T y : w ∈ C} with C = {w ∈ K* : w^T e = 1} turns lower-set order inequalities into scalar inequalities. At each iterate, minimizing V_x(a,d) = max_j φ(∇f^{a_j}(x)^T d) + 1/2 ||d||^2 over the finite set of active index partitions yields the pair (a_k, u_k); the norm ||u_k|| measures nonstationarity. The HZ parameter β_k, built from differences of φ along the previous direction and safeguarded by μ > 1/2, guarantees a sufficient descent condition with constant 1 − 1/(2μ), which powers the summability result and the contradiction proof of convergence.
What would settle it
Run Algorithm 1 on the infinite-cone test problem of Example 5.1 from a different initial point in [−14, −7], tracking ||u_k|| to machine precision; if a positive lower bound on ||u_k|| (or a limit cycle with ||u_k|| ≥ ε) appears, the liminf-zero conclusion is false. As a smaller check, the scalar-vector instance F(x) = {(0,x)} with K = R_+^2 and descent direction d = −1 satisfies the sufficient-decrease inequality for every α > 0, so no curvature step exists unless the global bounded-below assumption is in force.
Extended reading notes
Core claim
The paper's central claim is that the HZ conjugate-gradient update, with the scalar parameter truncated at zero and a line search satisfying sufficient-decrease and curvature conditions, solves the lower-set preorder set optimization problem even when the ordering cone is not finitely generated and no regularity is imposed at the optimum. Under bounded level-set and Lipschitz assumptions, the generated sequence satisfies liminf_{k→∞} ||u_k|| = 0, meaning a subsequence of iterates becomes stationary. Because vector optimization is the p = 1 special case, the result simultaneously covers unconstrained vector optimization with no finite-generation or regularity restrictions.
Load-bearing premise
The proof that the line search always succeeds assumes there exists a fixed bounded set lying below every value F(x) in the lower-set order; without this global bounded-below condition, the algorithm may be undefined.
Editorial extensions
If this is right
- At every nonstationary iterate, the HZ-generated direction is a K-descent direction, so the method always finds a way to lower the set objective.
- A step length satisfying the strong line-search conditions (sufficient decrease and curvature) exists along a K-descent direction when the set-valued image is bounded below by a fixed bounded set in the lower-set order.
- The directions and step lengths satisfy a summability condition, which is the engine for asymptotic convergence to stationarity.
- Since p = 1 recovers vector optimization, the same guarantee applies to unconstrained vector optimization without finite generation of the ordering cone or regularity at the optimum.
- On the paper's test problems, the HZ variant reaches the stopping tolerance with fewer iterations and less runtime than the compared conjugate-gradient variants in most cases, with the paper noting some instances where it does not.
Reading between the lines
- The sufficient-descent proof is largely structural, so a natural testable extension is to port other conjugate-gradient parameter families to the same set-optimization setting without finite generation or regularity, using the same scalarization machinery.
- The line-search existence theorem relies on a global bounded-below set B, which is stronger than a bounded level set; a concrete extension is to replace B by a one-dimensional boundedness condition along each descent ray and check whether the maximal-step argument still yields a line-search step.
- The asymptotic convergence (liminf ||u_k|| = 0) does not identify a particular weak minimal point; a future refinement could combine the method with a restart or continuation strategy to force full convergence to a weak minimum rather than merely a stationary point.
- Because the scalarization depends on the chosen interior point e, the practical behavior of the algorithm is tied to e; comparing different choices of e on the infinite-cone example may reveal a selection criterion that improves the rate of convergence.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a nonlinear Hager-Zhang conjugate gradient method for unconstrained set optimization problems of the form F(x) = {f^1(x), ..., f^p(x)} with the lower set preorder, where K is a closed convex pointed solid cone. The authors introduce Wolfe conditions based on the Drummond-Svaiter scalarization, define an HZ conjugate parameter, prove a sufficient descent property and a Zoutendijk-type condition, and claim global convergence in the sense liminf_k ||u_k|| = 0. Numerical experiments compare the method with PRP and HS variants on several test problems, including a nonpolyhedral cone. The main convergence theorem is not proved in the manuscript but deferred to a prior vector-optimization result.
Significance. If the central convergence claim were fully established, the paper would be a useful contribution: it extends Hager-Zhang conjugate gradient methods from vector optimization to set optimization, and it does so without requiring a finite generator of the ordering cone or regularity at a solution. The explicit algorithm, the descent theorem, and the Zoutendijk-type theorem are valuable pieces, and the numerical experiments include a nonpolyhedral cone, which is a nice test of the advertised generality. However, the paper's main global-convergence result, Theorem 4.3, is not established as written, and the line-search existence theorem relies on assumptions that are not part of the convergence framework. These gaps are load-bearing, so the contribution is currently conditional.
major comments (3)
- [Section 4, Theorem 4.3] The proof of the main claim, liminf_k ||u_k||=0, is a single sentence: 'by contradiction exactly as in [12, Theorem 2].' The material prepared in Lemmas 4.3 and 4.4 only shows, under the contrary assumption ||u_k|| >= l_b, that sum 1/||d_k||^2 < infinity and sum ||r_k - r_{k-1}||^2 < infinity. It does not establish the boundedness of {||d_k||}, which in [12, Theorem 2] is a separate and essential step needed to obtain a contradiction. Without that bound, no contradiction follows. The conclusion section even states that '{||d_k||} is bounded (Theorem 4.3)', but Theorem 4.3 is the assertion being proved, not a result about {||d_k||}. This is a load-bearing omission: the central convergence claim is unsubstantiated in this manuscript.
- [Section 3, Theorem 3.1 and well-definedness of Algorithm 1] Theorem 3.1 assumes there exists a bounded set B with B <=_l F(x) for all x (stated as x in R^m, probably R^n). This is a global bounded-below assumption, stronger than Assumptions 4.1 and 4.3, and it is not included in the convergence theorem or in the well-definedness discussion. The well-definedness paragraph invokes Theorem 3.1 to assert existence of a Wolfe step at every iteration, but under the assumptions used for convergence this existence is not established. Additionally, the proof contains an invalid inference: from f_j(x+alpha_k d) - ... notin -K for all k, the text concludes the limit is not in -K. A limit of points outside a closed set need not lie outside that set; the later contradiction is obtained by showing the limit lies in -int K, so the argument can be repaired, but as written it is not rigorous.
- [Section 4, Lemma 4.4, Eq. (38)] The first displayed inequality in the proof of Lemma 4.4, sum 1/||d_k||^2 <= (1/l_b) sum ||u_k||^4/||d_k||^2, is false in general. For example, if l_b = 0.1 and ||u_k|| = l_b, the right-hand side is 0.1/||d_k||^2, which is smaller than 1/||d_k||^2. The intended estimate can be repaired by using 1/||d_k||^2 <= ||u_k||^4/(l_b^4 ||d_k||^2), yielding the later constant 1/l_b^4. As written, the proof of the first part of (38) is invalid, though repairable.
minor comments (5)
- [Section 3, Theorem 3.1] The statement says 'for all x in R^m'; this should be 'x in R^n'.
- [Section 4, Theorem 4.2] In the proof, 'hat S := S - J(F(x0))' mixes a set with a scalar; the intended quantity is J(S) - J(F(x0)). The subsequent division by sigma-1 is also confusing; the convergence of the negative series follows from bounded-belowness of its partial sums, and the argument should be rewritten.
- [Section 4, Lemma 4.2] Continuity of the mapping d_zeta is deferred to a proof in [5]. Since this continuity is used to ensure boundedness of u_k, a short self-contained argument would improve readability.
- [Section 4, Theorem 4.2] The proof writes 'sigma max_j phi(...) < max_j phi(...)' though the Wolfe condition gives '>='. The strict inequality is not needed; use '<=' throughout.
- [Section 3] The text contains the typo 'Hazer-Zhang'; it should be 'Hager-Zhang'.
Circularity Check
No substantive circularity; central lemmas are independently proved, but the conclusion's proof summary contains a self-referential citation and Theorem 4.3 is deferred.
-
self definitional
[Section 6 (Conclusion), paragraph summarizing convergence proof]
"Then, it has been reported that {∥dk∥} is bounded (Theorem 4.3)."
The proof of Theorem 4.3 is a contradiction argument: under the contrary assumption ∥uk∥ ≥ lb, Lemma 4.4 yields ∑ 1/∥dk∥^2 < ∞, which implies ∥dk∥ → ∞. To obtain the contradiction one needs an independent bound on {∥dk∥}. The conclusion supplies that bound by citing Theorem 4.3 itself, the very theorem being proved. As written, the boundedness of {∥dk∥} is not established anywhere in the paper; Lemma 4.4 proves the opposite. Thus the convergence claim is presented as following from a premise that is identical to the missing half of the theorem.
full rationale
The paper's core derivation chain is largely self-contained: Theorem 4.1 establishes the sufficient descent condition with explicit constant c = 1 − 1/(2μ), Theorem 4.2 proves the Zoutendijk-type inequality from Assumptions 4.2 and 4.3, and Lemmas 4.2–4.4 provide the boundedness and series estimates used in the contradiction framework. The generator C in Lemma 2.1 and the stationarity characterization in Proposition 2.2 are cited from prior work with overlapping authors, but they are parameter-free mathematical facts with stated assumptions and are not the target convergence theorem. The genuine weakness is not circularity but a completeness gap: the proof of Theorem 4.3 is reduced to the one-sentence statement 'exactly as in [12, Theorem 2]', and the key boundedness step for {∥dk∥} is not supplied in the present setting. This is a missing proof rather than a circular derivation, so it does not raise the circularity score beyond a low value. The single self-referential sentence in the conclusion is a rhetorical/citational circularity but does not infect the formal lemmas, which are independent of the conclusion being established. No fitted parameters are hidden in the convergence machinery, and none of the numerical results are presented as theory-derived predictions.
Assumptions & free parameters
free parameters (4)
- ρ (Armijo parameter) =
10^{-4} in experiments
- σ (Wolfe curvature parameter) =
0.1 in experiments
- μ (HZ parameter) =
1 in experiments
- ε (stopping tolerance) =
10^{-4} in experiments
assumptions (4)
- standard math Properties of φ (Lemma 2.2) from [5]: sublinearity, monotonicity w.r.t. K, 1-Lipschitz, -K={φ≤0}, -int K={φ<0}
- standard math The set C = {w∈K* | w^T e = 1} is a compact generator of K* for e∈int(K) (Lemma 2.1)
- domain assumption Assumptions 4.1-4.3: bounded level set L, Lipschitz gradients, and existence of a common lower bound S for monotone set sequences
- ad hoc to paper There exists a bounded set B with B ⪯_ℓ F(x) for all x∈R^n (Theorem 3.1)
Cite this review
Pith. "Pith review of Hager-Zhang Conjugate Gradient Method for Set Optimization with Set-Valued Objective Map of Finite Cardinality." pith.science (2026). https://pith.science/paper/GPB2XZDT
@misc{pith2026260713383,
author = {Pith},
title = {Pith review of: Hager-Zhang Conjugate Gradient Method for Set Optimization with Set-Valued Objective Map of Finite Cardinality},
year = {2026},
howpublished = {\url{https://pith.science/paper/GPB2XZDT}},
note = {Machine review of arXiv:2607.13383}
}
read the original abstract
This work introduces a nonlinear Hager-Zhang conjugate gradient method for solving set optimization problems. The objective function under consideration is defined by a finite collection of continuously differentiable functions. Notably, the proposed approach imposes restrictions neither on the existence of a finite generator of the ordering cone nor on any regularity condition at the optimal solution. As a result, the proposed method holds considerable significance for both set optimization and vector optimization problems, with the latter serving as a special case of the former. The study begins by discussing Wolfe line search conditions using Drummond-Svaiter scalarization function. Thereafter, we establish the existence of a step length satisfying the Wolfe line search conditions along a descent direction. The Hager-Zhang scalar conjugate parameter is introduced to derive the search direction for the proposed method. It is established that the direction generated by the proposed method is a descent direction. The well-definedness of the proposed method is given. Furthermore, we discuss some important results and a Zoutendijk-like condition to ensure global convergence. Subsequently, the global convergence of the proposed method is established in an asymptotic manner. Finally, numerical experiments on various test problems validate the practical performance and effectiveness of the proposed technique.
Reference graph
Works this paper leans on
-
[12]
Gon¸ calves, M.L.N., Prudente, L.F.: On the extension of the Hage r–Zhang conjugate gradient method for vector optimization. Comput. Optim. Appl. 76(3), 889–916 (2020) 28
2020
-
[1]
Opt imization 67, 1389– 1407 (2018)
Ansari, Q.H., K¨ obis, E., Sharma, P.K.: Characterizations of set relations with respect to variable domination structures via oriented distance function. Opt imization 67, 1389– 1407 (2018)
2018
-
[2]
Bouza, G., Quintana, E., Tammer, C.: A steepest descent method for set optimization problems with set-valued mappings of finite cardinality. J. Optim. The ory Appl. 190(3), 711–743 (2021)
2021
-
[3]
Das, I., Dennis, J.E.: Normal-boundary intersection: a new metho d for generating the Pareto surface in nonlinear multicriteria optimization problems. SIAM J. Optim. 8(3), 631–657 (1998)
1998
-
[4]
Dai, Y.H., Yuan, Y.: A nonlinear conjugate gradient method with a st rong global convergence property. SIAM J. Optim. 10(1), 177–182 (1999)
1999
-
[5]
Drummond, L.M.G., Svaiter, B.F.: A steepest descent method for v ector optimization. J. Comput. Appl. Math. 175(2), 395–414 (2005)
2005
-
[6]
In: Ansari, Q., Yao, J.C
Eichfelder, G., Jahn, J.: Vector optimization problems and their so lution concepts. In: Ansari, Q., Yao, J.C. (eds.) Recent Developments in Vector Optimizat ion, vol. 1, pp. 1–27. Springer, Berlin (2012)
2012
-
[7]
Fletcher, R.: Practical Methods of Optimization, Unconstrained Optimization. Vol. 1. Wiley, New York (1987)
1987
Show all 28 references
-
[8]
Fletcher, R., Reeves, C.M.: Function minimization by conjugate gra dients. Comput. J. 7(2), 149–154 (1964)
1964
- [9]
-
[10]
Ghosh, D., Kishor, N., Zhao, X.: A Newton method for uncertain m ultiobjective opti- mization problems with finite uncertainty Set. J. Nonlinear Var. Anal. 9(1), 81–110 (2025)
2025
-
[11]
Ghosh, D., Raushan, R., Peng, Z.Y., Yao, J.C.: Nonlinear conjugat e gradient methods for optimization of set-valued maps of finite cardinality. J. Optim. Th eory Appl. 207, 28 (2025)
2025
-
[13]
Hager, W.W., Zhang, H.C.: A new conjugate gradient method with g uaranteed descent and an efficient line search. SIAM J. Optim. 16(1), 170–192 (2005)
2005
-
[14]
Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients fo r solving linear systems. J. Res. Nat. Bur. Stand. 49(6), 409–436 (1952)
1952
-
[15]
Hillermeier, C.: Generalized homotopy approach to multiobjective optimization. J. Optim. Theory Appl. 110(3), 557–583 (2001)
2001
-
[16]
IEEE Trans
Huband, S., Hingston, P., Barone, L., While, L.: A review of multiobj ective test prob- lems and a scalable test problem toolkit. IEEE Trans. Evol. Comput. 101(5), 477–506 (2006)
2006
-
[17]
Comput Optim Appl
Hu, Q., Zhu, L., Chen, Y.: Alternative extension of the Hager–Zh ang conjugate gradient method for vector optimization. Comput Optim Appl. 88, 217–250 (2024)
2024
-
[18]
Springer, Berlin (2011)
Jahn, J.: Vector Optimization: Theory, Applications, and Exten sions, 2nd edn. Springer, Berlin (2011)
2011
-
[19]
1042–1049 (2001 )
Jin, Y., Olhofer, M., Sendhoff, B.: Dynamic weighted aggregation f or evolutionary multi-objective optimization: why does it work and how? In: Proceed ings of the Genetic and Evolutionary Computation Conference, pp. 1042–1049 (2001 )
2001
-
[20]
CRC Press, Boca Ra ton (2019)
Khan, A.A., K¨ obis, E., Tammer, C.: Variational Analysis and Set Op timization: Developments and Applications in Decision Making. CRC Press, Boca Ra ton (2019)
2019
-
[21]
S pringer, Berlin (2016)
Khan, A.A., Tammer, C., Z˘ alinescu, C.: Set-Valued Optimization. S pringer, Berlin (2016)
2016
-
[22]
Kishor, N., Ghosh, D., Zhao, X.: Generalized ordered weighted ag gregation robustness to solve uncertain single objective optimization problems. J. Nonlinea r Convex Anal. Accepted (2024)
2024
-
[23]
Optimization
Kumar, K., Ghosh, D., Yao, J.C., Zhao, X.: Nonlinear conjugate gr adient methods for unconstrained set optimization problems whose objective func tions have finite cardinality. Optimization. 1–40 (2024)
2024
-
[24]
Kuroiwa, D.: Some criteria in set-valued optimization. Vol. 985, pp . 171–176 (1997). Investigations on nonlinear analysis and convex analysis (Japanese ) (Kyoto, 1996)
1997
-
[25]
ACM Trans
P´ erez, L.R.L., Prudente, L.F.: A Wolfe line search algorithm for ve ctor optimization. ACM Trans. Math. Softw. 45(4), 1–23 (2019)
2019
-
[26]
P´ erez, L.R.L., Prudente, L.F.: Nonlinear conjugate gradient me thods for vector optimization. SIAM J. Optim. 28(3), 2690–2720 (2018)
2018
-
[27]
USSR Comput
Polyak, B.T.: The conjugate gradients method in extreme proble ms. USSR Comput. Math Math Phys. 9(4), 94–112 (1969)
1969
-
[28]
Technical Report, The University of Namur, Depa rtment of Mathe- matics, Belgium (1983) 29
Toint, P.L.: Test problems for partially separable optimization and results for the routine PSPMIN. Technical Report, The University of Namur, Depa rtment of Mathe- matics, Belgium (1983) 29
1983
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.