Pith. sign in

REVIEW 3 major objections 6 minor 38 references

Learning complexity of gradient descent and conjugate gradient algorithms

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Gradient descent and conjugate gradient step sizes can be learned from finite samples with arbitrarily small error under a new sum-of-distances cost.

desk verdict Real idea (new cost function), but the sample bounds are internally inconsistent and the 'CG' section analyzes heavy-ball, not conjugate gradients. read the letter →

arxiv 2412.13473 v1 pith:OXVQEGX6 submitted 2024-12-18 math.OC cs.CC

Xianqi Jiao , Jia Liu , Zhiping Chen This is my paper · ORCID
classification math.OCcs.CC MSC 68Q3290C2568T05
keywords algorithmselectionlearningcomplexitypseudo-dimensiongradientdescentconjugatesamplestepsizetuningdata-drivendesign
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Algorithm selection — choosing the step size for gradient descent, or the step size and conjugate parameter for conjugate gradient — is treated as a statistical learning problem with an unknown distribution over optimization instances. This paper introduces a new cost function that sums, at every iteration, the distance from the current iterate to the optimum, a continuous analogue of the primal-dual integral used in integer programming. Using this cost, the authors prove that the optimal gradient-descent step size can be $(C+\varepsilon,\delta)$-learned from $m=\widetilde{O}(H^3/\varepsilon^2)$ samples for any $C>0$, and that the optimal conjugate-gradient parameters can be $(C+\varepsilon,\delta)$-learned from $m=\widetilde{O}(H^4/\varepsilon^2)$ samples. These are the first sample-complexity guarantees of this kind for conjugate gradient methods, and they improve the learning-error constant of the earlier iteration-count cost from $1$ to an arbitrarily small $C$.

What carries the argument

The driving object is the sum-of-distances cost function $c(A_\rho,x)=\sum_{j=1}^{M}\|z^*-g^j(z_0,\rho)\|$, which is bounded by $H\|z_0\|$ and Lipschitz-continuous in the step size under Assumption 1.4. Around this cost, the paper builds a $K$-net of step sizes: if the step-size gap is at most $K$, the cost difference is at most $C$, and the net's size is polynomial in $1/K$. The pseudo-dimension of the finite net is bounded by $\widetilde{O}(H)$, and an empirical-risk-minimization algorithm on the net inherits the uniform-convergence sample bound. For conjugate gradient, the same structure runs through the second-order homogeneous linear recurrence $x_n=[D(\rho)+\eta]x_{n-1}+\eta x_{n-2}$; its root formula delivers the Lipschitz constants $G^*$ and $H^*$ that control the cost error and hence the net size.

What would settle it

Run the discretization test on a convex $L$-smooth function whose contraction ratio is not uniformly bounded below $1$ over $[\rho_l,\rho_u]$ — for example, a strongly convex quadratic with step sizes approaching the upper stability limit $2/L$, where the ratio $\|z-\rho\nabla f(z)\|/\|z\|$ tends to $1$ along the largest-curvature eigenvector. If such a family admits no $\beta\in(0,1)$ satisfying Assumption 1.4, then the theorem's premise is violated; measuring $|c(A_\rho,x)-c(A_\eta,x)|$ for two step sizes at the proposed net spacing on that family shows whether the claimed $C$-error bound still holds or fails.

Watch

Extended reading notes

Core claim

The central claim is that a cost function defined by $c(A_\rho,x)=\sum_{j=1}^{M}\|z^*-g^j(z_0,\rho)\|$ makes the family of gradient descent algorithms learnable with error $C+\varepsilon$ using $\widetilde{O}(H^3/\varepsilon^2)$ samples, and that the analogous two-parameter family of conjugate gradient algorithms is learnable with $\widetilde{O}(H^4/\varepsilon^2)$ samples. The authors show that nearby step sizes produce nearby costs: the cost difference between two algorithms is controlled by the parameter gap times a constant that depends on the contraction factor $\beta$, the smoothness $L$, the horizon $H$, and the bounded initial distance $Z$. Discretizing the parameter interval into a net fine enough to keep this cost difference below $C$ reduces the infinite family to a finite one, whose pseudo-dimension is $\widetilde{O}(H)$; uniform convergence then yields the sample bounds. For conjugate gradient, the iteration error is propagated through a second-order linear recurrence, giving explicit bounds $G^*$ and $H^*$ that make the same net argument work.

Load-bearing premise

The load-bearing premise is Assumption 1.4, and its CG analogue Assumption 2.4: there is a single $\beta\in(0,1)$ such that every step size in the allowed interval shrinks the distance to the optimum by a factor at most $1-\beta$, uniformly for all iterates; without this interval-wide contraction, the cost bounds, the net discretization, and the sample-complexity theorems do not follow.

Editorial extensions

If this is right

  • With $\widetilde{O}(H^3/\varepsilon^2)$ samples, a data-driven learner can select a gradient descent step size whose expected sum-of-distances cost is within $C+\varepsilon$ of the best step size in the family, for any tolerance $C>0$.
  • For conjugate gradient, $\widetilde{O}(H^4/\varepsilon^2)$ samples suffice to tune both the step size and the conjugate parameter, giving the first learning-complexity guarantee for conjugate gradient in this setting.
  • Because the new cost can be evaluated from early-terminated runs, the sample guarantees hold even when optimization is stopped before convergence, unlike cost functions based on iteration counts.
  • The learning error is no longer floored at $1$ by the integrality of iteration counts; the cost difference between two algorithms can be made arbitrarily small by choosing step sizes close enough.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's stated claims, the same net argument would likely extend to stochastic-gradient variants if Assumption 1.4 were relaxed to an on-average contraction; testing this is a natural next step.
  • The new cost function is a smooth aggregate over the trajectory, so it could be used as a meta-learning objective for step-size prediction, where the iteration count is discontinuous.
  • The sample bounds are existence results: the paper does not give an efficient way to search the parameter net, so an open question is whether the optimum can be found in time polynomial in $H$ and $1/\varepsilon$.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies sample-complexity guarantees for tuning the parameters of gradient descent and of an algorithm it calls conjugate gradient, within the PAC algorithm-selection framework of Gupta and Roughgarden. It introduces a new cost function that sums, over iterations, the distance of each iterate to the optimum. The authors claim an improved GD bound, m = O~(H^3/eps^2), and the first learning-complexity bound for a CG family, m = O~(H^4/eps^2). The CG analysis is built on a two-step recurrence z_{n+1} = z_n - rho grad f(z_n) - eta (z_n - z_{n-1}) with constant rho and eta.

Significance. The new cost function is a reasonable performance measure and, if the bounds were correct, the GD part would be a useful incremental extension of Gupta-Roughgarden. However, the paper's central new claim regarding conjugate gradient is not about conjugate gradients: the recurrence analyzed in Section 4 is the heavy-ball / momentum method with constant coefficients, not a conjugate-gradient method with a recomputed beta_n and a line search. In addition, the stated sample bounds do not follow from the stated pseudo-dimension bounds, because the factor H in the pseudo-dimension is dropped when applying the uniform-convergence theorem. These two issues directly undermine the abstract and the two main theorems. The paper does provide a detailed, self-contained derivation of error-estimation lemmas for the heavy-ball family, but this does not compensate for the fact that the advertised contribution is not established.

major comments (3)
  1. [Theorem 5 and Appendix D] The sample bound m = O~(H^3/eps^2) in Theorem 5 does not follow from the stated pseudo-dimension bound. In Section 3.2 the paper sets the pseudo-dimension of the discretized GD class to at most H log|N|, where H = log(nu/LZ)/log(1-beta), and log|N| ~ O(H). Theorem 1 with range [0,H] then gives m = O((H^2/eps^2)(H log|N| + ln(1/delta))) = O~(H^3 log|N|/eps^2) = O~(H^4/eps^2), not O~(H^3/eps^2). Appendix D ignores the factor H in the pseudo-dimension. The same issue occurs in Theorem 9: the pseudo-dimension is H log|N_rho| log|N_eta|, so the uniform-convergence theorem yields m = O~(H^2 * H * H * H / eps^2) = O~(H^5/eps^2), not O~(H^4/eps^2).
  2. [Section 4, first paragraph] The recurrence analyzed in Section 4, z_1 = z_0 - rho grad f(z_0), z_{n+1} = z_n - rho grad f(z_n) - eta (z_n - z_{n-1}) with constant rho and eta, is the heavy-ball (momentum) method, not a conjugate gradient method. In CG, the search direction is d_n = -grad f(z_n) + beta_n d_{n-1} with beta_n recomputed from gradient inner products (e.g., Fletcher-Reeves), and the step length is chosen by a line search. Neither element appears in the paper's update. Consequently, Lemmas 3-8 and Theorems 6-9 establish results for a fixed-parameter two-step gradient family, not for any conjugate-gradient algorithm. The abstract and Section 4 claim a first learning-complexity result for the 'conjugate gradient algorithm group'; this claim is not supported by the proofs.
  3. [Assumptions 1.4 and 2.4; Appendix B] Assumption 1.4, and its two-parameter analogue Assumption 2.4, require the iteration map to be a uniform contraction, ||z - rho grad f(z)|| <= (1-beta)||z|| for every rho in [rho_l, rho_u] (and, for Assumption 2.4, for every rho, eta in the respective intervals). Appendix B demonstrates only that a single step size, rho = 2/(m+L), satisfies the condition for a strongly convex objective; it does not justify the interval-wide condition used in Theorems 4-9. Since the net construction and the iteration-error estimates depend on the contraction holding for all parameters in the intervals, the stated learning guarantees apply only to a limited, unspecified subclass of problems and parameter ranges. The paper should either restrict the algorithm class to parameters satisfying the contraction condition as an explicit definitional assumption for the whole scope of the theorems, or provide a proof that the condition holds on the full intervals.
minor comments (6)
  1. [Abstract] The abstract and contribution 3 state 'prove the existence a learning algorithm'; it should be 'prove the existence of a learning algorithm'.
  2. [Section 3.2] The new cost function is defined as c(A_rho,x) = sum_{j=1}^M ||z* - g_j(z0,rho)||, but in Section 4 the analogous sum is written as sum_{i=0}^M. Please use a consistent indexing convention.
  3. [Theorem 4] The expression for K in the statement of Theorem 4 uses D(rho_u) while the condition preceding it uses D(rho); clarify the relationship between the condition on eta - rho and the net spacing K, and state which D entry appears in each place.
  4. [Appendix F, Corollary 2 proof] In the display of the recurrence for A_n, the text writes 'A_n = [D(rho)+eta] A_{n-1} + A_{n-2}', but the characteristic equation and the surrounding equations use eta A_{n-2}. This appears to be a typo.
  5. [Appendix G] In the proof of Lemma 4 there is a stray bracket in '||grad f(z'_j)]||'; the closing bracket after the norm is a typo.
  6. [Appendix K and L] Lemma 7 and Lemma 8 define R* and R*' with absolute values, but the preceding derivations of the coefficients c1 and c2 already rely on signs; please make the sign conventions explicit and verify the inequalities are strict when r1=1 (if that case can occur under Assumption 2).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the sample-complexity theorems follow from the stated assumptions and the external Gupta-Roughgarden framework, with no fitted parameter renamed as a prediction; the notable defects are correctness and labeling issues, not circular reductions.

full rationale

I walked the derivation chain. The paper defines a new cost function as the sum of distances to the optimum, bounds the error between two step-size configurations via iteration-error lemmas, discretizes the parameter interval into a K-net, bounds the pseudo-dimension by the log of the finite net, and then invokes uniform convergence and ERM (Theorems 1 and Corollary 1). No step fits a parameter to data and then renames that fit as a prediction: the cost function is a performance measure, not a fitted quantity, and the optimal algorithm is selected by ERM on a finite class. Theorems 5 and 9 are covering arguments: the net spacing is chosen proportional to the desired error C, so any algorithm in A is within C of some net algorithm on every instance; the sample bound then follows from pseudo-dimension and uniform convergence. This is a standard reduction, not a circular one. The cited Gupta-Roughgarden material is external prior work used as lemmas, and there are no load-bearing self-citations. The manuscript's real weaknesses lie elsewhere and should be scored as correctness risk, not circularity: (1) Section 4 defines 'conjugate gradient' by the recurrence z_{n+1}=z_n-ρ∇f(z_n)-η(z_n-z_{n-1}) with constant ρ,η, which is the heavy-ball/momentum method rather than a conjugate-gradient method with a recomputed β_k, so the advertised 'first learning complexity for CG' claim is not supported by the analyzed object; and (2) Appendix B verifies Assumption 1.4 only for a single step size in the strongly convex case, not for the interval-wide condition used in the theorems. Both are internal-validity or definitional gaps, but neither makes a derived quantity equivalent to its inputs by construction. Therefore the circularity score is 0.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The bounds rest on two strong interval-wide contraction assumptions (Assumptions 1.4 and 2.4) that are only partially justified in Appendix B; the cost function convention requires knowing the optimum. C and β are user-chosen/existential constants rather than fitted parameters. No new physical or mathematical entities are introduced.

free parameters (2)
  • C = arbitrary positive constant (user-specified)
    Determines the net spacings Kρ and Kη; the sample complexity bound suppresses C as a constant, so the O~ notation depends on it.
  • β = exists in (0,1) but not specified
    Chosen as an existential constant to satisfy the contraction assumptions; it enters H and the recurrence bounds and is not derived from data.
assumptions (5)
  • domain assumption Assumption 1.1-1.4 for GD: convex L-smooth, bounded initial points, step size interval, and contraction ||z - ρ∇f(z)|| ≤ (1-β)||z|| for all ρ in the interval.
    Invoked in Section 3.1; the contraction condition is strong and only partially justified in Appendix B.
  • ad hoc to paper Assumption 2.1-2.4 for the two-parameter method: same convexity and boundedness, plus contraction ||zn - ρ∇f(zn) - η(zn-zn-1)|| ≤ (1-β)||zn|| for all ρ, η in intervals.
    Section 4; the contraction condition is specifically tailored to the chosen constant-parameter update and has no basis in standard CG convergence theory.
  • domain assumption Cost function conventions: z* = 0, f(z*) = 0, and iterates after termination stay at z*.
    Sections 3.2 and 4; translation invariance is legitimate but the post-termination convention changes the cost function and requires knowing the optimum.
  • standard math Uniform convergence theorem for pseudo-dimension (Anthony and Bartlett 1999) and ERM learning corollary (Gupta-Roughgarden 2016).
    Used in Theorems 5 and 9; accepted background results.
  • standard math Second-order homogeneous linear recurrence solution formula (Epp 2010).
    Used in Corollary 2 and Lemmas 4/5 to derive closed-form bounds on iteration errors.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning complexity of gradient descent and conjugate gradient algorithms." pith.science (2026). https://pith.science/paper/OXVQEGX6

@misc{pith2026241213473,
  author       = {Pith},
  title        = {Pith review of: Learning complexity of gradient descent and conjugate gradient algorithms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OXVQEGX6}},
  note         = {Machine review of arXiv:2412.13473}
}
read the original abstract

Gradient Descent (GD) and Conjugate Gradient (CG) methods are among the most effective iterative algorithms for solving unconstrained optimization problems, particularly in machine learning and statistical modeling, where they are employed to minimize cost functions. In these algorithms, tunable parameters, such as step sizes or conjugate parameters, play a crucial role in determining key performance metrics, like runtime and solution quality. In this work, we introduce a framework that models algorithm selection as a statistical learning problem, and thus learning complexity can be estimated by the pseudo-dimension of the algorithm group. We first propose a new cost measure for unconstrained optimization algorithms, inspired by the concept of primal-dual integral in mixed-integer linear programming. Based on the new cost measure, we derive an improved upper bound for the pseudo-dimension of gradient descent algorithm group by discretizing the set of step size configurations. Moreover, we generalize our findings from gradient descent algorithm to the conjugate gradient algorithm group for the first time, and prove the existence a learning algorithm capable of probabilistically identifying the optimal algorithm with a sufficiently large sample size.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 30 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    T.; Liggett, K.; Musco, C.; Tzamos, C.; and Vitercik, E

    Alabi, D.; Kalai, A. T.; Liggett, K.; Musco, C.; Tzamos, C.; and Vitercik, E. 2019. Learning to prune: speeding up repeated computations. In Proceedings of the Thirty-Second Conference on Learning Theory, 30--33

    work page 2019

  4. [4]

    Andrei, N. 2020. Nonlinear Conjugate Gradient Methods for Unconstrained Optimization. SpringerCham

    work page 2020

  5. [5]

    W.; Pfau, D.; Schaul, T.; Shillingford, B.; and de Freitas, N

    Andrychowicz, M.; Denil, M.; G\' o mez, S.; Hoffman, M. W.; Pfau, D.; Schaul, T.; Shillingford, B.; and de Freitas, N. 2016. Learning to learn by gradient descent by gradient descent. In Advances in Neural Information Processing Systems

    work page 2016

  6. [6]

    Anthony, M.; and Bartlett, P. L. 1999. Neural Network Learning: Theoretical Foundations. Cambridge University Press

    work page 1999

  7. [7]

    Babaie-Kafaki, S. 2023. A survey on the Dai–Liao family of nonlinear conjugate gradient methods. RAIRO-Operations Research, 57(1): 43--58

    work page 2023

  8. [8]

    Balcan, M.-F.; DeBlasio, D.; Dick, T.; Kingsford, C.; Sandholm, T.; and Vitercik, E. 2021. How much data is sufficient to learn high-performing algorithms? Generalization guarantees for data-driven algorithm design. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing, 919–932

    work page 2021

Show all 38 references
  1. [9]

    Balcan, M.-F.; Dick, T.; Sandholm, T.; and Vitercik, E. 2018. Learning to branch. In Proceedings of the 35th International Conference on Machine Learning, 344--353

    2018

  2. [10]

    Balcan, M.-F.; Nagarajan, V.; Vitercik, E.; and White, C. 2017. Learning-theoretic foundations of algorithm configuration for combinatorial partitioning problems. In Proceedings of the 2017 Conference on Learning Theory, 213--274

    2017

  3. [11]

    Balcan, M.-F.; Prasad, S.; Sandholm, T.; and Vitercik, E. 2022. Improved sample complexity bounds for branch-and-cut. arXiv:2111.11207

    2022 arXiv

  4. [12]

    Berthold, T. 2013. Measuring the impact of primal heuristics. Operations Research Letters, 41(6): 611--614

    2013

  5. [13]

    Bubeck, S. 2015. Convex Optimization: Algorithms and Complexity. Foundations and Trends® in Machine Learning, 8: 231--357

    2015

  6. [14]

    H.; and Yuan, Y

    Dai, Y. H.; and Yuan, Y. X. 1999. A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property. SIAM J. Optimization, 10(1): 177--182

    1999

  7. [15]

    Denevi, G.; Ciliberto, C.; Grazzi, R.; and Pontil, M. 2019. Learning-to-learn stochastic gradient descent with biased regularization. In Proceedings of the 36th International Conference on Machine Learning

    2019

  8. [16]

    Du, X.; Zhang, P.; and Ma, W. 2016. Some modified conjugate gradient methods for unconstrained optimization. Journal of Computational and Applied Mathematics, 305: 92--114

    2016

  9. [17]

    Epp, S. S. 2010. Discrete Mathematics with Applications. Brooks/Cole Publishing Company, 4th edition

    2010

  10. [18]

    Fletcher, R.; and Reeves, C. M. 1964. Function minimization by conjugate gradients. The Computer Journal, 7: 149--154

    1964

  11. [19]

    M.; Khalil, E.; Lichocki, P.; Lodi, A.; Lubin, M.; Maddison, C

    Gasse, M.; Bowly, S.; Cappart, Q.; Charfreitag, J.; Charlin, L.; Ch \'e telat, D.; Chmiela, A.; Dumouchelle, J.; Gleixner, A.; Kazachkov, A. M.; Khalil, E.; Lichocki, P.; Lodi, A.; Lubin, M.; Maddison, C. J.; Christopher, M.; Papageorgiou, D. J.; Parjadis, A.; Pokutta, S.; Pro...

    2022

  12. [20]

    Gupta, R.; and Roughgarden, T. 2016. A PAC approach to application-specific algorithm selection. In Proceedings of the 2016 ACM Conference on Innovations in Theoretical Computer Science, 123–134

    2016

  13. [21]

    S.; and Sandholm, T

    Harris, K.; Anagnostides, I.; Farina, G.; Khodak, M.; Wu, Z. S.; and Sandholm, T. 2023. Meta-learning in games. arXiv:2209.14110

    2023 arXiv

  14. [22]

    R.; and Stiefel, E

    Hestenes, M. R.; and Stiefel, E. 1952. Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 49: 409--436

    1952

  15. [23]

    Hospedales, T.; Antoniou, A.; Micaelli, P.; and Storkey, A. 2022. Meta-learning in neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(9): 5149--5169

    2022

  16. [24]

    J.; and Vazirani, U

    Kearns, M. J.; and Vazirani, U. V. 1994. An Introduction to Computational Learning Theory. Cambridge, MA, USA: MIT Press

    1994

  17. [25]

    Li, K.; and Malik, J. 2016. Learning to optimize. arXiv:1606.01885

    2016 arXiv

  18. [26]

    Liu, Z.; Liu, H.; and Dai, Y. 2020. An improved Dai-Kou conjugate gradient algorithm for unconstrained optimization. Computational Optimization and Applications, 75(1): 145--167

    2020

  19. [27]

    Maclaurin, D.; Duvenaud, D.; and Adams, R. 2015. Gradient-based hyperparameter optimization through reversible learning. In Proceedings of the 32nd International Conference on Machine Learning

    2015

  20. [28]

    Maria-Florina, B.; Khodak, M.; Sharma, D.; and Talwalkar, A. 2021. Learning-to-learn non-convex piecewise-Lipschitz functions. In Advances in Neural Information Processing Systems, 15056--15069

    2021

  21. [29]

    Pollard, D. 1984. Convergence of Stochastic Processes. Springer New York

    1984

  22. [30]

    Polyak, B. 1969. The conjugate gradient method in extremal problems. USSR Computational Mathematics and Mathematical Physics, 9(4): 94--112

    1969

  23. [31]

    Qu, Q.; Li, X.; Zhou, Y.; Zeng, J.; Yuan, M.; Wang, J.; Lv, J.; Liu, K.; and Mao, K. 2022. An improved reinforcement learning algorithm for learning to branch. arXiv:2201.06213

    2022 arXiv

  24. [32]

    Snyman, J. 2005. Practical Mathematical Optimization: An Introduction to Basic Optimization Theory and Classical and New Gradient-Based Algorithms. Springer New York

    2005

  25. [33]

    S.; Ivanov, B.; Ma, H.; and Mosić, D

    Stanimirović, P. S.; Ivanov, B.; Ma, H.; and Mosić, D. 2020. A survey of gradient methods for solving nonlinear optimization. Electronic Research Archive, 28(4): 1573--1624

    2020

  26. [34]

    Vapnik, V. 1991. Principles of risk minimization for learning theory. In Advances in Neural Information Processing Systems

    1991

  27. [35]

    Wang, X.; Yuan, S.; Wu, C.; and Ge, R. 2021. Guarantees for tuning the step size using a learning-to-learn approach. In Proceedings of the 38th International Conference on Machine Learning, 10981--10990

    2021

  28. [36]

    Wolfe, P. 1969. Convergence Conditions for Ascent Methods. SIAM Review, 11: 226--235

    1969

  29. [37]

    Yang, Z. 2022. Adaptive stochastic conjugate gradient for machine learning. Expert Systems with Applications, 206: 117719

    2022

  30. [38]

    Zhang, J.; Liu, C.; Li, X.; Zhen, H.; Yuan, M.; Li, Y.; and Yan, J. 2023. A survey for solving mixed integer programming via machine learning. Neurocomputing, 519: 205--217

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.