Pith. sign in

REVIEW 2 major objections 4 minor 1 cited by

Randomized Subspace Derivative-Free Optimization with Quadratic Models and Second-Order Convergence

T0 review · 2 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Random-subspace derivative-free optimization can reach approximate second-order critical points with complexity $\widetilde{O}(n^{4.5}\epsilon^{-3})$.

desk verdict First second-order complexity bound for random subspace model-based DFO, with a practical solver that scales to n≈1000; the n^4.5 bound is real but its ε-dependence is less clean than the presentation suggests. read the letter →

arxiv 2412.14431 v1 pith:CWNZ3LEW submitted 2024-12-19 math.OC

Coralia Cartis , Lindon Roberts This is my paper · ORCID
classification math.OC MSC 90C5690C3065K05
keywords derivative-freeoptimizationrandomsubspacessecond-orderconvergenceworst-casecomplexityquadraticinterpolationmodelsJohnson-Lindenstrausstransformsloweffectiveranktrust-regionmethods
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper extends random-subspace model-based derivative-free optimization (DFO) from first-order to second-order convergence. It proves the first high-probability worst-case complexity bound for such methods at approximate second-order critical points: $\widetilde{O}(n^{4.5}\epsilon^{-3})$ iterations and objective evaluations for an $n$-dimensional problem, versus $O(n^{9})$ iterations and $O(n^{11})$ evaluations for full-space model-based DFO. The improvement applies when the target accuracy $\epsilon$ is modest or the Hessian has low effective rank. The paper also introduces RSDFO-Q, a practical quadratic-interpolation subspace method, and shows numerically that it solves problems of dimension roughly 1000 where full-space quadratic DFO fails.

What carries the argument

The load-bearing object is the well-aligned subspace condition (Definition 3.6). A subspace $P$ is well-aligned when it satisfies $\|P\|\le P_{\max}$, captures a $(1-\alpha)$ fraction of the gradient via $\|P^T\nabla f(x_k)\|\ge(1-\alpha)\|\nabla f(x_k)\|$, captures a $(1-\alpha)$ fraction of the most negative Hessian eigenvector $v_r$, and keeps every other eigenvector $v_i$ nearly orthogonal to $v_r$ in the subspace via $(\hat v_i^T\hat v_r)^2\le 4\alpha^2$. Combined with the condition $\theta>0$ in (24), this implies the subspace criticality measure is at least $\min((1-\alpha)^2,\theta)\sigma_k$ (Lemma 3.8), which lets the trust-region analysis in Lemmas 3.10--3.13 run entirely in the low-dimensional space. Johnson-Lindenstrauss random matrices are shown to satisfy the alignment conditions with $p=O(\log r)$ and $P_{\max}=O(\sqrt{n/p})$, which is the source of the improved dimension dependence.

What would settle it

Construct a smooth nonconvex function in $\mathbb{R}^n$ whose Hessian at every relevant iterate has $r\approx n$ eigenvectors with nearly identical, strictly negative eigenvalues, so no single dominant negative direction exists. Draw Gaussian Johnson-Lindenstrauss subspaces of dimension $p=O(\log n)$ as prescribed, and compute the empirical probability that $\theta>0$: it should decay to zero as $n$ grows. Then run RSDFO-Q on that function; if it still reaches the predicted $\epsilon$-second-order critical complexity with uniformly good probability, the $\theta>0$ mechanism is not actually needed, whereas if it fails, the low-rank caveat in Remark 2 is confirmed.

Watch

Extended reading notes

Core claim

The central claim, Theorem 3.15, is that under boundedness and smoothness assumptions, fully quadratic subspace models, and a well-aligned random subspace condition, the iterate sequence satisfies $P[\min_{j\le k}\sigma_j \le \epsilon] \ge 1-\exp(-c(k+1))$ after $k=O(\kappa_d^3\epsilon^{-3})$ iterations, where $\sigma_k=\max(\|\nabla f(x_k)\|, -\lambda_{\min}(\nabla^2 f(x_k)))$ is the second-order criticality measure and $\kappa_d$ collects the fully quadratic model constants. With Johnson-Lindenstrauss subspaces of dimension $p=O(\log r)$, where $r$ is the rank of the Hessian, $\kappa_d=\widetilde{O}(n^{3/2})$ and the bound becomes $\widetilde{O}(n^{4.5}\epsilon^{-3})$ iterations and evaluations. The mechanism is to show that a well-aligned subspace preserves a constant fraction of the full-space criticality measure, so that trust-region progress in the subspace certifies progress for the original problem.

Load-bearing premise

The load-bearing premise is the well-aligned subspace condition together with $\theta>0$ (Equation 24): a random low-dimensional subspace must retain a fixed fraction of the gradient and of the most negative Hessian eigenvector, with all other eigenvectors nearly orthogonal to it inside the subspace. If $\theta\le 0$, the subspace criticality measure can vanish even when the full-space problem is far from second-order critical, and the complexity bound no longer holds.

Editorial extensions

If this is right

  • Second-order convergence theory now exists for random-subspace model-based DFO, matching the full-space $\epsilon^{-3}$ rate while improving the dimension dependence.
  • For problems with low effective rank $r\ll n$, the subspace dimension needs only $p=O(\log r)$, so values of $n$ near 1000 or larger become practical for derivative-free optimization.
  • RSDFO-Q with full-dimensional subspaces is broadly competitive with a state-of-the-art full-space quadratic-interpolation solver on medium-scale problems, while low-dimensional variants solve large-scale problems where the full-space solver hits runtime limits.
  • The high-probability complexity bound implies, by standard arguments the paper outlines, almost-sure convergence of a subsequence of criticality measures to zero and complexity bounds in expectation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The $\theta>0$ requirement means the advertised advantage is really a statement about detecting one dominant negative-curvature direction: when the Hessian has many comparable negative directions at once, the needed subspace dimension grows like $O(r\epsilon^{-1})$, so the $n^{4.5}$ bound should not be expected to hold at high accuracy without low effective rank.
  • A direct testable extension is to apply RSDFO-Q to large-scale neural-network fine-tuning, where Hessian low-rank structure has been reported; one would predict evaluation counts that track the effective rank rather than the ambient dimension.
  • The primary/secondary interpolation-set device could be extracted and reused in other derivative-free trust-region solvers as a cheap way to build underdetermined quadratic models in subspaces.
  • Combining the subspace second-order analysis with stochastic objective values, as has been done for the first-order case, would give a route to stochastic derivative-free second-order complexity bounds.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper develops a randomized subspace model-based derivative-free optimization (DFO) framework with convergence guarantees to approximate second-order critical points. After recalling the first-order random-subspace framework, it introduces a second-order variant (Algorithm 2) equipped with a notion of well-aligned random subspaces (Definition 3.6) and a probabilistic alignment assumption (Assumption 3.7). The main theoretical result, Theorem 3.15, gives a high-probability worst-case complexity bound for reaching σ_k ≤ ε under a positivity condition θ>0 in Eq. (24). The authors then interpret this as O(κ_d^3 ε^{-3}) iterations and O(p^2 κ_d^3 ε^{-3}) evaluations, and, using Johnson–Lindenstrauss subspaces of dimension O(log r), advertise an O~(n^{4.5} ε^{-3}) iteration and evaluation complexity versus O(n^9 ε^{-3}) and O(n^{11} ε^{-3}) for full-space methods. The second half of the paper introduces a practical subspace quadratic-model algorithm, RSDFO-Q, and demonstrates numerically that it scales to problems with n≈1000 where full-space Py-BOBYQA variants fail.

Significance. If the claimed complexity result were valid uniformly in ε, the paper would make a substantial contribution: it appears to be the first worst-case second-order convergence analysis for random subspace model-based DFO, and it identifies a realistic regime (low-accuracy solutions or low-effective-rank problems) in which random subspaces improve dimension dependence. The proof chain from Lemma 3.8 through Theorem 3.15 is coherent and follows standard trust-region arguments, and the numerical study of RSDFO-Q is a useful practical contribution with a credible benchmark against Py-BOBYQA. The paper is also honest in Remark 2 about the restrictiveness of θ>0. However, the central advertised O~(n^{4.5} ε^{-3}) law is not uniform in ε: the positivity condition forces the alignment parameter α to shrink with ε, which in turn forces the subspace dimension p to grow like ε^{-1} in the JLT construction. This directly affects the main complexity claim and requires a substantial re-statement of the result's scope.

major comments (2)
  1. [Section 3.1, Table 1, Remark 2; Lemma 3.8, Eq. (24)] The advertised complexity O~(n^{4.5} ε^{-3}) is not a uniform consequence of Theorem 3.15. For θ>0 in Eq. (24) to hold with M=O(1), the alignment parameter α must satisfy α² = O(ε/r). The JLT construction in Section 3.1 then requires p = O(α^{-2} log r) = O(r ε^{-1} log r), not p = O(log r). Since κ_d = O(p³ P_max³) and P_max = Θ(sqrt(n/p)), substituting p(ε) gives an iteration bound of O~(n^{9/2} r^{9/2} ε^{-15/2}) and an evaluation bound of O~(n^{9/2} r^{13/2} ε^{-21/2}) rather than the displayed ε^{-3} laws. Table 1 and the sentence 'the dependency on n decreases from O(n^9) to O(n^{4.5})' are therefore only valid in a fixed-ε low-accuracy regime, or when r log r = O(ε); the abstract's branch 'and/or the problem has low effective rank' is also misleading if r is merely small but ε is driven to zero. Theorem 3.15, the complexity summary, and Table 1 need to be restated so that κ_d and p are explicitly allowed to depend on ε, or the theorem should be restricted to the regime in which the p=O(log r) choice is compatible with θ>0.
  2. [Section 4 and Section 5] The practical algorithm RSDFO-Q is not covered by the second-order theory developed in Section 3. Algorithm 3 is run numerically with q=2p+1 interpolation points, and the paper itself notes that fully quadratic models in the sense of Definition 3.1 generically require q=(p+1)(p+2)/2. Thus the experiments in Section 5 do not demonstrate the complexity guarantee of Theorem 3.15, and the claim that RSDFO-Q is a 'practical implementation of the ideas in Algorithms 1 and 2' needs a clear caveat that the implemented model construction is not fully quadratic and hence the second-order convergence theory does not apply to the numerical results as presented.
minor comments (4)
  1. [Section 3.1] The text says 'using a complexity of O(κ5_d ε^{-3}) from above'; this should be O(κ_d^3 ε^{-3}) to be consistent with the complexity summary and with Eq. (14) in the trust-region argument.
  2. [Theorem 3.15] The theorem statement contains a typographical artifact in the lower bound on k: '− 1, ,' has an extra comma, and the notation δ_s is used inconsistently with δ_S elsewhere in the statement and proof.
  3. [Section 5.2] The text says RSDFO-Q with p=n achieves 'broadly comparable performance' to Py-BOBYQA, but Figure 1 shows it is worse than Py-BOBYQA with q=2n+1 in the data profile; the wording should be adjusted to reflect the actual comparison.
  4. [Section 5 and References] No code or data availability statement is provided for the numerical experiments; the authors should state whether code is available and describe problem/data sources beyond the CUTEst references.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the complexity bound is derived from stated assumptions and re-derived lemmas, with external benchmarks; the theta>0/Remark 2 limitation is a scope issue, not a circular reduction.

full rationale

The central second-order complexity result (Theorem 3.15) is obtained by a chain of lemmas whose proofs are included in the manuscript itself. Lemma 3.8, which converts the well-aligned subspace conditions (Definition 3.6) and theta>0 into a lower bound on the subspace criticality measure, is proved in full inside the paper, even though it is attributed to [11] and [31]. Lemmas 3.9-3.14 and the Chernoff bound are likewise proved or explicitly cited to standard arguments; they do not assume the conclusion they are used to prove. The final bound O(kappa_d^3 epsilon^{-3}) follows from the definitions of Delta_min and c_1 in the Summary of complexity, and the advertised dimension reduction O~(n^{4.5} epsilon^{-3}) comes from substituting p=O(log r) and P_max=Theta(sqrt(n/p)) into kappa_d=O(p^3 P_max^3). This substitution is explicitly qualified in Section 3.1 and Remark 2: the condition theta>0 forces alpha^2=O(epsilon/r), so the JLT dimension is actually p=O(epsilon^{-1} r log r), meaning the n^{4.5} improvement holds only in the low-accuracy or low-effective-rank regime. That is an honest scope limitation, not a circularity: the theorem is conditional on theta>0 and the paper flags the incompatibility rather than hiding it. The self-citations to [9], [11], and [31] are used as building blocks and are re-derived or standard, so they are not load-bearing in a circular sense. Numerical results are benchmarked against the external Py-BOBYQA solver on CUTEst problems, providing independent empirical content. No fitted parameter is renamed as a prediction, and no known result is repackaged under new coordinates. Accordingly, no circular step can be exhibited, and the derivation is self-contained apart from standard external facts.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

No new physical or mathematical entities are postulated. The free parameters are algorithmic and probabilistic choices that enter the constant factors and the validity regime of the bound. The axioms are a mix of standard smoothness and trust-region assumptions, the paper-specific well-aligned subspace condition, and known Johnson-Lindenstrauss embedding results. The most fragile item is the well-aligned subspace assumption together with theta > 0, which is also the source of the low-accuracy and low-rank limitation.

free parameters (4)
  • alignment parameter alpha
    User-chosen constant in Definition 3.6 controlling how well the subspace must preserve gradient and curvature information. It is load-bearing because theta > 0 in Eq. (24) forces alpha^2 = O(epsilon / r).
  • subspace dimension p = p = O(log r) in the advertised bound, but theta > 0 requires p = O(r epsilon^{-1} log r)
    Algorithm parameter and a key determinant of the complexity. The O~(n^{4.5}) evaluation bound assumes p = O(log r), which is only consistent with the theory for low accuracy or low effective rank.
  • failure probability delta_S
    Probability that the random subspace is not well-aligned. It must satisfy delta_S < 1/(1+c5), coupling the random embedding quality to the trust-region parameters.
  • trust-region parameters (eta, mu, gamma_inc, gamma_dec, Delta_max) = Defaults given in Remark 3 for RSDFO-Q, but treated as arbitrary constants in the theory
    Standard algorithmic parameters entering the constants c0, c1, c3, c4, c5. They do not affect the scaling but the probability bound requires delta_S < 1/(1+c5) and gamma_inc > gamma_dec^{-1}.
assumptions (6)
  • domain assumption f is bounded below, twice continuously differentiable, grad^2 f is L_H-Lipschitz, and Hessians are uniformly bounded (Assumption 3.2)
    Standard smoothness conditions for second-order trust-region analysis; invoked throughout Section 3.
  • domain assumption Model Hessians are uniformly bounded and computed steps satisfy a Cauchy and negative curvature decrease condition (Assumptions 2.3 and 3.3)
    Standard trust-region conditions used in Lemmas 3.5, 3.10 and 3.11.
  • ad hoc to paper Random subspaces P_k are well-aligned with high probability (Assumption 3.7 and Definition 3.6)
    The central probabilistic model of the paper, specifically designed for the second-order analysis; it is not a standard background axiom.
  • domain assumption theta > 0 in Eq. (24)
    Ensures the subspace lower bound on the second-order criticality measure is positive. It fails for high accuracy or high Hessian rank, and the authors flag this in Remark 2.
  • standard math Johnson-Lindenstrauss transforms preserve norms of O(r) vectors with p = O(alpha^{-2} log r)
    Known embedding result cited to [34] and [31], used in Section 3.1 to derive the subspace dimension and P_max values.
  • domain assumption Fully quadratic subspace models can be built with O(p^2) interpolation points and constants kappa_d = O(p^3 P_max^3 L_H Lambda)
    Cites [12] and [13] for the sampling mechanism and interpolation error bounds; required for Definition 3.1 to be achievable and for the complexity calculation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Randomized Subspace Derivative-Free Optimization with Quadratic Models and Second-Order Convergence." pith.science (2026). https://pith.science/paper/CWNZ3LEW

@misc{pith2026241214431,
  author       = {Pith},
  title        = {Pith review of: Randomized Subspace Derivative-Free Optimization with Quadratic Models and Second-Order Convergence},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CWNZ3LEW}},
  note         = {Machine review of arXiv:2412.14431}
}
read the original abstract

We consider model-based derivative-free optimization (DFO) for large-scale problems, based on iterative minimization in random subspaces. We provide the first worst-case complexity bound for such methods for convergence to approximate second-order critical points, and show that these bounds have significantly improved dimension dependence compared to standard full-space methods, provided low accuracy solutions are desired and/or the problem has low effective rank. We also introduce a practical subspace model-based method suitable for general objective minimization, based on iterative quadratic interpolation in subspaces, and show that it can solve significantly larger problems than state-of-the-art full-space methods, while also having comparable performance on medium-scale problems when allowed to use full-dimension subspaces.

Figures

Figures reproduced from arXiv: 2412.14431 by the authors.

Figure 1
Figure 1. Comparisons for medium-scale problems, n ≈ 100. q = n+1. However, this method is outperformed by all variants of the subspace method RSDFO-Q (including with p = n). We find that medium-dimensional subspaces p ∈ {n/10, n/4} (i.e. p ≈ 100 and p ≈ 250) are the best-performing variants. For comparison, in [PITH_FULL_IMAGE:figures/full_fig_p022_1.png] view at source ↗
Figure 2
Figure 2. Comparisons for large-scale problems, n ≈ 1000. Runtime capped at 12hrs per problem instance for all solvers. effective rank (i.e. low rank Hessians). We then introduce a practical algorithm (RSDFO-Q) for general unconstrained model-based DFO in subspaces based on iterative construction of quadratic models in subspaces. This extends the practical method from [9], which was limited to linear models (and specialized t… view at source ↗
Figure 3
Figure 3. Comparisons for large-scale problems, n ≈ 1000, based on runtime required to solve the problem (rather than objective evaluations). Runtime capped at 12hrs per problem instance for all solvers. Acknowledgement(s) We acknowledge the use of the University of Oxford Advanced Research Computing (ARC) facility8 in carrying out this work. Funding LR was supported by the Australian Research Council Discovery Early Career A… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Introduction to Model-Based Derivative-Free Optimization

    math.OC 2025-10 accept novelty 2.0 of 10

    A graduate-level introduction to interpolation-based derivative-free optimization, consolidating trust-region algorithms, interpolation-model accuracy theory, and worst-case complexity bounds for unconstrained, constr...

Reference graph

Works this paper leans on

36 extracted references · 31 canonical work pages · cited by 1 Pith paper

  1. [1]

    Alarie, C

    S. Alarie, C. Audet, A.E. Gheribi, M. Kokkolaras, and S. Le Digabel,Two decades of black- box optimization applications , EURO Journal on Computational Optimization 9 (2021), 8http://dx.doi.org/10.5281/zenodo.22558 24 p. 100011

    doi:10.5281/zenodo.22558 2021

  2. [2]

    Audet and W

    C. Audet and W. Hare, Derivative-Free and Blackbox Optimization , Springer Series in Operations Research and Financial Engineering, Springer, Cham, Switzerland, 2017

    work page 2017

  3. [3]

    Blanchet, C

    J. Blanchet, C. Cartis, M. Menickelly, and K. Scheinberg, Convergence rate analysis of a stochastic trust-region method via supermartingales , INFORMS Journal on Optimization 1 (2019), pp. 92–119

    work page 2019

  4. [4]

    Cartis, J

    C. Cartis, J. Fiala, B. Marteau, and L. Roberts, Improving the flexibility and robustness of model-based derivative-free optimization solvers , ACM Transactions on Mathematical Software 45 (2019), pp. 1–41

    work page 2019

  5. [5]

    Cartis, J

    C. Cartis, J. Fowkes, and Z. Shao, Randomised subspace methods for non-convex opti- mization, with applications to nonlinear least-squares , arXiv preprint arXiv:2211.09873 (2022)

    arXiv 2022

  6. [6]

    Cartis, X

    C. Cartis, X. Liang, E. Massart, and A. Otemissov, Learning the subspace of varia- tion for global optimization of functions with low effective dimension , arXiv preprint arXiv:2401.17825 (2024)

    arXiv 2024

  7. [7]

    Cartis and A

    C. Cartis and A. Otemissov, A dimensionality reduction technique for unconstrained global optimization of functions with low effective dimensionality , Information and Inference: A Journal of the IMA 11 (2022), pp. 167–201

    work page 2022

  8. [8]

    Cartis and L

    C. Cartis and L. Roberts, A derivative-free Gauss–Newton method , Mathematical Pro- gramming Computation 11 (2019), pp. 631–674

    work page 2019

Show all 36 references
  1. [9]

    Cartis and L

    C. Cartis and L. Roberts, Scalable subspace methods for derivative-free nonlinear least- squares optimization, Mathematical Programming 199 (2023), pp. 461–524

    2023

  2. [10]

    Cartis, L

    C. Cartis, L. Roberts, and O. Sheridan-Methven, Escaping local minima with local derivative-free methods: A numerical investigation , Optimization 71 (2022), pp. 2343– 2373

    2022

  3. [11]

    Cartis, Z

    C. Cartis, Z. Shao, and E. Tansley, Random subspace cubic regularization methods , De- cember (2024)

    2024

  4. [12]

    Y. Chen, W. Hare, and A. Wiebe, Q-fully quadratic modeling and its application in a random subspace derivative-free method , Computational Optimization and Applications (2024)

    2024

  5. [13]

    A.R. Conn, K. Scheinberg, and L.N. Vicente, Introduction to Derivative-Free Optimiza- tion, MPS-SIAM Series on Optimization, Society for Industrial and Applied Mathematics, Philadelphia, 2009

    2009

  6. [14]

    Dolan and J.J

    E.D. Dolan and J.J. Mor´ e,Benchmarking optimization software with performance profiles, Mathematical Programming 91 (2002), pp. 201–213

    2002

  7. [15]

    Dzahini and S.M

    K.J. Dzahini and S.M. Wild, Direct search for stochastic optimization in random subspaces with zeroth-, first-, and second-order convergence and expected complexity, arXiv preprint arXiv:2403.13320 (2024)

    2024 arXiv

  8. [16]

    Dzahini and S.M

    K.J. Dzahini and S.M. Wild, Stochastic trust-region algorithm in random subspaces with convergence and expected complexity analyses, SIAM Journal on Optimization 34 (2024), pp. 2671–2699

    2024

  9. [17]

    Ehrhardt and L

    M.J. Ehrhardt and L. Roberts, Inexact derivative-free optimization for bilevel learning , Journal of Mathematical Imaging and Vision 63 (2021), pp. 580–600

    2021

  10. [18]

    Fowkes, L

    J. Fowkes, L. Roberts, and ´A. B˝ urmen,PyCUTEst: An open source Python package of optimization test problems , Journal of Open Source Software 7 (2022), p. 4377

    2022

  11. [19]

    Garmanjani, Trust-region methods without using derivatives: Worst case complexity and the nonsmooth case , Ph.D

    R. Garmanjani, Trust-region methods without using derivatives: Worst case complexity and the nonsmooth case , Ph.D. diss., Universidade de Coimbra, 2015

    2015

  12. [20]

    Gould, D

    N.I.M. Gould, D. Orban, and P.L. Toint, CUTEst: A constrained and unconstrained test- ing environment with safe threads for mathematical optimization , Computational Opti- mization and Applications 60 (2015), pp. 545–557

    2015

  13. [21]

    Gratton, C.W

    S. Gratton, C.W. Royer, L.N. Vicente, and Z. Zhang, Direct search based on probabilistic descent, SIAM Journal on Optimization 25 (2015), pp. 1515–1541

    2015

  14. [22]

    Kimiaei, A

    M. Kimiaei, A. Neumaier, and P. Faramarzi, New subspace method for unconstrained derivative-free optimization, ACM Transactions on Mathematical Software 49 (2023), pp. 25 1–28

    2023

  15. [23]

    Kozak, S

    D. Kozak, S. Becker, A. Doostan, and L. Tenorio, A stochastic subspace approach to gradient-free optimization in high dimensions , Computational Optimization and Applica- tions 79 (2021), pp. 339–368

    2021

  16. [24]

    Larson, M

    J. Larson, M. Menickelly, and S.M. Wild, Derivative-free optimization methods , Acta Numerica 28 (2019), pp. 287–404

    2019

  17. [25]

    Malladi, T

    S. Malladi, T. Gao, E. Nichani, A. Damian, J.D. Lee, D. Chen, and S. Arora, Fine-Tuning Language Models with Just Forward Passes , in 37th Conference on Neural Information Processing Systems (NeurIPS 2023) . 2023

    2023

  18. [26]

    Menickelly, Avoiding geometry improvement in derivative-free model-based methods via randomization, arXiv preprint arXiv:2305.17336 (2023)

    M. Menickelly, Avoiding geometry improvement in derivative-free model-based methods via randomization, arXiv preprint arXiv:2305.17336 (2023)

    2023 arXiv

  19. [27]

    Mor´ e and S.M

    J.J. Mor´ e and S.M. Wild, Benchmarking derivative-free optimization algorithms , SIAM Journal on Optimization 20 (2009), pp. 172–191

    2009

  20. [28]

    Powell, Least Frobenius norm updating of quadratic models that satisfy interpola- tion conditions, Mathematical Programming 100 (2004), pp

    M.J.D. Powell, Least Frobenius norm updating of quadratic models that satisfy interpola- tion conditions, Mathematical Programming 100 (2004), pp. 183–215

    2004

  21. [29]

    Powell, The BOBYQA algorithm for bound constrained optimization without derivatives, Tech

    M.J.D. Powell, The BOBYQA algorithm for bound constrained optimization without derivatives, Tech. Rep. DAMTP 2009/NA06, University of Cambridge, 2009

    2009

  22. [30]

    Roberts and C.W

    L. Roberts and C.W. Royer, Direct search based on probabilistic descent in reduced spaces, SIAM Journal on Optimization 33 (2023)

    2023

  23. [31]

    Shao, On random embeddings and their application to optimisation , Ph.D

    Z. Shao, On random embeddings and their application to optimisation , Ph.D. diss., Uni- versity of Oxford, 2021

    2021

  24. [32]

    G. Ughi, V. Abrol, and J. Tanner, An empirical study of derivative-free-optimization algorithms for targeted black-box attacks in deep neural networks , Optimization and En- gineering 23 (2022), pp. 1319–1346

    2022

  25. [33]

    van de Berg, N

    D. van de Berg, N. Shan, and A. del Rio-Chanona, High-dimensional derivative-free opti- mization via trust region surrogates in linear subspaces , in Proceedings of the 34th Euro- pean Symposium on Computer Aided Process Engineering / 15th International Symposium on Process Sy...

    2024

  26. [34]

    Woodruff, Sketching as a tool for numerical linear algebra, Foundations and Trends® in Theoretical Computer Science 10 (2014), pp

    D.P. Woodruff, Sketching as a tool for numerical linear algebra, Foundations and Trends® in Theoretical Computer Science 10 (2014), pp. 1–157

    2014

  27. [35]

    Xie and Y.x

    P. Xie and Y.x. Yuan, A new two-dimensional model-based subspace method for large-scale unconstrained derivative-free optimization: 2D-MoSub , arXiv preprint arXiv:2309.14855 (2024)

    2024 arXiv

  28. [36]

    Zhang, On derivative-free optimization methods , Ph.D

    Z. Zhang, On derivative-free optimization methods , Ph.D. diss., Chinese Academy of Sci- ences, 2012. 26

    2012

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.